Key frame determination method and device, electronic equipment, and storage medium

Abstract

The disclosure provides a key frame determination method and device, electronic equipment and a storage medium. The method comprises: acquiring a plurality of image frames corresponding to an application interface of a target application during running of the target application on a terminal, and extracting first performance data of each image frame, wherein the first performance data is generated by a plurality of performance counters of a graphics processing device of the terminal when the graphics processing device renders the image frame; performing N times of clustering on the first performance data of the plurality of image frames to obtain N first clustering results of the first performance data and evaluation indexes of the N first clustering results, wherein the number of categories in each clustering in the N times of clustering is different; determining a target clustering result from the N first clustering results according to the evaluation indexes of the N first clustering results; and determining a target key frame in the plurality of image frames according to a clustering category in the target clustering result. According to the embodiments of the disclosure, the key frame of the application can be selected more efficiently and more accurately.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a key frame determination method and device, electronic equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] When an application (APP) runs on a terminal (such as a mobile phone, a tablet computer, etc.), a graphics processing device in the terminal, such as a display card or a GPU (Graphics Processing Unit), needs to render the interface (including various icons, texts, images, animations, etc.) of the application to display on the screen of the terminal, and the rendering process needs close cooperation between the application and the graphics processing device.

[0003] In actual application scenarios, in order to improve the performance and user experience of the application, it is often necessary to select image frames corresponding to the application interface for analysis to achieve optimization. However, if a part of the image frames are blindly selected for analysis, the entire application cannot be comprehensively optimized; if a large number of image frames are selected for analysis, a large amount of resources will be consumed, and the optimization efficiency will be reduced. Therefore, a small number of image frames that can represent the application need to be reasonably selected, so that relevant personnel can optimize these image frames to achieve the effect of optimizing the entire application. SUMMARY

[0004] The present disclosure provides a key frame determination method and device, electronic equipment, computer readable storage medium and computer program product.

[0005] In a first aspect, the present disclosure provides a key frame determination method, which comprises: acquiring a plurality of image frames corresponding to an application interface of a target application during running of the target application on a terminal, and extracting first performance data of each image frame, wherein the first performance data is generated by a plurality of performance counters of a graphics processing device of the terminal when the graphics processing device renders the image frame; performing N times of clustering on the first performance data of the plurality of image frames to obtain N first clustering results of the first performance data and evaluation indexes of the N first clustering results, N is an integer greater than 1, and the number of categories in each clustering of the N times of clustering is different; determining a target clustering result from the N first clustering results according to the evaluation indexes of the N first clustering results; and determining a target key frame in the plurality of image frames according to a clustering category in the target clustering result.

[0006] In a second aspect, the present disclosure provides a key frame determination apparatus, which comprises: a data extraction module configured to obtain a plurality of image frames corresponding to an application interface of a target application during running of the target application on a terminal, and extract first performance data of each image frame, wherein the first performance data is generated by a plurality of performance counters of a graphics processing device of the terminal when the graphics processing device renders the image frame; a clustering module configured to perform N times of clustering on the first performance data of the plurality of image frames to obtain N first clustering results of the first performance data and evaluation indexes of the N first clustering results, N being an integer greater than 1, and the number of categories in each time of clustering being different; a result determination module configured to determine a target clustering result from the N first clustering results according to the evaluation indexes of the N first clustering results; and a key frame determination module configured to determine a target key frame in the plurality of image frames according to a clustering category in the target clustering result.

[0007] In a third aspect, the present disclosure provides an electronic device, which comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the key frame determination method described above.

[0008] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the key frame determination method described above.

[0009] In a fifth aspect, the present disclosure provides a computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, when the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device performs the key frame determination method described above.

[0010] The embodiments provided by the present disclosure can extract performance data corresponding to a plurality of image frames of an application and generated by performance counters of a graphics processing device, perform clustering on the performance data and select an optimal target clustering result, and determine a key frame in the plurality of image frames according to a clustering category in the target clustering result, so as to more efficiently and accurately select a key frame of the application, to enable relevant personnel to optimize a small number of key frames to achieve the effect of optimizing the entire application, improve optimization efficiency, and reduce resources consumed by application optimization.

[0011] It should be understood that nothing in this section is intended to limit the scope of the embodiments of the present disclosure. Other aspects of the present disclosure will become apparent to those skilled in the art upon reading the following specification and appended claims. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and serve to explain the principles of the present disclosure, and are not intended to limit the disclosure. The above and other features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings, in which:

[0013] Figure 1 A flow chart of a key frame determination method provided by an embodiment of the present disclosure.

[0014] Figure 2a 、 Figure 2b 、 Figure 2c 、 Figure 2d A schematic diagram of a visualized image in a key frame determination method provided by an embodiment of the present disclosure.

[0015] Figure 3 A block diagram of a key frame determination apparatus provided by an embodiment of the present disclosure.

[0016] Figure 4 A block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] In order to make the skilled in the art better understand the technical solutions of the present disclosure, the following describes exemplary embodiments of the present disclosure in conjunction with the drawings, including various details of the embodiments of the present disclosure to help understanding, which should be considered only as exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of clarity and conciseness, the description below omits the description of well-known functions and structures.

[0018] The embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0019] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0020] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. "Coupled" or "connected" or similar terms are not restricted to physical or mechanical connections or associations, but can also include electrical connections, whether direct or indirect.

[0021] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an overly literal or overly formal sense unless expressly so defined herein.

[0022] As mentioned before, in order to improve the performance and user experience of an application, it is often necessary to select image frames corresponding to the interface of the application for analysis to achieve optimization. This process can be either the development party of the application actively optimizing for various graphics processing devices (such as certain brands of graphics cards), or the development party of the graphics processing device actively optimizing for various applications (such as game applications, benchmark applications). Among them, the benchmark application refers to an application that quantitatively evaluates the performance of hardware (such as CPU, graphics card) or software (such as algorithm) through a specific test program.

[0023] In the related art, a small number of image frames that can represent the application cannot be reasonably selected, resulting in low optimization efficiency.

[0024] According to embodiments of the present disclosure, a key frame determination method is provided, which can extract performance data corresponding to a plurality of image frames of an application generated by a performance counter of a graphics processing device; cluster the performance data and select an optimal target clustering result; and then determine a key frame in the plurality of image frames according to a clustering category in the target clustering result, which can represent the entire application and / or indicate possible abnormalities. In this way, the key frame of the application can be more efficiently and accurately selected, so that relevant personnel can optimize a small number of key frames to achieve the effect of optimizing the entire application, thereby improving the optimization efficiency and greatly reducing the resources consumed by application optimization.

[0025] The key frame determination method according to the embodiments of the present disclosure can be executed by an electronic device such as a terminal device or a server, and the terminal device can be a vehicle-mounted device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor invoking computer readable program instructions stored in a memory. Alternatively, the method can be executed by a server.

[0026] Figure 1 A flowchart of a key frame determination method according to an embodiment of the present disclosure is provided. Referring to FIG. 1, Figure 1 The method includes:

[0027] In step S11, a plurality of image frames corresponding to an application interface of a target application during running of the target application on a terminal are obtained, and first performance data of each image frame is extracted, wherein the first performance data is generated by a plurality of performance counters of a graphics processing device of the terminal when the graphics processing device renders the image frame.

[0028] In step S12, the first performance data of the plurality of image frames is clustered N times to obtain N first clustering results of the first performance data and evaluation indexes of the N first clustering results, N being an integer greater than 1, and the number of categories in each clustering of the N times of clustering is different.

[0029] In step S13, a target clustering result is determined from the N first clustering results according to the evaluation indexes of the N first clustering results.

[0030] In step S14, a target key frame in the plurality of image frames is determined according to a clustering category in the target clustering result.

[0031] For example, the target application can be any application installed on the terminal, such as a video application, a game application, etc., and the terminal can be any type of terminal device, such as a smartphone, a personal computer, etc. The terminal is provided with a graphics processing device, such as a graphics card, a GPU, etc., for performing graphics-related processing. The graphics processing device is provided with a plurality of performance counters (perf counter) for detecting the running state of each aspect of the graphics processing device to obtain corresponding performance indexes, such as the active state of the computing unit, the floating point operation throughput, the memory bandwidth utilization, etc., and the number of performance counters can be several thousand or more.

[0032] In some possible implementation manners, when the target application runs on the terminal, the interface (including various icons, texts, images, animations, and other elements) of the application needs to be rendered by the graphic processing device in the terminal to display the application interface on the screen of the terminal. In this way, during the running of the target application on the terminal, a plurality of image frames corresponding to the application interface are generated.

[0033] In some possible implementation manners, in step S11, the plurality of image frames corresponding to the application interface during the running of the target application on the terminal can be obtained. All the image frames in the process from the start of the display to the end of the display of the target application can be captured by a capture tool as the plurality of image frames to be processed, which can be hundreds to tens of thousands of image frames. The capture tool can be customized, for example, RenderDoc (an open-source graphics debugging tool) is used as the capture tool, and the present disclosure does not make any limitation in this regard.

[0034] In some possible implementation manners, the plurality of image frames can be played back and dumped from the graphic processing device, and the performance counter data corresponding to each image frame is obtained as the first performance data. The first performance data can be selected from the data of all the performance counters, or from the data of a part of important performance counters; and the dimension of the first performance data corresponds to the number of the selected performance counters, that is, the data of each performance counter corresponds to one or more dimensions of the first performance data. It should be understood that the number of the selected performance counters, the correspondence between the number of the performance counters and the dimension of the first performance data, and the order of the data of each performance counter in the first performance data can be set by the person skilled in the art according to the actual situation, and the present disclosure does not make any limitation in this regard.

[0035] Since the performance counters in different graphic processing devices are different, and the internal structure of the performance counter data is different, a suitable unsupervised clustering algorithm can have a better evaluation index. In some possible implementation manners, the corresponding clustering algorithm can be set according to the data scale and data structure of the first performance data, for example, KMeans clustering, spectral clustering, and the like; and the number of categories corresponding to N times of clustering is set, for example, the number of categories is traversed from 1 to N, that is, the number of categories corresponding to N times of clustering is 1 category, 2 categories, …, N categories, where N is an integer greater than 1, for example, 10. The present disclosure does not make any limitation on the specific type of the clustering algorithm, the number of times of clustering, and the number of categories corresponding to each time of clustering.

[0036] In some possible implementation manners, in step S12, the first performance data of the plurality of image frames can be clustered N times, and the number of categories in each clustering of the N times of clustering is different. After the N times of clustering, N first clustering results of the first performance data can be obtained, and each first clustering result includes information such as first performance data corresponding to each clustering category in the first clustering result and a clustering center of the clustering category.

[0037] In some possible implementation manners, an evaluation index, for example, a silhouette score, of the N first clustering results can be calculated respectively, the evaluation index is used to evaluate the compactness (similarity of samples in a cluster) and separation degree (difference of samples between clusters) of clustering, and the higher the evaluation index is, the better the corresponding clustering result is. The specific type of the evaluation index used in the present disclosure is not limited.

[0038] In some possible implementation manners, in step S13, the optimal target clustering result can be determined from the N first clustering results according to the evaluation index of the N first clustering results. The first clustering result with the highest evaluation index can be directly selected as the target clustering result, or a part of the first clustering results with the highest evaluation index can be selected as candidates, and then the target clustering result is selected from the candidate clustering results, so as to improve the accuracy of selection of the target clustering result and reduce the processing amount of subsequent processing.

[0039] In some possible implementation manners, for the M second clustering results, the average density or confidence of each second clustering result can be directly calculated, so that the target clustering result is selected based on the average density or confidence, or the second clustering result is visualized, and the target clustering result is selected based on the visualized image; or the first performance data can be dimensionally reduced to obtain second performance data, and then the second clustering result is visualized based on the second performance data, and the target clustering result is selected based on the visualized image; or the average density or confidence of the second clustering result can be calculated based on the second performance data, so that the target clustering result is selected based on the average density or confidence. The specific selection manner of the target clustering result is not limited in the present disclosure.

[0040] In some possible implementation manners, after the target clustering result is obtained, in step S14, a target key frame in the plurality of image frames can be determined according to the clustering categories in the target clustering result. If the target key frame includes a representative frame used to represent the entire application, the image frame corresponding to the clustering center of each clustering category in the target clustering result can be used as the target key frame; if the target key frame includes an abnormal frame used to indicate that there may be an anomaly, the image frame corresponding to the performance data deviating from the cluster in each clustering category can be used as the target key frame. In this way, the entire determination process of the target key frame is implemented.

[0041] According to an embodiment of the present disclosure, performance data corresponding to a plurality of image frames of an application and generated by a performance counter of a graphics processing device can be extracted; the performance data can be clustered and an optimal target clustering result can be selected; and then a key frame in the plurality of image frames can be determined according to a clustering category in the target clustering result, so that the key frame of the application can be more efficiently and accurately selected, a person concerned can optimize a small number of key frames to achieve the effect of optimizing the entire application, the optimization efficiency is improved, and the resources consumed by application optimization are reduced.

[0042] The key frame determination method according to an embodiment of the present disclosure is described below.

[0043] As described above, all image frames in a process from starting display to ending display of the target application can be captured by a capture tool in step S11 as the plurality of image frames to be processed; and performance counter data corresponding to each image frame can be dumped from the graphics processing device to obtain the first performance data of the plurality of image frames.

[0044] In step S12, the first performance data of the plurality of image frames is clustered N times according to a preset clustering algorithm and a category number corresponding to N times of clustering, to obtain N first clustering results of the first performance data, each first clustering result including first performance data corresponding to each clustering category in the first clustering result, a clustering center of the clustering category, and other information. Moreover, evaluation indexes of the N first clustering results can be calculated respectively.

[0045] In some possible implementation manners, the importance of data generated by the performance counter in the graphics processing device is different, and the importance of part of the performance counter data is higher. For example, the performance counter data “sm__cycles_active” is used to detect the proportion of SM (Streaming Multiprocessor) active period, and if the value of the data is lower than 80%, a memory / instruction bottleneck can be prompted; the performance counter data “sm__cycles_active” is used to detect the branch instruction execution efficiency, and greater than 95% is optimal, and if the value of the data is lower, it can be caused by thread bundle serialization in the processing process. In this case, a higher weight can be set for the performance counter data with higher importance, and weighted clustering can be performed to improve the clustering effect.

[0046] In some possible implementation manners, the plurality of performance counters of the graphics processing device include a first category and a second category, a first weight of the performance counters of the first category is greater than a second weight of the performance counters of the second category. That is, the first category corresponds to performance counters with higher importance, the second category corresponds to performance counters with lower importance, the first weight of the performance counters of the first category is greater, the second weight of the performance counters of the second category is smaller, and the first weight is greater than the second weight. For example, the first weight is set as 2 and the second weight is set as 1, and the disclosure does not limit the specific values of the first weight and the second weight.

[0047] In some possible implementation manners, step S12 can include: weighting data of the plurality of performance counters in the first performance data according to the first weight of the performance counters of the first category and the second weight of the performance counters of the second category, to obtain third performance data after weighting; and performing N times of clustering on the third performance data, to obtain the N first clustering results and evaluation indexes of the N first clustering results.

[0048] That is, data of the performance counters of the first category is multiplied by the corresponding first weight, and data of the performance counters of the second category is multiplied by the corresponding second weight, to realize weighting of the data; then, the weighted data is spliced to obtain the third performance data after weighting; and then, according to a preset clustering algorithm and a category number corresponding to N times of clustering, N times of clustering is performed on the third performance data of the plurality of image frames, to obtain N first clustering results.

[0049] In this way, the weight of the performance counter with higher importance in the clustering process can be improved, the clustering result is more accurate, and therefore the key frame selected subsequently can better represent the performance of the application, and the efficiency of subsequent optimization is improved.

[0050] In some possible implementation manners, in step S13, an optimal target clustering result is determined from the N first clustering results according to the evaluation indexes of the N first clustering results. The first clustering result with the highest evaluation index can be directly selected as the target clustering result, or a part of the clustering results can be selected as candidates, and then the target clustering result is selected from the candidate clustering results, so that the accuracy of selection of the target clustering result is improved, and the processing amount of subsequent processing is reduced.

[0051] In some possible implementation manners, step S13 can include: determining M second clustering results from the N first clustering results according to the evaluation indexes of the N first clustering results, M is an integer greater than 1 and M < N; and determining a target clustering result from the M second clustering results.

[0052] That is, a value of M smaller than N can be set, and the M clustering results with the highest evaluation indexes (Top M) among the N first clustering results can be taken as candidate second clustering results. For example, N = 10 and M = 5, and the first clustering results with the Top 5 evaluation indexes can be selected from the 10 first clustering results as the second clustering results.

[0053] In this way, preliminary screening can be implemented, so that the clustering results are subsequently evaluated from other dimensions, thereby reducing the number of clustering results for subsequent processing and reducing the data processing amount; and the possibility of missing the optimal clustering result can be reduced (possibly, the evaluation index of a clustering result is not the highest, but the subsequent analysis is the optimal target clustering result).

[0054] After preliminary screening, the target clustering result can be directly selected based on the average density or confidence of the clustering result for the M candidate second clustering results; or the first performance data can be dimensionally reduced, and the target clustering result can be selected based on the reduced performance data.

[0055] In some possible implementation manners, the step of determining the target clustering result from the M second clustering results in step S13 includes: for any second clustering result, determining a local density value of each clustering category of the second clustering result according to the first performance data of each clustering category in the second clustering result; determining an average density value of the second clustering result according to the local density value of each clustering category of the second clustering result; and determining the second clustering result with the maximum average density value in the M second clustering results as the target clustering result.

[0056] For example, for any second clustering result in the M second clustering results, a K-nearest neighbor algorithm or other processing method can be used to calculate the local density value of the first performance data of each clustering category in the second clustering result. For any data point x in a clustering category, the local density of the data point x can be represented as:

[0057] p(x) = K / V K (1)

[0058] In formula (1), p(x) represents the local density of the data point x, K represents the number of neighbor data points of the data point x, and V K represents the spatial volume defined by the K neighbor data points, such as the volume of the minimum enclosing sphere of the K neighbor data points.

[0059] In some possible implementation manners, after the local density of each data point x in a clustering category is obtained, the average value of the local densities of all data points x in the clustering category can be calculated as the local density value of the clustering category; and then, the average value of the local density values of all clustering categories in the second clustering result can be calculated as the average density value of the second clustering result.

[0060] In some possible implementation manners, the greater the average density value is, the more reasonable the data distribution in the clustering result is, and the better the clustering effect is. In this case, the second clustering result with the greatest average density value in the M second clustering results can be determined as the target clustering result.

[0061] In this way, the clustering result with the greatest average density value can be screened out, thereby improving the accuracy of selecting the optimal clustering result.

[0062] In some possible implementation manners, the performance counter of the graphics processing device includes a total cycle counter “total_cycle” configured to record the total number of clock cycles of the GPU core when performing the computing task. Each clock cycle is the minimum time unit of the basic operation (such as arithmetic operation and memory access) performed by the GPU, and therefore the “total_cycle” directly reflects the cumulative working time of the GPU computing resource.

[0063] In some possible implementation manners, the confidence of the clustering result can be calculated by using the data (hereinafter referred to as first sub-data) of the total cycle counter in the first performance data, and the target clustering result can be selected according to the confidence.

[0064] In some possible implementation manners, the step of determining the target clustering result from the M second clustering results in step S13 includes: for any second clustering result, determining the standard deviation of each clustering category in the second clustering result according to the first sub-data of the first performance data of the clustering category; determining the confidence interval of the clustering category according to the first sub-data corresponding to the clustering center of the clustering category and the standard deviation of the clustering category; determining the category confidence of the clustering category according to the data proportion of the first sub-data in the first sub-data of the clustering category that is within the confidence interval; determining the result confidence of the second clustering result according to the category confidence of each clustering category in the second clustering result; and determining the target clustering result from the M second clustering results according to the result confidence of the M second clustering results.

[0065] For example, for any one of the M second clustering results, the standard deviation of each clustering category can be calculated according to the first sub-data of all data points of the clustering category in the second clustering result, and the specific calculation manner of the standard deviation is not limited in the present disclosure.

[0066] In some possible implementation manners, for any cluster category, a confidence interval of the cluster category can be determined according to the cluster center corresponding to the first sub-data of the cluster category and the standard deviation of the cluster category, for example, taking the first sub-data corresponding to the cluster center as a predicted value, and taking 1.96 times the standard deviation added or subtracted from the predicted value as the confidence interval of the cluster category. The present disclosure does not limit the specific parameters of the confidence interval. If most of the data of the cluster category is within the confidence interval, it indicates that the cluster category in the clustering result is reasonable.

[0067] In some possible implementation manners, a category confidence of the cluster category can be determined according to a data proportion of the first sub-data within the confidence interval in the first sub-data of the cluster category. The data proportion is positively correlated with the category confidence, that is, the higher the data proportion of the first sub-data within the confidence interval, the higher the category confidence. The data proportion can be directly set as the category confidence, or the data proportion can be converted into the category confidence by other calculation manners, which are not limited in the present disclosure.

[0068] In some possible implementation manners, an average value of the category confidence of each cluster category in the second clustering result can be determined as a result confidence of the second clustering result, or the result confidence of the second clustering result can be determined based on the category confidence of each cluster category in the second clustering result by other calculation manners. The greater the result confidence, the more reasonable the data distribution in the second clustering result, and the better the clustering effect. In this case, the second clustering result with the maximum result confidence in the M second clustering results can be determined as the target clustering result.

[0069] In this way, the clustering result with the maximum result confidence can be screened out, thereby improving the accuracy of selecting the optimal clustering result.

[0070] In some possible implementation manners, the step of determining the target clustering result from the M second clustering results in step S13 includes: for any second clustering result, generating a visual image corresponding to the second clustering result according to the data of the first three dimensions in the first performance data corresponding to the second clustering result; and determining the target clustering result from the M second clustering results according to the position of the cluster center of the cluster category of the second clustering result in the visual image.

[0071] For example, for any of the M second clustering results, a corresponding three-dimensional visual image can be generated according to the data of the first three dimensions in the first performance data corresponding to the second clustering result, and the three-dimensional coordinate axes in the visual image correspond to the first three dimensions in the first performance data respectively. In this way, each data point in the second clustering result corresponds to a three-dimensional position on the visual image.

[0072] In some possible implementation manners, the clustering effect of the second clustering result can be determined according to positions of clustering centers of each clustering category of the second clustering result in the visual image. For example, if the clustering centers of each clustering category are relatively dispersed, that is, distances between the clustering centers all exceed a distance threshold, and a difference between distances between the clustering centers is less than a difference threshold, the visual effect of the visual image of the second clustering result is better, and the clustering effect of the second clustering result is better. In this way, according to the visual images corresponding to the M second clustering results, the optimal target clustering result can be selected from the M second clustering results.

[0073] In some possible implementation manners, a maximum key frame number for the target application can also be set. If resources for subsequent optimization are limited, the maximum key frame number is set to be smaller, for example, 3 frames; if resources for subsequent optimization are more, the maximum key frame number is set to be larger, for example, 10 frames. If there are visual images with the same or similar visual effects in the visual images corresponding to the second clustering results, the second clustering result with a number of clustering categories not exceeding the maximum key frame number and closest to the maximum key frame number can be selected as the optimal target clustering result according to the maximum key frame number.

[0074] Through the visual processing manner, the clustering effect of each clustering result can be more intuitively determined through the visual image, and therefore the efficiency and accuracy of selection of the target clustering result are improved.

[0075] In some possible implementation manners, the first performance data can also be reduced in dimension, and the target clustering result is selected based on the performance data after dimension reduction, so as to improve the accuracy of subsequent visual selection and reduce the amount of data for subsequent processing.

[0076] In some possible implementation manners, the key frame determination method according to the embodiments of the present disclosure can further include:

[0077] The first performance data is reduced in dimension to obtain second performance data, a dimension of the second performance data is less than a dimension of the first performance data, and a ratio between a total value of the first three dimensions of the second performance data and a total value of all dimensions of the second performance data is greater than or equal to a proportion threshold;

[0078] The second performance data is used to replace the first performance data, and the step of determining the target clustering result from the M second clustering results is performed.

[0079] For example, according to the type and number of performance counters in the graphics processing device, a suitable dimension reduction algorithm can be selected to reduce the first performance data corresponding to each image frame to obtain second performance data. The dimension reduction algorithm may, for example, use a PAC (Probably Approximately Correct) algorithm. The dimension of the second performance data is less than that of the first performance data. The dimension of the first performance data is several hundred to several thousand. The dimension of the second performance data can be set to within 10 dimensions, for example, the dimension of the first performance data is 500, and the dimension of the second performance data is 8. The specific type of the dimension reduction algorithm and the specific dimension of the second performance data are not limited in the present disclosure.

[0080] In some possible implementation manners, the data proportion of the first three dimensions of the second performance data is the proportion of the total value of the data of the first three dimensions of the second performance data in the total value of the data of all dimensions of the second performance data, that is, the ratio of the sum of the data of the first three dimensions to the sum of the data of all dimensions. The greater the data proportion, the more the data of the first three dimensions can represent the entire second performance data, and the more reliable the subsequent visualized result.

[0081] In some possible implementation manners, the data proportion of the first three dimensions of the second performance data can be greater than or equal to a proportion threshold. The proportion threshold may, for example, be 0.8, and the specific value of the proportion threshold is not limited in the present disclosure.

[0082] In some possible implementation manners, the original first performance data can be replaced by the second performance data after dimension reduction to perform the step of selecting the target clustering result.

[0083] In the case of selecting the target clustering result based on the visualized image, the step of determining the target clustering result from the M second clustering results in step S13 can include: for any second clustering result, generating a visualized image corresponding to the second clustering result according to the data of the first three dimensions in the second performance data corresponding to the second clustering result; and determining the target clustering result from the M second clustering results according to the position of the clustering center of the clustering category of the second clustering result in the visualized image.

[0084] For example, for any of the M second clustering results, a corresponding three-dimensional visualized image can be generated according to the data of the first three dimensions in the second performance data corresponding to the second clustering result. The three-dimensional coordinate axes in the visualized image correspond to the first three dimensions in the second performance data, respectively. In this way, each data point in the second clustering result corresponds to a three-dimensional position on the visualized image.

[0085] In some possible implementation manners, the clustering effect of the second clustering result can be determined according to positions of clustering centers of each clustering category of the second clustering result in the visual image. For example, if the clustering centers of each clustering category are relatively dispersed, that is, distances between the clustering centers are all greater than a distance threshold, and a difference between distances between the clustering centers is less than a difference threshold, the visual effect of the visual image of the second clustering result is better, and the clustering effect of the second clustering result is better. In this way, according to the visual images corresponding to the M second clustering results, the optimal target clustering result can be selected from the M second clustering results.

[0086] In some possible implementation manners, a maximum number of key frames for the target application can also be set. If resources for subsequent optimization are limited, the maximum number of key frames is set to be smaller, for example, 3 frames; if resources for subsequent optimization are more, the maximum number of key frames is set to be larger, for example, 10 frames. If there are visual images with the same or similar visual effects in the visual images corresponding to the second clustering results, the second clustering result with a number of clustering categories not greater than the maximum number of key frames and closest to the maximum number of key frames can be selected as the optimal target clustering result according to the maximum number of key frames.

[0087] Through the visual processing manner, the clustering effect of each clustering result can be more intuitively determined through the visual image, and therefore the efficiency and accuracy of selection of the target clustering result are improved.

[0088] In some possible implementation manners, the average density can be calculated based on the performance data after dimension reduction, and then the target clustering result is selected, so that the efficiency of calculation is improved.

[0089] In the case of selecting the target clustering result based on the average density, the step of determining the target clustering result from the M second clustering results in step S13 includes: for any second clustering result, determining a local density value of each clustering category of the second clustering result according to second performance data of each clustering category of the second clustering result; determining an average density value of the second clustering result according to the local density value of each clustering category of the second clustering result; and determining the target clustering result from the M second clustering results according to the average density values of the M second clustering results.

[0090] That is, for any one of the M second clustering results, the local density value of the second performance data of each clustering category in the second clustering result can be calculated by using a K-nearest neighbor algorithm or other processing method. The specific calculation manner is similar to the foregoing, and details are not described herein again.

[0091] In some possible implementation manners, an average value of the local density values of all clustering categories in the second clustering result can be calculated as an average density value of the second clustering result. The greater the average density value is, the more reasonable the data distribution in the clustering result is, and the better the clustering effect is. In this case, a second clustering result with the greatest average density value among the M second clustering results can be determined as the target clustering result.

[0092] In this way, the clustering result with the greatest average density value can be screened out, so as to improve the accuracy of selecting the optimal clustering result, and the calculation amount of the processing manner is small, further improving the calculation efficiency.

[0093] After obtaining the target clustering result in step S13, a target key frame in the plurality of image frames can be determined according to the clustering categories in the target clustering result in step S14.

[0094] In some possible implementation manners, the target key frame includes a representative frame of the target application, and the representative frame is used to indicate the image rendering performance of the graphics processing device for the target application. In this case, step S14 can include: determining first performance data corresponding to the clustering center of each clustering category in the target clustering result respectively, and determining an image frame corresponding to the first performance data as the representative frame of the target application.

[0095] For example, each clustering category in the target clustering result can represent a type of performance scenario of the target application, for example, an application has a function of browsing images and texts, a function of playing videos, and a function of live streaming, each function corresponds to a type of performance scenario, the first performance data in the same type of performance scenario is more similar, and is in the same clustering category in the target clustering result, and the clustering center of the clustering category can better represent the average performance in the performance scenario.

[0096] In this case, the first performance data corresponding to the clustering center of each clustering category in the target clustering result can be determined. If the clustering center directly corresponds to the first performance data, the first performance data is directly selected, and an image frame corresponding to the first performance data is determined as the representative frame of the target application. If the clustering center does not directly correspond to the first performance data, for example, the clustering center of some clustering algorithms is a mean value, the first performance data closest to the clustering center is selected, and an image frame corresponding to the first performance data is determined as the representative frame of the target application.

[0097] In this way, the accuracy and efficiency of selecting the representative frame can be improved.

[0098] In some possible implementation manners, the target key frame includes an abnormal frame, which is used to indicate that the graphics processing device renders an image abnormally for the target application. In this case, step S14 can include: for any cluster category in the target clustering result, determining, in the visual image corresponding to the target clustering result, third performance data deviating from the cluster category according to a distance between the second performance data in the cluster category and the cluster center of the cluster category; and determining, according to image frames corresponding to the third performance data in all cluster categories in the target clustering result, the abnormal frame of the target application.

[0099] For example, for any cluster category in the target clustering result, a distance between a data point of the second performance data and a data point of the cluster center can be determined in the corresponding visual image; if there is second performance data whose distance exceeds a preset distance threshold, the second performance data is determined as the performance data deviating from the cluster category, which is referred to as third performance data below. In this case, the image frame corresponding to the third performance data can be considered to deviate from the average performance in the corresponding performance scenario, and can be an abnormal frame of image rendering. The disclosure does not limit the specific setting of the distance threshold.

[0100] In some possible implementation manners, the third performance data in all cluster categories in the target clustering result can be obtained, and image frames corresponding to the third performance data are determined; then, the image frames can be excluded from the following cases: scene conversion and data loading in the target application; and the remaining image frames after the exclusion are used as the abnormal frame of the target application.

[0101] In this way, the abnormal frame of the target application can be determined, so that the abnormal frame is used as a rendering abnormality of the target application for analysis in subsequent processing, for example, a performance bottleneck of the graphics processing device is analyzed from a system level, thereby improving the optimization effect of the application.

[0102] The following gives an application example of the key frame determination method according to the embodiments of the disclosure.

[0103] In the example, a plurality of image frames corresponding to an application interface of an application running on a terminal can be obtained, and performance counter data corresponding to each image frame is obtained to obtain first performance data; the first performance data is clustered multiple times, and a clustering result with a top 2 contour coefficient is filtered out; and the first performance data is dimensionally reduced to obtain second performance data, and a corresponding visual image is generated according to data in the first three dimensions in the second performance data.

[0104] Figure 2a 、 Figure 2b 、 Figure 2c 、 Figure 2dA schematic diagram of a visualization image in a key frame determination method provided by an embodiment of the present disclosure.

[0105] As shown in Figure 2a , each data point in the visualization image corresponds to second performance data of an image frame. The clustering results of the silhouette coefficient Top2 correspond to three and four clustering categories respectively; the silhouette coefficient score when divided into three categories is 0.7235, and the silhouette coefficient score when divided into four categories is 0.7062, and the silhouette coefficient scores of the two are similar.

[0106] Among them, Figure 2b is the visualization image with a clustering category of three categories, and the data points of green, yellow, and blue are data points under the three clustering categories, and the clustering centers of the three clustering categories are 21, 22, and 23 respectively. Figure 2c is the visualization image with a clustering category of four categories, and the data points of green, yellow, blue, and brown are data points under the four clustering categories, and the clustering centers of the four clustering categories are 24, 25, 26, and 27 respectively.

[0107] In an example, the local density value of each clustering category in Figure 2b and Figure 2c can be calculated respectively, and then the average density value of the two clustering results is calculated. The larger the average density value is, the more reasonable the data distribution in the clustering result is, and the better the clustering effect is. In this case, Figure 2c the average density value of the clustering result in Figure 2c is larger, and the clustering result is taken as the optimal target clustering result; and then the image frames corresponding to the clustering centers of the four clustering categories in

[0108] In an example, the distance between the data points in each clustering category in Figure 2c and the corresponding clustering center can also be determined; the data points whose distance exceeds the distance threshold are determined as the data points deviating from the group in the clustering category, as the abnormal data points. As shown in Figure 2c , three abnormal data points 31, 32, and 33 are obtained. Further, after excluding the cases such as scene conversion and loading data, the image frames corresponding to the remaining abnormal data points are taken as abnormal frames.

[0109] In this way, the selection of the representative frames and the abnormal frames of the application can be realized, the efficiency and accuracy of the selection are improved, and the relevant personnel can subsequently optimize a small number of key frames to achieve the effect of optimizing the entire application.

[0110] According to an embodiment of the present disclosure, performance data corresponding to a plurality of image frames of an application and generated by a performance counter of a graphics processing device can be extracted; a plurality of clustering of the performance data is performed and an optimal target clustering result is selected; and a key frame in the plurality of image frames is determined according to a clustering category in the target clustering result, so that the key frame of the application can be selected more efficiently and more accurately, the optimization of the entire application can be achieved by optimizing a small number of key frames, the optimization efficiency is improved, and the resources consumed by the optimization of the application are reduced.

[0111] According to an embodiment of the present disclosure, when clustering, a weight can be assigned to performance counter data according to the importance of the performance counter, weighted clustering of the performance data is implemented, the clustering result is more accurate, and the key frame selected subsequently can better represent the performance of the application. In the process of selecting the clustering result, the number of clustering results for subsequent processing can be reduced, and the data processing amount can be reduced by performing preliminary screening according to the evaluation index; and the performance data can be reduced in dimension, a visual image corresponding to each clustering result is generated based on the first three dimensions of the performance data after dimension reduction, and the optimal target clustering result is selected based on the visual image. Through the processing mode of the visual image, the clustering effect of each clustering result can be more intuitively judged, and the efficiency and accuracy of the selection of the target clustering result are improved.

[0112] According to an embodiment of the present disclosure, according to the clustering category in the target clustering result, a representative frame and an abnormal frame of the application can be determined at the same time, the range of the selected key frame is expanded, the flexibility of the selection of the key frame is improved, and the applicability and application range of the scheme are improved.

[0113] It can be understood that the above-mentioned various method embodiments of the present disclosure can be combined with each other to form combined embodiments without deviating from the principle logic. Limited by the length, the present disclosure will not be repeated. Those skilled in the art can understand that the specific execution order of each step in the above-mentioned method should be determined according to its function and possible internal logic.

[0114] In addition, the present disclosure also provides a key frame determination device, an electronic device, a computer readable storage medium, and a computer program product, which can be used to implement any one of the key frame determination methods provided by the present disclosure. The corresponding technical solutions and descriptions are described in the method part and are not repeated.

[0115] Figure 2d A block diagram of a key frame determination device provided by an embodiment of the present disclosure is shown.

[0116] Referring to Figure 3 , the embodiment of the present disclosure provides a key frame determination device, which comprises:

[0117] The data extraction module 31 is configured to acquire a plurality of image frames corresponding to an application interface of the target application during running of the target application on the terminal, and extract first performance data of each image frame, wherein the first performance data is generated by a plurality of performance counters of a graphic processing device of the terminal when the graphic processing device renders the image frame.

[0118] The clustering module 32 is configured to cluster the first performance data of the plurality of image frames for N times to obtain N first clustering results of the first performance data and evaluation indexes of the N first clustering results, N being an integer greater than 1, and a number of categories in each clustering of the N times of clustering being different.

[0119] The result determination module 33 is configured to determine a target clustering result from the N first clustering results according to the evaluation indexes of the N first clustering results.

[0120] The key frame determination module 34 is configured to determine a target key frame in the plurality of image frames according to a clustering category in the target clustering result.

[0121] In some possible implementation manners, the result determination module 33 is configured to determine M second clustering results with the highest evaluation indexes from the N first clustering results, M being an integer greater than 1 and M < N, and determine the target clustering result from the M second clustering results.

[0122] In some possible implementation manners, the result determination module 33 is configured to, for any second clustering result, generate a visual image corresponding to the second clustering result according to data in the first performance data corresponding to the second clustering result in the first three dimensions, and determine the target clustering result from the M second clustering results according to a position of a clustering center of the clustering category of the second clustering result in the visual image.

[0123] In some possible implementation manners, the result determination module 33 is configured to, for any second clustering result, determine a local density value of each clustering category of the second clustering result according to the first performance data of each clustering category in the second clustering result, determine an average density value of the second clustering result according to the local density value of each clustering category of the second clustering result, and determine a second clustering result with the maximum average density value from the M second clustering results as the target clustering result.

[0124] In some possible implementation manners, the apparatus further includes: a dimension reduction module, configured to reduce the dimension of the first performance data to obtain second performance data, a total value of the first three dimensions of the second performance data is greater than or equal to a proportion threshold value relative to a total value of all dimensions of the second performance data; and a data replacement module, configured to replace the first performance data with the second performance data, and perform the step of determining the target clustering result from the M second clustering results.

[0125] In some possible implementation manners, the performance counter includes a total cycle number counter, the first performance data includes first sub-data obtained by the total cycle number counter, and the result determination module 33 is configured to: for any second clustering result, determine a standard deviation of each clustering category in the second clustering result according to the first sub-data in the first performance data of the clustering category; determine a confidence interval of the clustering category according to the first sub-data corresponding to the clustering center of the clustering category and the standard deviation of the clustering category; determine a category confidence of the clustering category according to a data proportion of the first sub-data in the first sub-data of the clustering category that is within the confidence interval; determine a result confidence of the second clustering result according to the category confidence of each clustering category in the second clustering result; and determine the target clustering result from the M second clustering results according to the result confidence of the M second clustering results.

[0126] In some possible implementation manners, the target key frame includes a representative frame of the target application, the representative frame is used to indicate the image rendering performance of the graphics processing device for the target application, and the key frame determination module 34 is configured to: determine the first performance data corresponding to the clustering center of each clustering category in the target clustering result respectively, and determine an image frame corresponding to the first performance data as the representative frame of the target application.

[0127] In some possible implementation manners, the target key frame includes an abnormal frame, the abnormal frame is used to indicate an image rendering abnormality of the graphics processing device for the target application, and the key frame determination module 34 is configured to: for any clustering category in the target clustering result, determine third performance data deviating from the group in the clustering category according to a distance between the performance data in the clustering category and the clustering center of the clustering category in a visual image corresponding to the target clustering result; and determine the abnormal frame of the target application according to image frames corresponding to the third performance data of all clustering categories in the target clustering result.

[0128] In some possible implementation manners, the plurality of performance counters of the graphics processing device include a first category and a second category, a first weight of the performance counters of the first category is greater than a second weight of the performance counters of the second category; and the clustering module 32 is configured to: weight data of the plurality of performance counters in the first performance data according to the first weight of the performance counters of the first category and the second weight of the performance counters of the second category, to obtain third performance data after weighting; and perform N times of clustering on the third performance data, to obtain the N first clustering results and the evaluation indexes of the N first clustering results.

[0129] Figure 3 A block diagram of an electronic device is provided for the embodiments of the present disclosure.

[0130] With reference to Figure 4 Figure 4 The embodiments of the present disclosure provide an electronic device, which includes: at least one processor 501; at least one memory 502, and one or more I / O interfaces 503 connected between the processor 501 and the memory 502; wherein the memory 502 stores one or more computer programs executable by the at least one processor 501, and the one or more computer programs are executed by the at least one processor 501 to enable the at least one processor 501 to perform the key frame determination method described above.

[0131] The embodiments of the present disclosure also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the key frame determination method described above. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0132] The embodiments of the present disclosure also provide a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the key frame determination method described above.

[0133] Those of ordinary skill in the art will realize and understand that all or some of the steps in the methods disclosed above and the functional modules / units in the systems and devices can be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementation, the division between the functional modules / units referred to in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on computer readable storage media, which can include computer storage media (or non-transitory media) and communication media (or transitory media).

[0134] As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable program instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disc read only memory (CD-ROM), digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. Further, it is well known to those of ordinary skill in the art that communication media typically embodies computer readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. As used herein, the term "modulated data signal" means a signal that has one or more of its characteristics changed or set in a manner so as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as wireless networks, cellular telephone networks, code division multiple access (CDMA) networks, and other terrestrial and satellite radio frequency communication networks or frequency (RF) media. The computer software can be downloaded to the respective computing / processing device from a computer software distribution medium or to an external computer or external storage device from a network, for example, the Internet, a local area network, a wide area network, and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer software from the network and forwards the computer software for storage in a computer readable storage medium within the respective computing / processing device.

[0135] The computer software can be downloaded to the respective computing / processing device from a computer software distribution medium or to an external computer or external storage device from a network, for example, the Internet, a local area network, a wide area network, and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer software from the network and forwards the computer software for storage in a computer readable storage medium within the respective computing / processing device.

[0136] Computer readable program instructions for carrying out operations of the present disclosure can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or any combination of one or more of the above in any combination, written in any combination of one or more programming languages, including object oriented programming languages such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0137] The computer program product described herein can be embodied in a specific manner by hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), and the like.

[0138] The various aspects of the present disclosure are described herein with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer readable program instructions.

[0139] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage cycles that change state. The instructions can be executed by one or more processors of a computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process such that the instructions which execute via the one or more processors of the computer or other programmable data processing devices create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0140] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0141] The flow and block diagrams in the drawings show architectural, functional, and operational aspects of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of instructions which comprise one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may

[0142] Example embodiments have been disclosed and, although a specific terminology is employed, it is merely for the convenience of the reader and is not intended to be limiting. In some instances, specific details have been included for the purpose of providing a thorough understanding of the embodiments presented, but it will be apparent to those skilled in the art that embodiments of the application can be practiced without these specific details. In some instances, features, characteristics and / or elements described in connection with a particular embodiment can be used in conjunction with, or switched to, features, characteristics and / or elements described in connection with other embodiments, unless otherwise explicitly stated. Accordingly, it will be understood that various changes in form and details can be made without departing from the scope of the disclosure as set forth in the appended claims.

Claims

1. A keyframe determination method, characterized in that, The method comprises: obtaining a plurality of image frames corresponding to an application interface during running of a target application on a terminal, and extracting first performance data of each image frame, wherein the first performance data is generated by a plurality of performance counters of a graphics processing device of the terminal when the graphics processing device renders the image frame; performing N times of clustering on the first performance data of the plurality of image frames to obtain N first clustering results of the first performance data and evaluation indexes of the N first clustering results, N being an integer greater than 1, the number of categories in each time of clustering being different, and the number of categories being traversed from 1 to N; determining a target clustering result from the N first clustering results according to the evaluation indexes of the N first clustering results; determining a target key frame from the plurality of image frames according to a clustering category in the target clustering result.

2. The method of claim 1, wherein, The method further comprises: performing dimension reduction on the first performance data to obtain second performance data, the number of dimensions of the second performance data being less than that of the first performance data, and a ratio between a total value of the first three dimensions of the second performance data and a total value of all dimensions of the second performance data being greater than or equal to a proportion threshold value; replacing the first performance data with the second performance data to perform the step of determining the target clustering result from the M second clustering results.

3. The method of claim 2, wherein, The performance counters comprise a total cycle number counter, and the first performance data comprises first sub-data obtained by the total cycle number counter, The method further comprises: for any second clustering result, determining a standard deviation of each clustering category in the second clustering result according to the first sub-data in the first performance data of each clustering category in the second clustering result.

4. The method of claim 2, wherein, ​ ​ ​ ​ 5. The method according to claim 3 or 4, characterized in that, ​ ​ ​ 6. The method of claim 2, wherein, ​ ​ ​ determine a confidence interval of the cluster category according to the first sub-data corresponding to the cluster center of the cluster category and the standard deviation of the cluster category; determine a category confidence degree of the cluster category according to a data proportion of the first sub-data in the first sub-data of the cluster category within the confidence interval; determine a result confidence degree of the second clustering result according to the category confidence degree of each cluster category in the second clustering result; determine a target clustering result from the M second clustering results according to the result confidence degrees of the M second clustering results.

7. The method of claim 1, wherein, The target key frame includes a representative frame of the target application, and the representative frame is used to indicate the image rendering performance of the graphics processing device for the target application, wherein, according to the cluster category in the target clustering result, determining the target key frame in the plurality of image frames, comprising: determining the first performance data corresponding to the cluster center of each cluster category in the target clustering result respectively, and determining the image frame corresponding to the first performance data as the representative frame of the target application.

8. The method of claim 3, wherein, The target key frame includes an abnormal frame, and the abnormal frame is used to indicate the image rendering exception of the graphics processing device for the target application, wherein, according to the cluster category in the target clustering result, determining the target key frame in the plurality of image frames, comprising: for any cluster category in the target clustering result, in the visualization image corresponding to the target clustering result, according to the distance between the performance data in the cluster category and the cluster center of the cluster category, determining the third performance data of the cluster category deviating from the group; determining the abnormal frame of the target application according to the image frame corresponding to the third performance data of all cluster categories in the target clustering result.

9. The method of claim 1, wherein, The plurality of performance counters of the graphics processing device include a first category and a second category, and the first weight of the performance counter of the first category is greater than the second weight of the performance counter of the second category; wherein, the N times clustering of the first performance data of the plurality of image frames to obtain the N first clustering results of the first performance data and the evaluation index of the N first clustering results, comprising: weighting the data of the plurality of performance counters in the first performance data according to the first weight of the performance counter of the first category and the second weight of the performance counter of the second category to obtain the third performance data after weighting; performing N times clustering on the third performance data to obtain the N first clustering results and the evaluation index of the N first clustering results.

10. An apparatus for determining key frames, the apparatus comprising: comprising: a data extraction module, configured to acquire a plurality of image frames corresponding to an application interface of a target application during running of the target application on a terminal, and extract first performance data of each image frame, wherein the first performance data is generated by a plurality of performance counters of a graphics processing device of the terminal when the graphics processing device renders the image frame; The clustering module is configured to perform N times of clustering on the first performance data of the plurality of image frames to obtain N first clustering results of the first performance data and evaluation indexes of the N first clustering results, where N is an integer greater than 1, the number of categories in each of the N times of clustering is different, and the number of categories is traversed from 1 to N; The result determination module is configured to determine a target clustering result from the N first clustering results according to the evaluation indexes of the N first clustering results. The key frame determination module is configured to determine a target key frame in the plurality of image frames according to a clustering category in the target clustering result.

11. An electronic device, comprising: comprise: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the key frame determination method according to any one of claims 1-9.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the key frame determination method according to any one of claims 1-9.

Images