A system for tuning application performance based on platform-level benchmarking
By establishing performance benchmarks and comparing performance data of different applications, identifying performance abnormalities of applications on the computing platform, and implementing solutions, the problem of application performance degradation is solved, and the application execution efficiency and user experience is improved.
Patent Information
- Application Number
- CN201880092343.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-05-07
- Filing Date
- 2018-06-20
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2038-06-20
AI Technical Summary
The prior art is difficult to effectively identify and resolve performance issues when an application is executed on a computing platform, especially when these issues are attributed to defects in hardware components, operating systems, or computing services, resulting in poor application performance and poor user experience.
By establishing performance benchmarks, collecting and comparing performance data from different applications, identifying performance abnormalities, and recommending or automatically implementing solutions to improve application performance, including modifying application resources or updating software, so applications can perform more consistently and efficiently on the computing platform .
Before developers even realize performance issues, identify and solve the performance issues of applications, improve the execution efficiency and user experience of applications on the computing platform, reduce developer fatigue caused by unnecessary work, and provide more consistent users Experience.
Smart Images

Figure CN112041818B_ABST
Abstract
Description
Background Art
[0001] After receiving explicit permission from an end user to collect and utilize performance data collected during the execution of an application, a software developer can analyze the performance data (e.g., data regarding application stability, rendering time, battery usage) to debug problems or otherwise improve the application. For example, a developer can release an application update that fixes a periodic crash identified based on performance data collected from a subset of devices executing the application.
[0002] Sometimes, a problem may not be visible in performance data but may still be observable by users or detectable by other means. As a result, application developers may be unaware that their applications have performance issues. Sometimes, problems may be attributed to defects in hardware components, operating systems or computing platforms, or computing services maintained outside the control of the application developer. Due to these defects, different software developers may duplicate work to solve similar problems that arise during the execution of their respective applications. Sometimes, even if application developers are aware of performance issues with their applications, the application developers may lack the knowledge to solve the problem. As a result, the performance of the application will be degraded, which may lead to frustration of end users. Summary of the invention
[0003] The overall technology of the present invention relates to enabling a computing system to identify anomalies in application performance relative to the performance of other applications executed on a similar or corresponding computing platform (or operating system). The computing system can recommend or automatically implement solutions to resolve the identified anomalies, thereby improving the relative performance of the application on the computing platform. As an example, a computing system can obtain application performance data associated with multiple different applications executed on a computing device operating a corresponding (or at least similar) computing platform (e.g., data about application stability, rendering time, battery usage, permission denial, startup time, radio scanning, and other information indicating anomalies or performance metrics). The computing system can standardize the performance data of each application by deriving one or more performance metrics of each application based on the performance data. The computing system can compare the performance metric of a specific application with a corresponding performance benchmark established by the computing system. Comparing the performance metric of a specific application with the established performance metric can indicate the performance of the application relative to other different applications executed on the same or similar computing platform. The computing system can determine one or more ways in which a specific application can improve the performance metric to reach or exceed the performance benchmark. The computing system may output recommended improvements that a software developer may selectively implement (e.g., in a patch or software update), or in some cases the computing system may automatically (i.e., without user intervention) reconfigure application executables or other application resources to automatically implement the improvements.
[0004] By establishing a benchmark and then evaluating the performance metrics of applications executed on a computing platform against the benchmark, the exemplary computing system can identify performance problems of an application before the developer even considers seeking performance improvements. The exemplary computing system can output information about performance gaps to teach the developer how the performance of his or her application compares to other applications available on the platform. The developer does not need to know which areas can be improved, or even how to improve these areas. Alternatively, the exemplary computing system can automatically identify areas for improvement and recommend or automatically implement fixes to improve the performance of the application to be more consistent with the performance of other applications executed on the computing platform. If the problem can be attributed to defects in hardware components, operating systems, or computing platforms, or computing services maintained outside the control of the application developer, the exemplary computing system can provide information to the developer that indicates how other software developers solve similar problems, thereby preventing the developer from performing unnecessary work to find a solution. Therefore, the exemplary computing system can make applications executed on a specific computing platform perform better, operate more efficiently, and may result in a more consistent user experience on the computing platform. Therefore, the exemplary computing system can minimize the frustration that users may otherwise experience from interacting with applications that are slow or perform poorly on the computing platform.
[0005] Throughout this disclosure, examples are described in which a computing device and / or computing system may analyze information (e.g., application performance data, etc.) only when the computing device and / or computing system receives explicit permission from a user of the computing device and / or computing system. For example, in the following case, where a computing device and / or computing system may collect information about the performance of an application executed on a computing device executing a similar or corresponding computing platform, an individual user may be provided with an opportunity to provide input to control whether a program or feature (e.g., a developer service module) of the computing device and / or computing system may collect and utilize the information. In addition, an individual user may be provided with an opportunity to control what the program or feature can and cannot do with the information. In addition, the collected information may be pre-processed in one or more ways to remove personally identifiable information before being transmitted, stored, or otherwise used by the computing device and / or computing system. For example, before the exemplary computing system stores performance data associated with an application executed on the computing device, the exemplary computing system may pre-process the performance data to ensure that any user-identifying information or device-identifying information embedded in the performance data is removed. Thus, the user can control whether information about the user and the user's device is collected, and if so, how the computing device and / or computing system uses the information.
[0006] In one example, a method is described that includes obtaining, by a computing system, first performance data collected during execution of a first application at a first computing device group, determining, based on the first performance data, at least one metric for quantifying performance of the first application, and comparing, by the computing system, the at least one metric to a corresponding benchmark derived from second performance data collected during execution of one or more second applications at a second computing device group, each of the one or more second applications being different from the first application. The method also includes determining whether the at least one metric is within a threshold amount of the corresponding benchmark, determining a fix to the first application based at least in part on determining that the at least one metric is not within the threshold amount of the corresponding benchmark, and outputting, by the computing system, an indication of the fix to the first application for presentation on a developer device.
[0007] In another example, a computer-readable storage medium is described that includes instructions that, when executed, cause at least one processor to obtain first performance data collected during execution of a first application at a first computing device group, determine at least one metric for quantifying performance of the first application based on the first performance data, and compare the at least one metric to a corresponding benchmark derived from second performance data collected during execution of one or more second applications at a second computing device group, each of the one or more second applications being different from the first application. The instructions, when executed, also cause the at least one processor to determine whether the at least one metric is within a threshold amount of the corresponding benchmark, determine a fix to the first application based at least in part on determining that the at least one metric is not within the threshold amount of the corresponding benchmark, and output an indication of the fix to the first application for presentation on a developer device.
[0008] In another example, a computing system is described that includes at least one processor configured to obtain first performance data collected during execution of a first application at a first computing device group, determine at least one metric for quantifying performance of the first application based on the first performance data, and compare the at least one metric to a corresponding benchmark derived from second performance data collected during execution of one or more second applications at a second computing device group, each of the one or more second applications being different from the first application. The at least one processor is further configured to determine whether the at least one metric is within a threshold amount of the corresponding benchmark, determine a fix to the first application based at least in part on determining that the at least one metric is not within the threshold amount of the corresponding benchmark, and output an indication of the fix to the first application for presentation on a developer device.
[0009] In another example, a system is described that includes: means for obtaining first performance data collected during execution of a first application at a first computing device group, means for determining at least one metric for quantifying performance of the first application based on the first performance data, and means for comparing the at least one metric to a corresponding benchmark derived from second performance data collected during execution of one or more second applications at a second computing device group, each of the one or more second applications being different from the first application. The system also includes means for determining whether the at least one metric is within a threshold amount of the corresponding benchmark, means for determining a repair to the first application based at least in part on determining that the at least one metric is not within the threshold amount of the corresponding benchmark, and means for outputting an indication of the repair to the first application for presentation on a developer device.
[0010] In another example, a method is described that includes: outputting, by a computing device, first performance data collected during execution of a first application to a computing system; receiving, by the computing device, an instruction from the computing system to perform a repair of the first application, wherein the repair improves performance of the first application relative to performance of one or more second applications executed at a computing device group, each of the one or more second applications being different from the first application; and executing, by the computing device, the instruction to perform the repair of the first application.
[0011] In another example, a computer-readable storage medium is described that includes instructions that, when executed, cause at least one processor to output first performance data collected during execution of a first application to a computing system; receive instructions from the computing system to perform a repair of the first application, the repair improving performance of the first application relative to performance of one or more second applications executed at a computing device group, each of the one or more second applications being different from the first application; and execute the instructions to perform the repair of the first application.
[0012] In another example, a computing device is described that includes at least one processor configured to output first performance data collected during execution of a first application to a computing system; receive instructions from the computing system to perform a repair of the first application, wherein the repair improves performance of the first application relative to performance of one or more second applications executed at a computing device group, each of the one or more second applications being different from the first application; and execute the instructions to perform the repair of the first application.
[0013] In another example, a system is described that includes a device for outputting first performance data collected during execution of a first application to a computing system; a device for receiving instructions from the computing system to perform a repair of the first application, wherein the repair improves the performance of the first application relative to the performance of one or more second applications executed at a computing device group, each of the one or more second applications being different from the first application; and a device for executing the instructions to perform the repair of the first application.
[0014] In another example, a method is described, the method comprising: receiving, by a computing device, from a computing system, a recommendation for a repair of a first application executed at a first computing device group, the repair improving performance of the first application relative to performance of one or more second applications executed at a second computing device group, the one or more second applications being different from the first application; receiving, by the computing device, user input authorizing the repair of the first application; receiving, by the computing device, from the computing system an instruction to perform the repair of the first application in response to sending an indication of the user input authorizing the repair of the first application to the computing system; and performing, by the computing device, the repair of the first application based on the instruction.
[0015] In another example, a computer-readable storage medium is described that includes instructions that, when executed, cause at least one processor to receive from a computing system a recommendation for a repair of a first application executed at a first computing device group, the repair improving performance of the first application relative to performance of one or more second applications executed at a second computing device group, each of the one or more second applications being different from the first application; receive user input authorizing the repair of the first application; in response to sending an indication of the user input authorizing the repair of the first application to the computing system, receive from the computing system an instruction to perform the repair of the first application; and perform the repair of the first application based on the instruction.
[0016] In another example, a computing device is described that includes at least one processor configured to receive a recommendation from a computing system for a repair of a first application executed at a first computing device group, the repair improving performance of the first application relative to performance of one or more second applications executed at a second computing device group, each of the one or more second applications being different from the first application; receive user input authorizing the repair of the first application; receive an instruction from the computing system to perform the repair of the first application in response to sending an indication of the user input authorizing the repair of the first application to the computing system; and perform the repair of the first application based on the instruction.
[0017] In another example, a system is described that includes a device for receiving from a computing system a recommendation for a repair of a first application executed at a first computing device group, the repair improving performance of the first application relative to performance of one or more second applications executed at a second computing device group, each of the one or more second applications being different from the first application; a device for receiving user input authorizing the repair of the first application; a device for receiving from the computing system an instruction to perform the repair of the first application in response to sending an indication of the user input authorizing the repair of the first application to the computing system; and a device for performing the repair of the first application based on the instruction.
[0018] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the disclosure will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 is a conceptual diagram illustrating an exemplary computing system that is configured to identify performance issues of an application relative to the performance of other applications executed on a similar or corresponding computing platform and determine ways to improve the performance of the application in accordance with one or more aspects of the present disclosure.
[0020] Figure 2 is a block diagram illustrating an exemplary computing system configured to identify performance issues of an application relative to the performance of other applications executed on a similar or corresponding computing platform and determine ways to improve the performance of the application in accordance with one or more aspects of the present disclosure.
[0021] Figure 3 are exemplary screenshots of a display screen of a computing device accessing an application performance evaluation service provided by an exemplary computing system in accordance with one or more aspects of the present disclosure.
[0022] Figure 4 is a flow chart illustrating exemplary operations performed by an exemplary computing system configured to identify performance issues of an application relative to the performance of other applications executed on similar or corresponding computing platforms and determine ways to improve the performance of the application in accordance with one or more aspects of the present disclosure. DETAILED DESCRIPTION
[0023] Figure 11 is a conceptual diagram illustrating an exemplary computing system that is configured to identify performance issues of an application relative to the performance of other applications executed on similar or corresponding computing platforms and determine ways to improve the performance of the application in accordance with one or more aspects of the present disclosure. System 100 includes computing system 160 that communicates with computing device 110, computing devices 116A-116N (collectively, “computing devices 116”), and computing devices 118A-118N (collectively, “computing devices 118”) via network 130. Although operations attributed to system 100 are described as being primarily performed by computing system 160 and computing devices 110, 116, and 118, in some examples, operations of system 100 may be performed by other computing devices. Figure 1 More or fewer computing devices and systems may be shown. For example, computing devices 110, 116, and 118 may be a single computing device.
[0024] Computing system 160 includes developer services module 162 and application data storage 164. Computing device 110 includes developer client module 120, and also includes user interface component ("UIC") 112 configured to output user interface 114. Each computing device 116 executes a respective instance of application 122A, and each computing device 118 executes a respective instance of application 122B.
[0025] Network 130 represents any public or private communication network, such as cellular, Wi-Fi, and / or other types of networks, for transmitting data between computing systems, servers, and computing devices. Network 130 may include one or more network hubs, network switches, network routers, or any other network equipment that are operatively interconnected to provide information exchange between computing system 160 and computing devices 110, 116, and 118. Computing system 160 and computing devices 110, 116, and 118 may transmit and receive data across network 130 using any suitable communication technology.
[0026] Computing system 160 and computing devices 110, 116, and 118 may each be operatively coupled to network 130 using a respective network link. The links coupling computing system 160 and computing devices 110, 116, and 118 to network 130 may be Ethernet, ATM, or other types of network connections, and such connections may be wireless and / or wired connections.
[0027] The computing system 160 represents any combination of one or more computers, mainframes, servers, blades, cloud computing systems, or other types of remote computing systems that are capable of exchanging information via the network 130 as part of the application performance evaluation service. That is, the computing system 160 can receive application performance data via the network 130 and analyze the performance data to determine performance issues and ways to resolve performance issues of applications executed at the computing devices 116, 118. The computing system 160 can output recommendations to fix any identified performance issues found during the performance data analysis process. In some cases, the computing system 160 outputs the recommendations and other information about its analysis to the computing device 110, such as to be subsequently presented to a developer or other user of the computing device 110 via the user interface 114. In some examples, the computing system 160 can interact directly with the computing devices 116, 118 to implement fixes for improving the performance of individual applications executed at the computing devices 116, 118.
[0028] Developer service module 162 controls computing system 160 to perform specific operations to implement application performance evaluation services provided by computing system 160. Developer service module 162 may provide an interface between computing system 160 and a client device, such as computing device 110, that accesses the performance evaluation services provided by computing system 160, for example, to obtain information about application performance and ways to improve application performance. Developer service module 162 may perform the operations described herein using software, hardware, firmware, or a mixture of hardware, software, and firmware resident and / or executed in computing system 160. Computing system 160 may execute developer service module 162 via multiple processors or multiple devices, as a virtual machine executing on underlying hardware, as one or more services of an operating system or computing platform, and / or as one or more executable programs of an application layer of a computing platform of computing system 160.
[0029] Developer service module 162 is configured to collect application performance data from different applications executing on similar or corresponding computing platforms. Developer service module 162 analyzes the application performance data to determine issues in the performance of a specific application relative to other different applications executing in similar or corresponding execution environments. That is, unlike other debugging or analysis tools that may analyze application performance data for a single application to determine performance issues that may occur when the application is executed on a different computing architecture, operating system, or computing platform, developer service module 162 instead identifies performance differences between a single application and other different applications executing in similar or corresponding computing environments.
[0030] Developer services module 162 collects and stores application performance data collected from network 130 in application data storage 164. Performance data (e.g., data regarding application stability, rendering time, battery usage, permission denials, launch time, radio scans, and other information indicating anomalies or performance metrics) contained in application data storage 164 may be organized and searchable according to application identifiers (e.g., names, publishers, etc.), application categories (e.g., travel, leisure, entertainment, etc.), application types (e.g., navigation, transportation, games, etc.), or other searchable criteria. For example, application data storage 164 may store first performance data collected from computing device 116 during corresponding execution of application 122A by computing device 116 separately from second performance data collected from computing device 118 during corresponding execution of application 122B by computing device 118.
[0031] In some examples, performance data stored in application data storage 164 can be used as a proxy indicator or indirect indicator of performance. In other words, the data collected and maintained in data storage 164 may be low-level performance details that, by themselves, do not provide insight into a particular problem, but are indicative of performance when aggregated together. For example, radio scans and location queries tend to consume a lot of battery power, so the computing platform may require that applications minimize the execution of these functions unless necessary or only done with limited frequency. Therefore, an application that performs more radio scans or location queries than other applications may indicate that the application is performing poorly in terms of battery consumption relative to other applications. Developer service module 162 can further determine things about "rendering performance" or other categories of performance by viewing multiple types of performance data simultaneously.
[0032] When collecting and storing performance data, the developer service module 162 may take precautions to ensure that user privacy is preserved. That is, the developer service module 162 may only collect, store, and analyze performance data if the developer service module 162 receives explicit permission from the user of the computing device from which the performance data comes. For example, in the case discussed below where the developer service module 162 may collect information about the performance of applications executed on the computing devices 116, 118, individual users of the computing devices 116, 118 may be provided with the opportunity to provide input to the computing devices 116, 118 and / or the computing system 160 to control whether the developer service module 162 may collect and utilize their information. Individual users may also be provided with the opportunity to control what the developer service module 162 can or cannot do with the information. Before the application performance data is transmitted to the developer service module 162, stored by the developer service module 162, or otherwise used, it may be pre-processed in one or more ways to remove personally identifiable information. For example, before developer services module 162 collects performance data associated with application 122A when executed on computing device 116A, computing device 116A may pre-process the performance data to ensure that any user-identifying information or device-identifying information embedded in the performance data is removed prior to transmission to computing system 160. In other examples, developer services module 162 may pre-process the performance data upon receipt and prior to storing the performance data in data store 164. In both cases, the user may control whether performance data is collected and, if so, how computing device 116A and computing system 160 use the information.
[0033] Computing device 110 represents any suitable computing device or computing system capable of exchanging information via network 130 to access the performance evaluation service provided by computing system 160 to obtain information about application performance and ways to improve application performance. That is, computing device 110 may be a software developer or designer workstation configured to access performance data stored in data storage 164, obtain analysis performed by developer service module 162, and obtain performance improvement recommendations generated by developer service module 162.
[0034] Developer client module 120 may provide an interface between computing device 110 and services provided by computing system 160. For example, developer client module 120 may be a standalone application executed on computing device 110, or in some examples, client module 120 may be a subroutine or Internet application accessed from an Internet browser executed on computing device 110. In either case, developer client module 120 is configured to exchange information with computing system 160 to implement performance evaluation services.
[0035] The developer client module 120 is configured to receive the analysis results of the application performance data and the recommendations for improving the results determined by the computing system 160 from the computing system 160 via the network 130. The module 120 may perform the operations described herein using software, hardware, firmware, or a mixture of hardware, software, and firmware resident and / or executed in the computing device 110. The computing device 110 may execute the module 120 through multiple processors or multiple devices, as a virtual machine executed on the underlying hardware, as one or more services of the operating system or computing platform, and / or as one or more executable programs of the application layer of the computing platform of the computing device 110.
[0036] The UIC 112 of the computing device 110 may act as an input and / or output device for the computing device 110. Various technologies may be used to implement the UIC 112. For example, the UIC 112 may act as an input device using a presence-sensitive input screen, microphone technology, infrared sensor technology, or other input device technology for receiving user input. The UIC 112 may act as an output device configured to present output to a user using any one or more display devices, speaker technology, tactile feedback technology, or other output device technology for outputting information to a user.
[0037] The developer client module 120 may cause the UIC 112 to output a user interface associated with the services provided by the computing system 160. Figure 1 As shown, the developer client module 120 may send instructions to the UIC 112 so that the UIC 112 displays the user interface 114 on the display screen of the UIC 112. The user interface 114 may present information obtained from the developer service module 162 in a visual, auditory, or tactile format, so that the developer may better understand how the performance of a particular application compares to the performance of other applications executed on similar or corresponding computing platforms.
[0038] Computing devices 116, 118 represent any suitable computing device or computing system capable of locally executing applications such as applications 122A and 122B, respectively, and also capable of exchanging information regarding the performance of locally executed applications with computing system 160 via network 130. Examples of computing devices 116, 118 include mobile phones, tablet computers, laptop computers, desktop computers, servers, mainframes, blades, wearable devices (e.g., computerized watches, etc.), home automation devices, assistant devices, gaming consoles and systems, media players, e-book readers, television platforms, automotive navigation or infotainment systems, or any other type of mobile, non-mobile, wearable, and non-wearable computing devices configured to execute applications on a computing platform being analyzed by developer service module 162. Each computing device 116, 118 executes applications on a similar or corresponding computing platform, such as a common operating system.
[0039] Applications 122A and 122B represent two different machine-readable executable files that are configured to operate at an application layer of a similar or same computing platform operating on each of computing devices 116 and 118. Computing device 116 may execute instructions associated with the particular computing platform in which application 122A is executed, and computing device 118 may execute similar instructions associated with the same computing platform as computing device 116, but for executing application 122B instead of application 122A.
[0040] Examples of applications 122A and 122B are too numerous to list. As some examples, applications 122A and 122B may include business applications, developer tools, educational applications, entertainment applications, financial applications, gaming applications, graphics or design applications, health and fitness applications, lifestyle or assistant applications, medical applications, music applications, news applications, photography, video and other multimedia applications, productivity applications, reference applications, social networking applications, sports applications, travel applications, utility applications, weather applications, communication applications, calendar applications, or any other category or type of application.
[0041] Applications 122A and 122B are completely different applications and can be associated with different application developers, producers, versions, etc. In some examples, applications 122A and 122B are "peers" because they are from the same category or type of application (e.g., travel) and perform similar functions or provide similar features (e.g., both can implement transportation reservations). In other examples, applications 122A and 122B are from the same category or type of application (e.g., travel), but are not "peers" because the two applications can perform different functions or provide different features (e.g., one can implement transportation reservations, while the other can facilitate navigation). In any case, regardless of whether applications 122A and 122B are simply from similar categories or types, or whether applications 122A and 122B are peers, applications 122A and 122B are completely different applications from different application developers, producers, etc.
[0042] In operation, Figure 1 A user of computing device 110, who in an example is a software developer or designer of application 122A, may wish to understand how the performance of application 122A compares to the performance of other applications, such as application 122B, executing on the same or similar computing platform. The user may provide input on a presence-sensitive screen of UIC 112 at or near the location of the UIC displaying user interface 114. UIC 112 may provide information about the input to developer client module 120, and in response to the information about the input, developer client module 120 may access services provided by developer service module 162 to obtain analysis of the performance of application 122A relative to the performance of other applications executing on similar or corresponding computing platforms.
[0043] After receiving explicit permission from each end user of computing devices 116, 118 to collect and use performance data collected during execution of applications 122A, 122B, developer services module 162 of computing system 160 may obtain performance data collected by computing devices 116, 118 while computing devices 116, 118 are executing respective instances of applications 122A, 122B. For example, when application 122A is executed on computing device 116A, application 122A may output performance data indicating how well application 122A is executing in the execution environment of computing device 116A. Computing device 116A may send the performance data to computing system 160 via network 130, where developer services module 162 may store the performance data in application performance data storage 164. Each of computing devices 116 , 118 may perform similar operations as computing device 116A to collect and send performance data to computing system 160 , the performance data indicating how applications 122A, 122B are performing in the respective execution environments of computing devices 116 , 118 .
[0044] The developer service module 162 can analyze the performance data of each of the applications 122A and 122B to determine one or more performance metrics for each of the applications 122A and 122B. The basis behind the performance metrics is described in more detail with respect to the other figures. But in general, a performance metric can be any quantitative and measurable value that can be derived from the application performance data to indicate a performance level associated with certain aspects of the application. Some examples of performance metrics include: battery statistics (e.g., consumption rate, etc.), stability measurements (e.g., crash rate, etc.), rendering metrics (e.g., latency between frame rendering, etc.), timing metrics (e.g., startup time, delay in transitioning from one mode to another, delay in implementing intent, delay in retrieving or uploading information, etc.), licensing metrics (e.g., frequency of failed requests or user-rejected requests to use device components or user information, etc.).
[0045] With the defined set of performance metrics, developer services module 162 can establish one or more benchmarks for evaluating the performance metrics of individual applications; specifically, determining the performance of an application on a computing platform relative to the performance of other applications running on the computing platform. In other words, developer services module 162 can determine performance targets based on the performance metrics of one or more applications toward which other applications should operate, so that a user interacting with an application on the computing platform has a pleasant, consistent, and non-frustrating user experience regardless of which application he or she interacts with.
[0046] To establish a benchmark, developer service module 162 may rank the performance metrics of multiple applications for each particular metric to determine the best performing application. In some examples, developer service module 162 may determine a composite metric (e.g., an average, a median, etc.) based on the metrics of two or more applications to use as a benchmark for evaluating the performance of other applications. In some examples, developer service module 162 may determine a composite metric based on the two or more highest rankings, the two or more lowest rankings, or some other combination of application metrics to use as a benchmark for evaluating the performance of other applications.
[0047] The developer service module may evaluate the application by comparing the performance metric of the application to the corresponding benchmark. For example, if the performance data stored at the data storage 164 indicates that the application 122A has a performance metric within a threshold of the benchmark value, the developer service module 162 may determine that the application 122A does not have a performance problem because the performance of the application is related to the particular metric. Otherwise, the developer service module 162 may determine that if the application 122A has a performance metric outside the threshold of the benchmark value, the application 122A does have a performance problem because the performance of the application 122A is related to the particular metric.
[0048] exist Figure 1 In the example of , assume that developer service module 162 establishes a benchmark as a specific metric (e.g., launch time) based on a corresponding metric associated with application 122B. Developer service module 162 can then evaluate the launch time of application 122A by comparing the launch time of application 122A to the benchmark.
[0049] The developer services module 162 may determine whether the launch time metric is within a threshold amount of the benchmark. In some examples, the threshold amount may be a percentage (e.g., ten percent) or a value (e.g., two milliseconds). In some examples, the threshold amount may be zero, and meeting the benchmark may require meeting or exceeding the benchmark performance. In some examples, the threshold amount may be a percentile ranking of the application relative to other applications executing on the computing platform. That is, in order to meet the benchmark, the developer services module may require that the application's performance be better than the next "x" percentiles of other applications that have been rated against the benchmark (e.g., "x" is any percentile value).
[0050] exist Figure 1 In the example of , the developer service module 162 can determine that the launch time of the application 122A does not meet the launch time benchmark because the launch time of the application 122A (e.g., average) exceeds the benchmark value by more than a threshold amount of time (e.g., 5 milliseconds). In response to determining that the performance metric is not within the threshold amount of the corresponding benchmark, the developer service module can determine a fix for the application to improve its performance.
[0051] Repairing the application may include one or more modifications to source code or configuration files associated with the application. In some examples, repairing the application may include disabling a library, function call, software development kit, application programming interface, or service used by the application. In some examples, repairing the application may include replacing a first library, function call, or service utilized by the application with an alternative library, function call, or service.
[0052] For example, in addition to tracking performance data associated with applications executing on computing devices 116, 118, developer services module 162 may also maintain information about ways to improve application performance relative to each particular metric. In some examples, the information may be based on feedback obtained from other developers of applications that have the metric that forms the basis of the benchmark. In some examples, the information may be based on feedback obtained from other developers of applications that have found ways to improve their metric relative to the benchmark. In some examples, the information may be based on work-arounds obtained from developers of computing platforms that may have discovered fixes after identifying that other applications in similar situations executing on the computing platform have improved their metric relative to the benchmark.
[0053] In some examples, information about ways to improve application performance may be automatically derived by developer services module 162 (e.g., using a machine learning model, trial and error, or other ways described in more detail with respect to other figures). For example, developer services module 162 may determine which application programming interfaces (APIs) or libraries are used by the poorly performing application, determine that other better performing applications on the computing platform use different APIs or libraries for the same functionality (e.g., different third-party libraries for user authentication), and conclude that the poorly performing application may improve its performance by replacing existing API or library calls with the API or library calls used by the better performing application.
[0054] exist Figure 1 In an example of , developer services module 162 may determine that the identified launch time problem of application 122A may be attributed to a slow-executing application programming interface (API) that application 122A relies on during launch, which developer services module 162 has observed to cause slow launch times for other applications. As some examples, developer services module 162 may determine that a fix to the launch time problem of application 122A is to use a more responsive API, relocate API calls outside of the launch thread, or adjust the launch order so that the slow API is not used until other resources of application 122A are otherwise up and running.
[0055] In any case, once the developer service module 162 determines the fix, the developer service module 162 may share information about the fix with the developer client module 120 for subsequent presentation to the user developer. The developer service module 162 may output an indication of the fix to the application 122A for presentation at the computing device 110. For example, the developer service module 162 may send data via the network 130 that is interpreted by the developer client module 120 as an instruction to update the user interface 114 with information about the fix.
[0056] In response to receiving an instruction to update user interface 114 with information about a fix, developer client module 120 may cause UIC 112 to output user interface 114. That is, developer client module 120 may cause UIC 112 to output for display any identified performance anomalies or problems and / or any fixes identified to address the identified performance anomalies or problems. In this way, a computing system operating according to the described techniques may output indications of performance problems and possible fixes that a software developer, designer, or other user of the computing system may select to implement or not implement, thereby improving the performance of an application relative to the performance of other applications executing in a similar or corresponding computing platform.
[0057] By establishing a baseline and then evaluating performance metrics of applications executing on a computing platform against the baseline, the exemplary computing system can identify performance issues of an application before the developer even considers seeking performance improvements. The exemplary computing system can output information about performance gaps to educate the developer on how the performance of his or her application compares to other applications available on the platform. The developer does not need to know which areas need to be improved, which areas can be improved, or even how to improve those areas. Instead, the exemplary computing system can automatically identify areas for improvement and recommend or automatically implement fixes to improve the performance of the application to be more consistent with the performance of other applications executing on the computing platform. As a result, the exemplary computing system can enable applications executing on a particular computing platform to operate more efficiently and may provide a more consistent user experience when a user interacts with different applications on the computing platform.
[0058] Figure 2 is a block diagram illustrating an exemplary computing system configured to identify performance issues of an application relative to the performance of other applications executed on a similar or corresponding computing platform and determine ways to improve the performance of the application in accordance with one or more aspects of the present disclosure. Figure 2 The computing system 260 is described as Figure 1 An example of a computing system 160 . Figure 2 Only one particular example of computing system 260 is shown, and many other examples of computing system 260 may be used in other instances and may include a subset of the components included in computing system 260, or may include Figure 2 Additional components not shown.
[0059] As in Figure 2 As shown in the example of , computing system 260 includes one or more processors 270, one or more communication units 272, and one or more storage components 276 communicatively coupled via communication channel 274. Storage component 276 includes developer service module 262 and application performance data storage 264. Developer service module 262 includes UI module 266, analysis module 267, and correction module 268.
[0060] Communication channel 274 can interconnect each of components 266, 270, 272, and 276 for inter-component communication (physically, communicatively, and / or operationally). In some examples, communication channel 274 can include a system bus, a network connection, an inter-process communication data structure, or any other method for transferring data.
[0061] The one or more communication units 272 of the computing system 260 can communicate with external devices (e.g., Figure 1 The communication unit 272 may communicate with the computing devices 110, 116, 118 of the present invention. Examples of the communication unit 272 include a network interface card (e.g., such as an Ethernet card), an optical transceiver, a radio frequency transceiver, a GPS receiver, or any other type of device that can send and / or receive information. Other examples of the communication unit 272 may include a shortwave radio, a cellular data radio, a wireless network radio, and a universal serial bus (USB) controller.
[0062] One or more storage components 276 within computing system 260 may store information that is processed during operation of computing system 260 (e.g., computing system 260 may store application performance data collected and accessed by modules 262, 266, 267, and 268 and data store 264 during execution at computing system 260). In some examples, storage component 276 is temporary storage, meaning that the primary purpose of storage component 276 is not long-term storage. Storage component 276 on computing system 260 may be configured for short-term storage of information, as volatile memory, and therefore does not retain stored contents if power is removed. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory known in the art.
[0063] In some examples, storage component 276 also includes one or more computer-readable storage media. In some examples, storage component 276 includes one or more non-transitory computer-readable storage media. Storage component 276 can be configured to store more information than the amount of information usually stored by volatile memory. Storage component 276 can be further configured for long-term storage of information as a non-volatile storage space and retain information after power on / off cycles. Examples of non-volatile memory include magnetic hard disks, optical disks, floppy disks, flash memory, or electrically programmable memory (EPROM) or electrically erasable programmable memory (EEPROM). Storage component 276 can store program instructions and / or information (e.g., data) associated with modules 262, 266, 267, and 268 and data storage 264. Storage component 276 may include a memory configured to store data or other information associated with modules 262, 266, 267, and 268 and data storage 264.
[0064] One or more processors 270 may implement functionality and / or execute instructions associated with computing system 260. Examples of processor 270 include an application processor, a display controller, a graphics processor, an auxiliary processor, one or more sensor hubs, and any other hardware configured to act as a processor, processing unit, or processing device. Modules 262, 266, 267, and 268 may be operated by processor 270 to perform various actions, operations, or functions of computing system 260. For example, processor 270 of computing system 260 may retrieve and execute instructions stored by storage component 276 that cause processor 270 to perform operations attributed to modules 262, 266, 267, and 268. When executed by processor 270, these instructions may cause computing system 260 to store information within storage component 276.
[0065] The developer service module 262 may include Figure 1 The developer service module 262 may additionally include some or all of the functionality of the developer service module 162 of the computing system 160. Figure 1 Developer services module 262 may perform some or all of the functionality of client services module 120 of computing device 110. Developer services module 262 may perform operations similar to modules 162 and 120 for providing application performance evaluation services that identify performance anomalies in applications relative to other applications executing on the computing platform and recommend or implement fixes to improve the performance of applications on the computing platform.
[0066] UI module 280 may provide a user interface associated with the services provided by developer services module 262. For example, UI module 280 may host a web interface from which a client, such as computing device 110, may access the services provided by developer services module 262. For example, a user of computing device 110 may interact with a web browser or other application (e.g., developer client module 120) executing on or accessible from computing device 110. UI module 280 may send information to a web browser or cause a client to display information such as Figure 1 Other applications of the user interface 114 of the user interface.
[0067] Analysis module 267 may analyze application performance data stored in data storage 264. Analysis module 267 may determine performance metrics associated with individual applications executing on the computing platform, determine a benchmark for evaluating the performance metrics, and compare the performance metrics to the benchmark to determine whether the individual applications have potential performance anomalies or issues relative to other applications executing on the computing platform.
[0068] As an example of a performance metric, analysis module 267 can determine a permission rejection rate associated with an application. For example, a permission can grant an application access to a user's calendar, camera, user's contact list, user or device's location, microphone, phone, other sensors, messaging services, and storage. Applications often rely on specific permissions to function properly, but developers may not know what percentage of users actually agree to grant each permission to the application, and how user behavior in granting or not granting each permission to the application compares to other applications executed on the computing platform. The computing platform may need to minimize the frequency of permission rejections to improve user satisfaction with the computing platform. Analysis module 267 can define the permission rejection rate as the number (e.g., percentage) of daily permission sessions during which users do not grant or otherwise deny application permissions.
[0069] As another example of a performance metric, the analysis module 267 may determine the launch time of an application. The launch time may indicate the amount of time it takes for an application to launch. If a user can choose between two peer applications, he or she may more often choose to launch the peer with the shortest launch time. The computing platform may want all applications to launch as quickly as possible to provide the best user experience for the computing platform.
[0070] The analysis module 267 can quantify the start time based on a measure of time (eg, seconds, milliseconds, etc.) The analysis module 267 can quantify the start time into one of a plurality of different levels, eg, slow, medium, fast.
[0071] Analysis module 267 may further quantify the launch time metric in one or more other ways. For example, analysis module 267 may assign a "slow cold launch" label to an application at a percentage of daily sessions during which users of the application experience at least one cold launch time of more than five seconds. A cold launch of an application is when the application is started from scratch (e.g., frequently displaying a splash screen, etc.). However, analysis module 267 may assign a "slow hot launch" label to an application at a percentage of daily sessions during which users of the application experience at least one hot launch time of more than 1.5 seconds. A hot launch of an application occurs when an application enters the foreground of a user interface after previously executing in the background (e.g., typically displaying a previously viewed screen, etc., which was last viewed the last time the application was viewed). Analysis module 267 may assign other types of labels to the launch time metric that are specific to a particular computing platform.
[0072] As another example of a performance metric, the analysis module 267 can determine a failed wireless signal scan metric. For example, some applications may cause the computing device to scan for available Near Field Communication (NFC) or other wireless signals. Such scans are power intensive and often result in increased battery drain. Therefore, the computing platform may wish to encourage prevention of failed wireless signal scans that result in wasteful battery drain. Failed wireless signal scans may indicate the percentage of battery sessions during which users of the application experienced at least one failed wireless signal scan lasting more than 30 minutes or some other duration (i.e., the period between full charges of the device).
[0073] Other examples of performance metrics generated by the analysis module 267 may include an application unresponsiveness (ANR) rate, which is defined as the percentage of daily sessions during which an application user experienced at least one application unresponsiveness and required a restart, or a crash rate, which indicates the percentage of daily sessions during which users experienced at least one application crash. Other performance metric examples include a stuck wake lock rate, which indicates the percentage of battery sessions during which users experienced at least one partial wake lock for more than one hour while the application was running in the background.
[0074] Correction module 268 can obtain information from analysis module 267 about potential problems with applications executed on the computing platform relative to other applications executed on the same or similar computing platform, and determine repairs or other actions taken by computing system 260 to resolve the problem and improve the performance of the application. For example, correction module 268 can cause UI module 266 to output recommendations for repairing the application when the license rejection rate is outside the appropriate range for the computing platform, thereby providing detailed instructions for the license request.
[0075] In some cases, correction module 268 may alert computing system 160 so that the poorly performing application is downgraded in the rating of the application store. If the application has a performance metric that is far outside the threshold amount of the benchmark, correction module 268 may alert computing system 160 so that the poorly performing application cannot be discovered or obtained from the application store.
[0076] For each performance metric, the correction module 268 may maintain information about ways to improve application performance relative to each specific problem identified by the analysis module 267. For example, the correction module 268 may obtain feedback from application developers who have the metrics that form the basis of the benchmark or who have otherwise discovered ways to improve their application performance, and use the feedback to provide solutions or fixes to specific problems. The correction module 268 may obtain information about workarounds from the computing platform developer, who may have discovered the workarounds after identifying ways that other similarly situated applications executing on the computing platform have improved their metrics relative to the benchmark.
[0077] In some examples, the correction module 268 can automatically determine a fix for a problem by performing automatic error repair techniques, without any help from the user. For example, the correction module 268 can generate one or more potential software patches that may improve application performance, and through trial and error, try different patches to see which patch, if any, results in a fix. The correction module 268 can perform search-based program mutation, machine learning, and / or genetic programming techniques to identify one or more suitable fixes. After determining a potential fix, the correction module 268 can verify the fix by testing the fix or delivering the fix to the application developer so that he or she can verify the fix.
[0078] Figure 3 are exemplary screenshots of a display screen of a computing device accessing an application performance evaluation service provided by an exemplary computing system in accordance with one or more aspects of the present disclosure. Figure 3 A user interface 314 is shown. Figure 1 The system 100 is described in the context of Figure 3 .
[0079] When the developer client module 120 requests to access the application performance evaluation service provided by the developer service module 162, the computing system 160 may cause the computing device 110 to present the user interface 314 on the UIC 112. The user interface 314 is only one example of such a user interface, and there may be many other examples.
[0080] The purpose of user interface 314 is to provide application developers with an in-depth understanding of specific performance metrics obtained for a target application in order to understand how the target application performs relative to the best performing application executed on the computing platform. In some examples, the relative performance of an application can be based on the best performing applications of a similar category or type to the target application. In other examples, the relative performance of an application can be based on the best performing applications that are considered peers. In any case, the benefits of user interface 314 include leading to observable or measurable improvements to the entire computing platform or application ecosystem, rather than just improvements to individual applications executed in the ecosystem. For example, a developer can watch user interface 314 to understand where the performance of an application lags behind relative to peers or other applications, and therefore be motivated to improve the performance of the application, thereby leading to an improvement in the overall performance of the computing platform.
[0081] The user interface 314 is divided into several tabs or sections, each tab being associated with a different performance metric or group of performance metrics. Figure 3 As shown, the launch metric tab is in the foreground of the user interface 314, and the metrics BE and important metric groups appear in the background of the user interface 314 on tabs hidden from view.
[0082] Each tab may include a results section (e.g., displayed in a graph, table, chart, or other data format) that provides information about the relative performance of the application compared to a baseline and acceptable threshold levels within the baseline. Figure 3 In the example of , the launch performance metric of the application is outside of an acceptable threshold amount of the baseline.
[0083] In some examples, the results section includes information about where the performance of the application compares to other applications executing on the computing platform. Figure 3 In the example of , the launch metric of the application corresponds to the last 25% of the application executed on the computing platform.
[0084] The user interface 314 may use color or other formats to indicate the performance of the application. For example, in the case where the performance metric is outside the acceptable threshold level of the benchmark, the user interface 314 may use a red font or line color to emphasize that the application may have potential performance issues. In the case where the performance metric is within the acceptable threshold level of the benchmark, the user interface 314 may use a green font or line color to emphasize that the application performs similarly to other best performing applications on the computing platform.
[0085] Each tab of user interface 314 may include a summary and resolution aspect portion in which potential problems are clearly identified and recommended fixes for the potential problems are displayed. For example, user interface 314 includes a detailed summary of launch time problems and possible fixes that developers can implement to improve the launch time metric of an application.
[0086] User interface 314 may also include selectable elements that cause computing system 160 to perform various actions to further assist the developer in improving the performance of the application relative to the performance of other applications executing on the computing platform. As one example, a user may select a graphical element that causes computing system 160 to link to a source file editor to view highlighted portions of source code that computing system 160 suspects may be the cause of slow start times. As another example, a user may select a graphical element that causes computing system 160 to automatically modify the suspect portion of the source code without further user input.
[0087] Figure 4 is a flow chart illustrating exemplary operations performed by an exemplary computing system configured to identify performance issues of an application relative to the performance of other applications executed on similar or corresponding computing platforms and determine ways to improve the performance of the application in accordance with one or more aspects of the present disclosure. Operations 400-490 may be performed by a computing system such as computing systems 160, 260. In some examples, the computing system may be performed by communicating with Figure 4The computing system may perform operations 400-490 in an order different from the order shown. In some examples, the computing system may perform more or fewer operations than operations 400-490. For ease of description, Figure 2 Description of the context of computing systems 260 Figure 4 .
[0088] like Figure 4 As shown, in operation, computing system 260 may obtain user consent to use application performance data collected during execution of an application on the application's corresponding device (400). For example, before computing system 260 or any of computing devices 116, 118 stores or transmits application performance data, the user of computing device 116, 118 will be provided with an opportunity to give or not give computing system 260 permission to collect and utilize such data. Computing system 260 will use application performance data from the user's device only if the user explicitly and unambiguously consents to such data collection.
[0089] Computing system 260 may obtain first performance data collected during execution of a first application at a first computing device group (410), and computing system 260 may obtain second performance data collected during execution of one or more second applications at a second computing device group (420). For example, analysis module 267 may receive performance data transmitted from computing devices 116, 118 via network 130. Analysis module 267 may store the performance data in data storage 264.
[0090] In some examples, each computing device from the first computing device group and the second computing device group executes a respective instance of a common computing platform. In other words, each of the first computing device group and the second computing device group can operate a common operating system or computing platform such that the analysis module, when determining the performance of a particular application, determines the performance of the application relative to other applications executing on the same or similar computing system as the particular application.
[0091] In some examples, each of the first application and the one or more second applications can be associated with a common category or class of applications. For example, the first application and the second application can be travel category applications or navigation category applications within the same travel category.
[0092] In some examples, the first application and each of the one or more second applications are peer applications, and the first application differs from each of the one or more second applications in at least one of functionality, title, or developer. For example, the first application and the second application may be games of a particular type (e.g., a first-person shooter, a crossword puzzle, etc.), and while the applications may share some overlap in functionality, the applications are not identical and differ in functionality, title, gameplay, developer, appearance, or other characteristics.
[0093] Computing system 260 may determine at least one metric quantifying the performance of the first application based on the first performance data (430). For example, analysis module 267 may calculate a permission denial rate metric based on the performance data of application 122A, the performance data indicating how often a user blocks application 122A from accessing a particular permission of computing device 116 (e.g., camera, microphone, etc.).
[0094] Computing system 260 may determine one or more benchmarks based on the second performance data (440). For example, analysis module 267 may determine an average license denial rate for one or more other applications (e.g., application 122B) executing on computing devices 116, 118. Analysis module 267 may use the average license denial rate as a benchmark to determine whether an application has a license denial rate that is too high relative to other applications executing on the computing platform.
[0095] Computing system 260 may compare at least one metric to a corresponding benchmark derived from the second performance data ( 450 ). For example, analysis module 267 may compare the license denial rate of application 122A to the benchmark established during operation 440 .
[0096] If the metric is within a threshold amount of the benchmark, computing system 260 may determine that the first application does not have a performance issue with the particular metric (460, yes branch). Conversely, if the metric is not within a threshold amount of the benchmark, computing system 260 may determine that the first application has a performance issue with the particular metric (460, no branch). Analysis module 267 may determine that the license denial rate for application 122A exceeds the benchmark established during operation 440 by 10% or some other amount that exceeds an acceptable threshold.
[0097] Computing system 260 may determine a fix for the first application (470). For example, analysis module 267 may trigger correction module 268 to determine a way for application 122A to improve a permission rejection rate metric so that the overall performance of application 122A on the computing platform is closer to the performance of other best performing applications on the computing platform. Correction module 268 may examine source files or other properties of application 122A and determine that when application 122A requested permission to use the device location or camera, no explanation or reason for the request was provided to the user. Correction module 268 may determine that the permission rejection rate may be improved if application 122A includes such information when making future requests.
[0098] Computing system 260 may output an indication of the repair (480). For example, correction module 268 may send a command to UI module 266 so that a user interface such as user interface 114 or 314 provides information to a developer user of computing system 260 that he or she may wish to modify application 122A to provide more information when making a permission request.
[0099] Computing system 260 may implement the fix (490). For example, after outputting an indication of the recommended fix, UI module 266 may receive an indication of user input from computing device 110 or other developer device that authorizes automatic implementation of the fix by computing system 260. In response to receiving the indication of user input, correction module 268 may output instructions to computing device 116 for automatically implementing the fix for application 122A. The instructions may include an updated source file, an updated executable file, or an updated configuration file generated by correction module 268 and used during execution of application 122A to cause the fix to be implemented.
[0100] In addition to or as an alternative to implementing the fix, computing system 260 may take other actions in response to determining that performance in the application is insufficient relative to the performance of other applications on the computing platform. For example, the cause of the performance problem may not be obvious from existing application performance data. Computing system 260 may send instructions via network 130 to cause computing device 116 to begin collecting more detailed performance data than is normally collected when application 122A is executed on computing device 116. For example, computing system 260 may cause computing device 116 to increase the rate at which performance data is collected, or may cause computing device 116 to collect additional data beyond the normal data collected.
[0101] Clause 1. A method comprising: obtaining, by a computing system, first performance data collected during execution of a first application at a first computing device group; determining, based on the first performance data, at least one metric for quantifying the performance of the first application; comparing, by the computing system, the at least one metric with a corresponding benchmark, the corresponding benchmark being derived from second performance data collected during execution of one or more second applications at a second computing device group, each of the one or more second applications being different from the first application; determining whether the at least one metric is within a threshold amount of the corresponding benchmark; determining a repair to the first application in response to determining that the at least one metric is not within the threshold amount of the corresponding benchmark; and outputting, by the computing system, an indication of the repair to the first application for presentation on a developer device.
[0102] Clause 2. The method of clause 1, wherein each computing device from the first computing device group and the second computing device group executes a respective instance of a common computing platform.
[0103] Clause 3. The method of any one of Clause 1 or Clause 2, wherein the first application and each of the one or more second applications are associated with a common category or class of applications.
[0104] Clause 4: A method according to clause 3, wherein the first application and each of the one or more second applications are peer applications, and the first application differs from each of the one or more second applications in at least one of the following: functionality, title, or developer.
[0105] Clause 5. The method according to any one of Clauses 1 to 4 further includes: receiving an indication of user input from the developer device, the user input authorizing the automatic implementation of the repair by the computing system; and in response to receiving the indication of the user input, the computing system outputting instructions for automatically implementing the repair of the first application to one or more computing devices from the first computing device group.
[0106] Clause 6. The method of any one of clauses 1 to 5, wherein the repair of the first application comprises one or more modifications to source code or configuration files associated with the first application.
[0107] Clause 7. The method of any one of clauses 1 to 6, wherein the repairing of the first application comprises disabling a library, function call, or service used by the first application.
[0108] Clause 8. The method of any one of clauses 1 to 7, wherein the repairing of the first application comprises replacing a first library, function call, or service used by the first application with an alternative library, function call, or service.
[0109] Clause 9. The method of any one of clauses 1 to 8, wherein the first computing device group and the second computing device group comprise a single computing device.
[0110] Clause 10. A computing system comprising at least one processor configured to perform any of the methods of clauses 1 to 9.
[0111] Clause 11. A computing system comprising means for performing any of the methods of clauses 1 to 9.
[0112] Clause 12. A computer-readable storage medium comprising instructions that, when executed, cause at least one processor to perform any of the methods of clauses 1 to 9.
[0113] Item 13. A method, comprising: outputting, by a computing device, first performance data collected during execution of a first application to a computing system; receiving, by the computing device, from the computing system, an instruction to perform a repair of the first application, the repair improving the performance of the first application relative to the performance of one or more second applications executed at a computing device group, each of the one or more second applications being different from the first application; and executing, by the computing device, the instruction to perform the repair of the first application.
[0114] Item 14. A method, comprising: receiving, by a computing device, from a computing system, a recommendation for a repair of a first application executed at a first computing device group, the repair improving performance of the first application relative to performance of one or more second applications executed at a second computing device group, each of the one or more second applications being different from the first application; receiving, by the computing device, user input authorizing the repair of the first application; receiving, by the computing device, an instruction from the computing system to perform the repair of the first application in response to sending an indication of the user input authorizing the repair of the first application to the computing system; and performing, by the computing device, the repair of the first application based on the instruction.
[0115] Clause 15. A computing device comprising at least one processor, the at least one processor configured to perform any of the methods of Clause 13 or Clause 14.
[0116] Clause 16. A computing system comprising means for performing any of the methods of clause 13 or clause 14.
[0117] Clause 17. A computer-readable storage medium comprising instructions that, when executed, cause at least one processor to perform any of the methods of Clause 13 or Clause 14.
[0118] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, flash memory or any other storage medium that can be used to store the required program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave is used to transmit instructions from a website, server or other remote source, the definition of the medium includes coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave. However, it should be understood that computer-readable storage media and media and data storage media do not include connections, carriers, signals or other transient media, but are directed to non-transient tangible storage media. The disks and optical disks used herein include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks and blue-ray disks, wherein the disks usually copy data magnetically, while the optical disks copy data optically by lasers. The above combination should also be included in the scope of computer-readable media.
[0119] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, as used herein, the term "processor" may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules. In addition, these techniques may be fully implemented in one or more circuits or logic elements.
[0120] The techniques of the present disclosure may be implemented in a variety of devices or equipment including wireless handsets, integrated circuits (ICs) or groups of ICs (e.g., chipsets). Various components, modules, or units are described in the present disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily need to be implemented by different hardware units. In addition, as described above, various units may be combined in hardware units or provided by a collection of interoperable hardware units, including one or more processors in combination with appropriate software and / or firmware, as described above.
[0121] Various embodiments have been described. These and other embodiments are within the scope of the following claims.
Claims
1. A method for adjusting application performance, comprising: obtaining, by a computing system, first application performance data collected during execution of a first application at a first set of end-user computing devices; determining, based on the first application performance data, at least one metric for quantifying performance of the first application; comparing, by the computing system, the at least one metric to a corresponding benchmark derived from second application performance data collected during execution of one or more second applications at a second set of end-user computing devices, each of the one or more second applications being different from the first application; determining whether the at least one metric is within a threshold amount of the corresponding benchmark; determining, based at least in part on determining that the at least one metric is not within a threshold amount of the corresponding benchmark, a fix for improving performance of the first application, wherein the fix comprises one or more of a modification to source code of the first application or a modification to a configuration file associated with the first application; and Outputting, by the computing system, an indication of the repair of the first application for presentation on a developer device associated with the first application; receiving an indication of user input from the developer device, the user input authorizing automatic implementation of the repair by the computing system; and In response to receiving an indication of the user input, the computing system automatically modifies one or more of a portion of the source code of the first application or a portion of the configuration file associated with the first application, wherein automatically modifying the portion of the source code or the portion of the configuration file includes: reducing the startup time of the first application by at least automatically replacing one or more of the application programming interfaces with different application programming interfaces, adjusting the startup order of the first application to relocate application programming interface calls outside the startup order, or adjusting the startup order of the first application so that the application programming interface calls occur after other resources of the first application are loaded.
2. The method according to claim 1, wherein: Each end-user computing device from the first group of end-user computing devices and the second group of end-user computing devices executes a respective instance of a common computing platform.
3. The method according to claim 1, wherein: The first application and each of the one or more second applications are associated with a common category or class of applications.
4. The method according to claim 3, wherein: The first application and each of the one or more second applications are peer applications, and the first application differs from each of the one or more second applications in at least one of: functionality, title, or developer.
5. The method according to claim 1, wherein: The repairing of the first application further includes disabling one or more of libraries, function calls, or services used by the first application.
6. The method according to claim 1, wherein: The repairing of the first application further includes replacing one or more of the first libraries, function calls, or services used by the first application with one or more replacement libraries, one or more function calls, or one or more services.
7. The method according to any one of claims 1 to 6, wherein: The first group of end-user computing devices and the second group of end-user computing devices include a single computing device.
8. A computing system comprising at least one processor, the at least one processor being configured to perform any one of the methods of claims 1 to 7.
9. A computing system comprising means for performing any one of the methods of claims 1 to 7.
10. A computer-readable storage medium comprising instructions which, when executed, cause at least one processor to perform any one of the methods of claims 1 to 7.
Citation Information
Patent Citations
Multi-core system energy consumption optimization
CN103270470A
System and method for performance monitoring and repair of computers
US20090144584A1
Device validation, distress indication, and remediation
US20120290870A1
Methods and architecture for enhanced computer performance
US20160378545A1
Extensible indexing system evaluation and recommendation
US20170004155A1
