Providing application error data for use by third-party library development system

Abstract

To provide application error data for use by third-party development system.SOLUTION: A method includes receiving application error data from a client computing device by an application server system and receiving mapping data between library-dependent source code of the application and a third-party library. The library-dependent source code is loaded from the third-party library during execution of the application. The method also includes determining a match between the library-dependent source code and the application error data on the basis of the application error data and the mapping data, attributing an error to the third-party library, generating library error data associated with the third-party library, and sending the library error data to a third-party library development system.SELECTED DRAWING: Figure 11

Description

Background Art

[0001] Background Existing computing devices, including mobile computing devices, are configured to run any number of different applications (e.g., mobile applications, games) over time. Developers of various different applications may use one or more application development systems to develop and distribute these applications, which may then be executed by client computing devices associated with end users. In many cases, one or more centralized application servers may store those applications and corresponding metadata after they are developed, and end users may download and use these applications on their respective client computing devices. Unfortunately, in some situations, any given application may experience an error during its execution and, in some cases, may even crash or abort its execution based on the severity or type of the error that occurs.

Summary of the Invention

Means for Solving the Problems

[0002] Summary The present disclosure is directed to providing one or more portions of application error data for use by a third-party library development system that develops third-party libraries used by these applications based on one or more errors that occur during the execution of one or more applications. According to the techniques of the present disclosure, an application server system may be configured to process application error data associated with such errors that occur during the execution of at least one application on one or more client computing devices and output at least a portion of such data included in the library error data to a third-party library development system. In such a manner, library developers of one or more third-party libraries may receive and review such library error data, which may be related to the execution of corresponding library-dependent code included in or used by the application during execution. By reviewing and analyzing the library error data, the third-party library developer and / or the third-party library development system may attempt to identify any coding defects or other problems in the third-party library that may have caused a runtime error during application execution.

[0003] In one example, a method includes an application server system including one or more processors receiving application error data associated with at least one error that occurred during execution of at least one application on one or more client computing devices from the one or more client computing devices, and the application server system receiving mapping data that provides a mapping between (i) library-dependent source code of at least one application and (ii) at least one third-party library, where the library-dependent source code is loaded from at least one third-party library during execution of at least one application, and the method further includes the application server system determining a match between the library-dependent source code and at least one portion of the application error data based on the application error data and the mapping data. An exemplary method further includes, in response to determining the match, the application server system attributing at least one error that occurred during execution of at least one application to at least one third-party library, and the application server system generating library error data associated with at least one third-party library, where the library error data includes at least one portion of the application error data, and the method further includes the application server system transmitting the library error data to at least one third-party library development system that develops at least one third-party library. In one example, a method includes an application server system including one or more processors receiving application error data associated with at least one error that occurred during execution of at least one application on one or more client computing devices from the one or more client computing devices, and the application server system receiving mapping data that provides a mapping between (i) library-dependent source code of at least one application and (ii) at least one third-party library, where the library-dependent source code is loaded from at least one third-party library during execution of at least one application, and the method further includes the application server system determining a match between the library-dependent source code and at least one portion of the application error data based on the application error data and the mapping data. An exemplary method further includes, in response to determining the match, the application server system attributing at least one error that occurred during execution of at least one application to at least one third-party library, and the application server system generating library error data associated with at least one third-party library, where the library error data includes at least one portion of the application error data, and the method further includes the application server system transmitting the library error data to at least one third-party library development system that develops at least one third-party library.

[0004] In another example, an application server system includes at least one processor and at least one computer-readable storage device configured to store instructions, the instructions being executable by the at least one processor to receive, from one or more client computing devices, application error data associated with at least one error that occurred during execution of at least one application on the one or more client computing devices, and to receive mapping data that provides a mapping between (i) library-dependent source code of at least one application and (ii) at least one third-party library, the library-dependent source code being loaded from at least one third-party library during execution of at least one application, the instructions further being executable by the at least one processor to determine a match between the library-dependent source code and at least one portion of the application error data based on the application error data and the mapping data, and in response to determining the match, attribute at least one error that occurred during execution of at least one application to at least one third-party library and generate library error data associated with the at least one third-party library, the library error data including at least one portion of the application error data, the instructions further being executable by the at least one processor to transmit the library error data to at least one third-party library development system that develops the at least one third-party library.

[0005] In another example, a computer-readable storage device stores instructions that, when executed, cause at least one processor to perform operations. These exemplary operations include receiving, from one or more client computing devices, application error data associated with at least one error that occurred during the execution of at least one application on the one or more client computing devices, and receiving mapping data that provides a mapping between (i) library-dependent source code of at least one application and (ii) at least one third-party library, where the library-dependent source code is loaded from at least one third-party library during the execution of at least one application, and the exemplary operations further include determining a match between the library-dependent source code and at least one portion of the application error data based on the application error data and the mapping data. The exemplary operations further include, in response to determining the match, attributing at least one error that occurred during the execution of at least one application to at least one third-party library, and generating library error data associated with the at least one third-party library, where the library error data includes at least one portion of the application error data, and the exemplary operations further include transmitting the library error data to at least one third-party library development system that develops the at least one third-party library.

[0006] Details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the disclosure will be apparent from the description and drawings, and from the claims.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Best Mode for Carrying Out the Invention

[0008] Detailed Description FIG. 1 is a block diagram showing an exemplary distributed system 100 that includes an application server system 102 configured to process application error data for use by one or more third - party library development systems 132 according to one or more aspects of the present disclosure. In the example of FIG. 1, system 100 includes an application server system 102, one or more application development systems 128, one or more third - party library development systems 132, and one or more client computing devices 140. The application server system 102, application development systems 128, third - party library development systems 132, and client computing devices 140 may be communicatively coupled to each other via one or more networks 126 that may include one or more wireless and / or wired networks.

[0009] The application server system 102, application development systems 128, and The third-party library development system 132 may each include one or more computing devices or servers, and each computing device or server includes one or more processors. The client computing device 140 may also include one or more processors. Examples of the client computing device 140 include, but are not limited to, mobile phones, tablet computers, personal digital assistants (PDAs), laptop computers, portable game devices, portable media players, wearable computing devices (e.g., wristwatches, wrist-mounted computing devices, head-mounted computing devices), television platforms, or other types of computing devices. The application server system 102, the application development system 128, the third-party library development system 132, and the client computing device 140 may each include one or more communication units (such as those shown in FIG. 3). These communication units may transmit and / or receive data from one or more other computing devices or systems. In some examples, the communication units may support wireless and / or wired communication and may transmit and / or receive data using any of a variety of communication protocols.

[0010] As described above, the client computing device 140 is configured to execute any number of one or more applications 104 (e.g., mobile applications, games) over time. Various different application developers may use the application development system 128 to develop and distribute such applications 104, and the applications 104 may then be executed by the client computing device 140. Often, as shown in FIG. 1, the application development system 128 may store a local copy of the application 104 and may also distribute a copy of the application 104 for storage on the application server system 102. The end user of the client computing device 140 may then download and execute the application 104 on the client computing device 140, and such applications 104 may be developed by any number of different application developers using the application development system 128. The applications 104 may perform various functions for the client computing device 140 or may access one or more services. Email applications, camera applications, calendar applications, messaging applications, social media applications, travel applications, game applications, stock applications, and weather applications are all examples of applications 104.

[0011] In some situations, one or more of the applications 104 may experience an error during execution on the client computing device 140 and, in some cases, may even crash or abort execution, depending on the severity or type of error that occurs. In these situations, when an application has an error or crash during execution on one or more of the client computing devices, the runtime system of the client computing device 140 may generate an error report that is sent to one or more of the application server system 102 and / or the application development system 128 associated with the application. In such a manner, the application developer of the application may receive and review these error reports via the application development system 128 in an attempt to identify any coding defects or other problems within the application that may have caused the runtime error on the client computing device 140.

[0012] Often, the application 104 includes application-dependent source code and It includes both library-dependent source code and application-dependent source code. The application-dependent source code is typically source code developed, particularly for application 104 (e.g., by an application developer using application development system 128). The library-dependent source code is typically library-based code developed (e.g., by a library developer using third-party library development system 132) for incorporation into or otherwise use by various different applications 104 of application 104. As a result, a large percentage of the source code of application 104 may actually include library-dependent source code. Thus, source code that causes an application 104 to experience errors and crashes during execution may potentially include one or more portions of the library-dependent source code included within application 104.

[0013] Defects present in such library-dependent source code can cause an error or even a crash in any of the applications 104 that include such code. Conventionally, as described above, application error data 122 associated with an error or a crash of application 104 in application 104 is provided only to the application development system 128 in which these applications 104 were developed. However, this application error data 122 has not conventionally been provided to the third-party library development system 132 such that a library developer has access to such application error data 122 and can assess whether any of the third-party libraries 138 developed by the library developer could be the cause of an application error or crash.

[0014] Accordingly, according to the techniques of the present disclosure, the application server system 102 processes application error data 122 associated with errors that occur during the execution of one or more applications 104 on one or more client computing devices 140, and outputs at least a portion of such data included in the library error data 124 to a third-party library developer using the third-party library development system 132. In this way, the library developers of one or more third-party libraries 138 may receive and review such library error data 124, which may be related to the execution of library-dependent code from the third-party libraries 138 included in the application 104 for execution. By reviewing and analyzing the library error data 124, the third-party library developers and / or the third-party library development system 132 may attempt to identify any coding defects or other problems within the third-party libraries 138 that may have caused a runtime error during the execution of the application 104 on the client computing device 140.

[0015] As shown in FIG. 1 and described in further detail below, application server system 102 includes various functional modules. These include a library error data generation unit 110, an application error processing module 120, and a library error reporting module 121. The library error data generation unit 110 includes a library attribution module 114, a scrubbing module 116, a clustering module 118, and an optional de-obfuscation module 112. These modules may use any combination of software, hardware, and / or firmware that resides on and / or executes on application server system 102 to perform the operations described herein either individually or collectively. Application server system 102 may use one or more processors to execute these modules. In some cases, application server system 102 may execute these modules as one or more virtual machines running on a base hardware. In some cases, one or more of these may execute as services of an operating system or computing platform. It may also execute as a service of an operating system or computing platform.

[0016] As described in further detail below, the application error processing module 120 of application server system 102 may receive application error data 122 associated with at least one error that occurred during the execution of one or more applications 104 on client computing device 140 from the application error reporting module 142 of client computing device 140. Accordingly, the application error reporting module 142 is configured to send such application error data 122 associated with the execution of application 104 to the application error processing module 120 of application server system 102.

[0017] If one or more of the applications 104 experience a crash or other unexpected termination, the application error data 122 may include application crash report data associated with at least one error that caused one or more of the applications 104 to crash. The application crash report data may include stack trace data representing the nested function calls of the application 104, and at least one part of the application crash report data includes at least one part of the stack trace data. The nested function calls of the stack trace data are, for example, calls that may have come from the program loader to the location where a crash occurred during the execution of each of one or more of the applications 104. FIG. 4 shows an example of such stack trace data.

[0018] The application server system 102 also receives (e.g., from the application development system 128) mapping data 108 that provides a mapping between (i) the library-dependent source code of the application 104 and (ii) at least one third-party library of the third-party libraries 138 where the library-dependent source code is loaded during the execution of the application 104. In some cases, the application development system 128 may provide the application data 106 to the application server system 102, and the application data 106 may include metadata associated with the application 104. For example, the application data 106 may include the mapping data 108. The application data 106 may also include other information associated with the application 104, such as application version number information, information associated with the supported operating system of the application 104 (e.g., the version numbers of one or more operating systems), information associated with the supported device type of the client computing device, and deobfuscation data or files associated with the code of the application 104.

[0019] The application server system 102 determines a match between the library-dependent source code and at least one part of the application error data 122 based on the application error data 122 and the mapping data 108. For example, the library attribution module 114 of the library error data generation unit 110 on the application server system 102 may determine this match. When the application error data 122 provided by the application error reporting module 142 includes stack trace data, the library attribution module 114 may be configured to determine the match by collating one or more patterns of the library-dependent source code with one or more code positions in at least one part of the stack trace data based on this stack trace data and the mapping data 108. As a result, the library attribution module 114 may identify a mapping between the code in the application 104 and any of the third-party libraries 138. This mapping provides a mechanism for the library attribution module 114 to determine whether the stack trace traverses any third-party library-dependent source code, and the library attribution module 114 may attribute one or more errors (e.g., an application crash) to the identified one of the third-party libraries 138.

[0020] In response to determining consistency, the library attribution module 114 may attribute at least one error that occurred during the execution of the application 104 to at least one third-party library included in the third-party library 138. The library attribution module 114 may then generate library error data 124 associated with the at least one third-party library, and the library error data 124 may include at least one portion of the application error data 122. For example, if the application error data 122 includes application crash report data, the library error data 124 may include library crash report data that includes at least one portion of the application crash report data.

[0021] After the library attribution module 114 generates library error data 124, the library error reporting module 121 of the application server system 102 transmits the library error data 124 to one or more of one or more third-party library development systems 132 that develop at least one third-party library via the network 126. The at least one third-party library is included in the third-party library 138. For example, the library error reporting module 121 may transmit the library error data 124 to one or more library console interfaces 134 of or associated with the third-party library development system 132. In some examples, the library console interface 134 may locally store the library error data 124 on the third-party library development system 132. The library console interface 134 may also output the library error data 124 for display on one or more display devices 130, as shown in the examples of FIGS. 9-10. The display device 130 may include, in some examples, a liquid crystal display (LCD), a dot matrix display, a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a micro light emitting diode (micro LED) display, an active matrix organic light emitting diode (AMOLED) display, e-ink, or one or more of similar monochrome or color displays capable of outputting visual information to third-party developers using the third-party library development system 132.

[0022] In some examples, the library error reporting module 121 may include or provide an application programming interface (API) that is called by the library console interface 134 to output library error data 124 for display on the display device 130. The application server system 102 may be configured to send library error data 124 to a third-party library development system 132 via the library error reporting module 121. The third-party library development system 132 may execute the library console interface 134 to provide a graphical user interface (e.g., within a browser) that may output library error data 124 for display to one or more developers using the display device 130. The library console interface 134 may, in some cases, include executable code provided by the application server system 102 to the third-party library development system 132 for execution.

[0023] In some cases, the third-party library development system 132 may also store library data 136 that may include metadata associated with the third-party library 138. For example, as further described with reference to FIG. 2, the library data 136 may include obfuscation removal data usable by an optional obfuscation removal module 112 of the application server system 102 to de-obfuscate one or more portions of the application error data 122 (e.g., stack trace data) received from the client computing device 140, and the de-obfuscated portions are then used by the library error data generation unit 110 to generate library error data 124. This obfuscation removal data is associated with the library-dependent source code of the application 104. In some cases, the application data 106 provided to the application server system 102 by the application development system 128 may also include obfuscation removal data usable by the obfuscation removal module 112 to de-obfuscate one or more portions of the application error data 122, and such obfuscation removal data is associated with the application-dependent source code of the application 104.

[0024] Often, the application error data 122 (e.g., stack trace data) provided by the application error reporting module 142 may include application-specific data for the application 104 and / or confidential internal data associated with the application 104. Additionally, the application error data 122 may have variability associated with execution errors of multiple different applications 104 of the application 104 because different applications may potentially utilize faulty library-dependent source code in different contexts. Further, the application error data 122 may optionally include an indication of library-dependent source code for multiple different third-party libraries of the third-party libraries. As a result, in these cases, the library error data generation unit 110 may determine to remove one or more portions of the application error data 122 when generating the library error data 124 provided to the third-party library development system 132.

[0025] For example, the scrubbing module 116 of the library error data generation unit 110 may be configured to remove all application-dependent information (e.g., application-dependent source code) from the application error data 122 when generating the library error data 124. As a result, the third-party library development system 132 does not receive any such application-dependent information within the library error data 124 for use or display by the library console interface 134.

[0026] When the application error data 122 includes library-dependent source code for a plurality of different third-party libraries among the third-party libraries 138 developed by the third-party library development system 132, the scrubbing module 116 may also, when generating the library error data 124, remove one or more portions of such library-dependent source code, such as any library-dependent source code not associated with the identified third-party library that the library attribution module 114 attributes the current error to. For example, the application error data 122 may have at least one portion of data including the first library-dependent source code associated with the first third-party library 138 of the third-party library, and the application error data 122 may also have at least one portion of data including the second library-dependent source code associated with the second third-party library 138 of the third-party library. When the library attribution module 114 attributes at least one error associated with the application error data 122 to the first third-party library, the scrubbing module 116 may remove the second library-dependent source code from the application error data 122 (e.g., stack trace data) when generating the library error data 124, as shown in FIGS. 7-8.

[0027] As shown in FIG. 1, the library error data generation unit 110 also includes a clustering module 118. As will be described in more detail below, in various examples, the clustering module 118 may be configured to cluster portions of application error data 122 (e.g., stack trace data) generated by a plurality of different applications of an application 104 that may be developed by an application development system 128. These applications 104 may be developed by different application developers. The clustered portions of the application error data 122 may include, for example, clustered or aggregated groups of different application error reports having similar features or characteristics. The application error reporting modules 142 of the client computing devices 140 are all configured to provide the application error data 122 to the application server system 102, and since the application error data 122 is associated with errors that occur during the execution of potentially multiple different applications 104 developed by different application developers, the application server system 102 may be able to aggregate and cluster such data into library error data 124. As a result, a library developer who develops a third-party library 138 may view such aggregated and / or clustered information regarding the library developed by the library developer via the library console interface 134.

[0028] For example, if a plurality of different applications 104 among the applications 104 developed by a plurality of different application developers each include a potentially faulty third-party library included in a third-party library 138 provided by a third-party library development system 132, one or more of these applications 104 may experience similar errors or crashes during execution. As a result, similar application error data 122 may be generated when an error occurs in such an application 104, and the clustering module 118 of the application server system 102 may identify similar library-dependent source code in portions of the application error data 122 corresponding to each of the errors that occur during execution of these applications 104. Next, the library attribution module 114 may generate library error data 124 including clustered error data (e.g., stack trace data) that identifies library-dependent source code associated with the potentially faulty third-party library. The library error reporting module 121 may transmit such library error data 124 to the third-party library development system 132 so that the library developer of this library can view the library error data 124 using the library console interface 134. This library error data 124 may also include additional information that may be useful to the developer, such as the number of different applications that experience an error when using this library, the type of client computing device 140 used during application execution, the operating system of the client computing device 140, and the application or library version number.

[0029] As will be outlined in more detail below, the scrubbing module 116 may first remove application-dependent and / or other library-dependent source code information from the application error data 122, as described above, and the clustering module 118 may then cluster similar application error data 122 for one or more applications into library error data 124. In some cases, to identify similar features or characteristics of application error data 122 generated for different applications, the clustering module 118 may determine that the error is For different applications 104 that occur, fingerprint identifiers (e.g., hash identifiers) may be generated for portions of the application error data 122 generated, and these portions of the application error data 122 may be stored or included in the library error data 124. Then, the clustering module 118 may cluster the application error data 122 (e.g., in real time, more than once a day via batch processing of the application error data 122) when generating the library error data 124 based on matching fingerprint identifiers of these portions of the application error data 122. For example, using the example outlined above, multiple different applications 104 that include a potentially faulty third-party library may each be associated with portions of the application error data 122 that include or identify similar portions of the library-dependent source code of this library. The clustering module 118 may generate similar or matching fingerprint identifiers for these portions of the application error data 122, and the fingerprint identifiers may identify or be associated with this portion of the library-dependent source code. Then, the clustering module 118 may include this portion of the application error data 122 that includes or matches the library-dependent source code within the library error data 124. The library error data 124 may also include this code and may include information indicating the number of applications that have experienced the error.

[0030] For example, the clustering module 118 may determine a first fingerprint identifier associated with at least one portion of the application error data included in the library error data 124. This at least one portion may include library-dependent source code. The clustering module 118 may also determine a match between the first fingerprint identifier and a second fingerprint identifier associated with previously generated library error data included in the library error data 124, where the previously generated library error data is associated with at least one third-party library. This previously generated library error data is also associated with at least one portion (e.g., the same or similar library-dependent source code) of the application error data associated with at least one error that occurred during one or more executions of the application 104 on the client computing device 140. In response to determining the match, the clustering module 118 may cluster the library error data with the previously generated library error data to generate clustered library error data within the library error data 124, where the clustered library error data is associated with at least one third-party library. In some cases, as described in further detail below, the clustered library error data indicates at least some of the applications affected by at least one error. The library error reporting module 121 may then send the clustered library error data to the third-party library development system 132.

[0031] In some cases, the clustering module 118 may execute one or more application anonymization functions regarding the applications of the application development system 128 and the application developers. For example, when including library error information associated with one or more of the applications 104 in the library error data 124, the clustering module 118 may refrain from including any specific information or identification information regarding those of the applications 104 that are associated with the library error data 124. Instead, the clustering module 118 may include only general information regarding such applications 104, such as the number of applications in general. In some cases, the clustering module 118 may be by the client computing device 140 (e.g., by the application error reporting module 142) the application Depending on the type and details of the information included in the application error data 122 provided to the application server system 102, further information such as the type of application, one or more version numbers of one or more third-party libraries 138, application information, at least one version number of at least one operating system executed by the client computing device 140, information regarding the client computing device 140 (e.g., operating system or device type information), etc. may be included. In addition, as will be described in more detail below, the clustering module 118 may also include in the library error data 124, for a certain period (e.g., 30 days, 60 days, all cumulative time up to the present, etc.), the number of applications 104 affected by at least one error, the number of users affected by at least one error over that period, or the number of occurrences of at least one error over that period, one or more of which may be included.

[0032] In some cases, the clustering module 118 may execute one or more application thresholding functions. For example, the clustering module 118 may simply include in the library error data 124 error data that affects applications exceeding a threshold number in order to further potentially preserve the anonymity of applications experiencing errors. In some cases, only errors (e.g., crashes) observed across applications exceeding this threshold number may be reported to the third-party library development system 132. These errors may be related to common problems or bugs that are of particular interest to the library developers of one or more third-party libraries associated with the library error data 124. Thus, in one or more examples, the application server system 102 may determine that the number of applications 104 affected by at least one error exceeds a threshold number, and the library error reporting module 121 may be configured only to send the clustered library error data 124 in response to determining that the number of applications affected by at least one error exceeds a threshold number. In some cases, this threshold number may be a predetermined number or a default number of applications.

[0033] According to the techniques disclosed herein, the storage of certain data, such as application data 106, library data 136, application error data 122, and / or library error data 124, may, in various examples, occur only in response to a computing device or system (e.g., client computing device 140 and / or systems 102, 128, 132) receiving affirmative consent or response from one or more users, such as one or more application or library developers or end users. The user may be provided with controls that enable the user to make selections regarding both whether the systems, programs, or features described herein can enable the collection and / or storage of information and when they can do so, and / or whether the systems, programs, or features described herein can enable the transmission of content or communication between devices and when they can do so. Additionally, as described above, certain data (e.g., library error data 124) may be processed in one or more ways before storage or use such that identifiable information associated with the application 104 is removed.

[0034] FIG. 2 is a block diagram showing another exemplary distributed system 200 that includes an application server system 202 configured to process application crash report data 222 for use by a third-party software development kit (SDK) development system 232, according to one or more aspects of the present disclosure. System 200 is an example of system 100, and the various components shown in FIG. 2 provide similar functionality to the similarly numbered components shown in FIG. 1.

[0035] For example, system 200 includes one or more application server systems 202 communicatively coupled to one or more application development systems 228, one or more SDK development systems 232, and one or more client computing devices 240 via one or more networks 226. The application server system 202 is an example of the application server system 102 shown in FIG. 1; the network 226 is an example of the network 126; the application development system 228 is an example of the application development system 128; the SDK development system 232 is an example of the third-party library development system 132; and the client computing device 240 is an example of the client computing device 140.

[0036] FIG. 2 shows an example where a third-party library includes an SDK 238 developed by a third-party SDK developer using an SDK development system 232. In various examples, an SDK includes a collection of software development tools or components provided in an installable library. As shown in FIG. 2, the SDK development system 232 includes one or more SDKs 238 that may be developed by one or more third-party SDK developers using the SDK development system 232. Various different applications, such as an application 204 developed using an application development system 228, may include one or more of the SDKs 238. Similar to the third-party library development system 132 of FIG. 1, the SDK development system 232 of FIG. 2 includes an SDK 238, SDK data 236, SDK crash report data 224, one or more SDK console interfaces 234, and one or more display devices 230. The SDK crash report data 224 is an example of library error data 124, and one or more of the errors associated with the execution of the application 204 include one or more application crashes. In some cases, the SDK data 236 provided to the application server system 202 by the SDK development system 232 may include, as described above, de-obfuscation data associated with the library-dependent source code of the SDK 238. The application server system 202 may locally store such data in de-obfuscation data 209. The de-obfuscation module 212 of the SDK crash report generation unit 210 may use such de-obfuscation data 209 to de-obfuscate one or more portions (e.g., library-dependent source code) of the application crash report data 222 so that the SDK crash report generation unit 210 can generate the SDK crash report data 224.

[0037] Similar to the application development system 128 shown in FIG. 1, the application development system 228 of FIG. 2 includes an application 204 and corresponding application data 206 including mapping data 208. One or more application developers may use the application development system 228 to create the application 204. In some cases, the application data 206 provided to the application server system 202 by the application development system 228 may also include obfuscation removal data associated with the application-dependent source code of the application 204. The application server system 202 may locally store such data in obfuscation removal data 209.

[0038] Similar to the client computing device 140 shown in FIG. 1, the client computing device 240 of FIG. 2 includes the application 204. One or more of the applications 204 may crash or otherwise terminate execution, and the application crash report data 222 may include data associated with these crashes. The application crash report data 222 is an example of the application error data 122 shown in FIG. 1. One or more application crash reporting modules 242 are configured to send the application crash report data 222 to the application server system 202. port data 222 to the application server system 202.

[0039] The application server system 202 includes an application 204 and corresponding application data 206. The application data 206 includes mapping data 208 and optional decryption data 209. The application server system 202 also includes an SDK crash report generation unit 210, an application error processing module 220, an SDK crash reporting module 221, application crash report data 222, and SDK crash report data 224 (these are examples of a library error data generation unit 110, an application error processing module 120, a library error reporting module 121, application error data 122, and library error data 124, respectively).

[0040] Similar to the library error data generation unit 110, the SDK crash report generation unit 210 includes an SDK attribution module 214, a scrubbing module 216, a clustering module 218, and an optional de-obfuscation module 212. The SDK crash report generation unit 210 is configured to generate SDK crash report data 224 based on application crash report data 222 that is transmitted by the application crash reporting module 242 and processed by the application error processing module 220 of the application server system 202. The application crash report data 222 may include stack trace data 223 provided by the application crash reporting module 242, and the SDK crash report data 224 generated by the SDK crash report generation unit 210 may also include stack trace data 225. The stack trace data 225 may include one or more portions of the stack trace data 223, as described above with reference to the library error data 124 and the application error data 122 of FIG. 1. The SDK crash reporting module 221 of the application server system 202 may transmit the SDK crash report data 224 to the SDK development system 232, and then the SDK development system 232 may display one or more portions of this display on the display device 230.

[0041] FIG. 3 is a block diagram showing an exemplary application server system 302 according to one or more aspects of the present disclosure. The application server system 302 may be an example of the application server system 102 (FIG. 1) and / or the application server system 202 (FIG. 2). FIG. 3 shows only one specific example of the application server system 302, and many other examples of the application server system 302 may be used in other examples. In various cases, the application server system 302 may include a subset of the components shown in FIG. 3, or may include additional components not shown in FIG. 3.

[0042] In the example of FIG. 3, the application server system 302 includes a display device 348, one or more processors 344, one or more input components 346, one or more communication units 350, one or more output components 354, and one or more storage devices 358. The communication channel 347 may interconnect each of the components 344, 346, 348, 350, 354, and / or 358 for (physically, communicatively, and / or operably) component - to - component communication. In some examples, the communication channel 347 may include a system bus, a network connection, an inter - process communication data structure, or any other means for communicating data between hardware and / or software.

[0043] One or more input components 346 of the application server system 302 may receive inputs such as user input. Examples of inputs are touch / haptic, presence sensing, and voice input. Examples of input components 346 include a presence - sensing screen, a touch - sensing screen, a touch screen, a mouse, a keyboard, a trackpad, a voice - response system, a video camera, a microphone, or any other type of device for detecting input from a human or a machine.

[0044] One or more output components 354 of the application server system 302 may generate outputs. Examples of outputs are tactile output, audio output, and visual output. Examples of output components 354 include a presence - sensing screen, a touch - sensing screen, a touch screen, a sound card, a video graphics adapter card, a speaker, a liquid crystal display (LCD), an organic light - emitting diode (OLED) display, a micro - light - emitting diode (micro - LED) display, an active - matrix organic light - emitting diode (AMOLED) display, a tactile device, or any other type of device for generating output to a human or a machine.

[0045] One or more communication units 350 of the application server system 302 may communicate with external devices via one or more networks by transmitting and / or receiving network signals on one or more networks (e.g., one or more wired and / or wireless networks). For example, the application server system 302 may use the communication unit 350 to transmit and / or receive wireless signals on a wireless network such as a cellular wireless network. Similarly, the communication unit 350 may transmit and / or receive satellite signals on a satellite network such as a Global Positioning System (GPS) network. Examples of the communication unit 350 include a network interface card (e.g., an Ethernet (registered trademark) card, etc.), an optical transceiver, a radio frequency transceiver, a GPS receiver, or any other type of device capable of transmitting and / or receiving information. Other examples of the communication unit 350 may include a shortwave radio, a cellular data radio, a wireless Ethernet network radio, and a Universal Serial Bus (USB) controller.

[0046] In some cases, the application server system 302 may include a display device 348. In some examples, the display device 348 may provide output to the user using tactile, audio, or visual stimuli, as described above with reference to the output component 354. For example, the display device 348 may provide a display or video output, as described with reference to the output component 354. The display device 348 may also provide an input function as described above with reference to the input component 346. In some cases, the application server system 302 may not include the display device 348.

[0047] One or more storage devices 358 may store information for processing during the operation of the application server system 302. In some examples, the storage device 358 includes a temporary memory, which means that the primary purpose of the storage device 358 is not long-term storage. The storage device 358 on the application server system 302 may be configured for short-term storage of information as volatile memory, and thus does not retain the stored content when the power is turned off. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory known in the art.

[0048] In some examples, the storage device 358 includes one or more computer-readable storage media. The storage device 358 may be configured to store a larger amount of information than volatile memory. The storage device 358 may further be configured as non-volatile memory space for long-term storage of information and may retain the information after power-on / off cycles. Examples of non-volatile memory include magnetic hard disks, optical disks, floppy (registered trademark) disks, flash memory, or forms such as electrically programmable memory (EPROM) or electrically erasable programmable (EEPROM) memory. The storage device 358 may include examples of components or modules with similar numbers shown in FIGS. 1-2, and may store program instructions and / or data associated with one or more of the application 304, application data 306, mapping data 308, library error data generation unit 310, application error processing module 320, library error reporting module 321, application error data 322, and library error data 324. The library error data generation unit 310 includes a decryption module 312, a library attribution module 314, a scrubbing module 316, and a clustering module 318.

[0049] One or more processors 344 may implement functions and / or execute instructions within the application server system 302. For example, a processor 344 on the application server system 302 may receive and execute instructions stored by a storage device 358 that implement the functions of an application 304, a library error data generation unit 310 (including a deobfuscation module 312, a library attribution module 314, a scrubbing module 316, and a clustering module 318), an application error processing module 320, and / or a library error reporting module 321. These instructions executed by the processor 344 may cause the application server system 302 to store information in the storage device 358 during program execution.

[0050] FIG. 4 is a conceptual diagram showing exemplary stack trace data 423 that includes both library-dependent source code and application-dependent source code according to one or more aspects of the present disclosure. The stack trace data 423 is an example of the stack trace data 223 shown in FIG. 2 and may be included in the application crash report data 222 provided by the client computing device 240 to the application server system 202. The stack trace data 423 is associated with at least one error that occurred during one execution of, for example, the application 204 shown in FIG. 2. For purposes of illustration only, the related examples shown in FIGS. 4-8 will be described with reference to the system 200 shown in FIG. 2.

[0051] As shown in FIG. 4, the stack trace data 423 includes an error identifier (e.g., an exception identifier) 460 that identifies one or more errors associated with the stack trace data 423. The stack trace data 423 also includes a plurality of code portions, namely, library-dependent source code 461, application-dependent source code 462, and library-dependent source code 463. The stack trace data 423 can represent nested function calls having a call order 464, which may lead to the location where a crash occurred during the execution of one or more of the applications (e.g., application 204 shown in FIG. 2), for example, from a program loader. The call order 464 of the function calls that may have led to the possible crash is from bottom to top in the example of FIG. 4 (e.g., in the order of library-dependent source code 463, application-dependent source code 462, and library-dependent source code 461).

[0052] In various examples, each line of the stack trace data 423 shown in FIG. 4 can be referred to as a frame or a code location of the stack trace. In some cases, each frame or code location of the stack trace may indicate information associated with the source code of that frame of the stack trace (e.g., library-dependent or application-dependent source code). For example, this information may include a library name or identifier. In some examples, the name or identifier may include a package name, a namespace name, a module name, etc. In some examples, the information for each frame or code location may separately or, in any combination, include additional information such as one or more of a class name, a method name, and / or a function name. In some cases, each frame or code location of the stack trace may further indicate the file name of the file containing the source code and / or the line number of the source code for that frame.

[0053] The application-dependent source code 462 is associated with code specific to a particular application among, for example, the applications 204. The library-dependent source code 461 is associated with code from, for example, the first SDK (named "examplehttp") of the SDK 238 shown in FIG. 2. The library-dependent source code 463 is associated with code from a second SDK named "OS.app" of the SDK 238, where "OS" may represent the name of the operating system in this particular example.

[0054] FIG. 5 is a conceptual diagram showing exemplary mapping data 508 that provides a mapping between one or more portions of the library-dependent source code of an application and at least one third-party library into which the library-dependent source code is loaded during execution of the application, according to one or more aspects of the present disclosure. The mapping data 508 may be an example of the mapping data 208 shown in FIG. 2, included in the application data 206 provided to the application server system 202 by the application development system 228. The use of such mapping data enables, in various cases, the SDK crash report generation unit 210 to determine whether stack trace data, such as the stack trace data 423, traverses any library code of the application within the application 204, such that the SDK attribution module 214 may attribute the crash to the library corresponding to such code.

[0055] In some cases, the application developer and / or the application development system 228 may develop or create mapping data 508 associated with each of the developed applications 204 deployed on the application server system 202 shown in FIG. 2. The application development system 228 may provide such mapping data 508 (which is an example of the mapping data 208 shown in FIG. 2) to the application server system 202 for use by the SDK crash report generation unit 210. In some cases, the application server system 202 may obtain one or more portions of the mapping data 508 from one or more external sources or repositories (e.g., a publicly accessible repository that provides such information regarding the application 204).

[0056] As shown in FIG. 5, the mapping data 508 provides a mapping between one or more portions of the library-dependent source code of an application among the applications 204 and at least one third-party library (e.g., an SDK) into which the library-dependent source code is loaded during the execution of the application. In FIG. 5, an exemplary mapping between an identified portion or pattern of the library-dependent source code and a first SDK ("examplehttp") is shown. An exemplary mapping is shown.

[0057] For example, the mapping data 508 maps a first portion or pattern 570 of the library-dependent source code of the application to the SDK identifier 571 of this first SDK. As shown in FIG. 5, the pattern 570 identifies library-dependent source code whose package name is "okttp3" and whose class name is "ExampleHttpClient". The "*" wildcard character in the pattern 570 indicates that the library with this package and class name Indicates that any function / method in the library-dependent source code is mapped to the SDK identifier 571. The mapping data 508 maps the pattern 570 of the library-dependent source code to a first SDK having the SDK identifier 571. The SDK identifier 571 identifies this first SDK ("examplehttp"), and the SDK identifier 571 may also identify the SDK name and version number ("4.8.1").

[0058] The mapping data 508 also maps a second part or pattern 572 of the library-dependent source code to the SDK identifier 571 of the first SDK. The pattern 572 identifies library-dependent source code having the package name "examplehttp" and the class name "ExampleHttpClient$Builder". The "*" wildcard character in the pattern 572 indicates that any function / method in the library-dependent source code with this package and class name is mapped to the SDK identifier 571.

[0059] The mapping data 508 also maps a third part or pattern 573 of the library-dependent source code to the SDK identifier 571 of the first SDK. The pattern 573 identifies library-dependent source code associated with platform code having the package name "examplehttp" and the class name "OSPlatform". The "*" wildcard character in the pattern 573 indicates that any function / method in the library-dependent source code with this package and class name is mapped to the SDK identifier 571.

[0060] The mapping data 508 also maps a fourth part or pattern 574 of the library-dependent source code to the SDK identifier 571 of the first SDK. The pattern 574 has the package name "examplehttp" and the class name "Platform" for the platform​ Identify the library-dependent source code associated with the code. The "*" wildcard character in pattern 574 represents library-dependent source code with this package and class name Indicates that any function / method in the code is mapped to the SDK identifier 571. Thus, in the example of FIG. 5, a portion of the library-dependent source code or all of patterns 570, 571, 572, 573 are mapped to the same first SDK ("examplehttp") be mapped

[0061] FIG. 6 is a conceptual diagram showing exemplary stack trace data 625 included in an SDK crash report provided to one or more SDK development systems 232 according to one or more aspects of the present disclosure. In various examples, the SDK crash report generation unit 210 may generate (e.g., using the SDK attribution module 214) SDK crash report data 224 including the stack trace data 625 shown in FIG. 6. The SDK crash reporting module 221 may then transmit this SDK crash report data 224 to the SDK development system 232 for output to the display device 230.

[0062] In various examples, as described above, the SDK attribution module 214 may attribute a crash to one or more SDKs included in the SDK 238. These SDKs may include library-dependent source code that is the cause of this crash. The SDK attribution module 214 may determine a match between the library-dependent source code of the application and at least one portion of the stack trace data 423 (FIG. 4) based on the stack trace data 423 (FIG. 4) and the mapping data (including the mapping data 508 shown in FIG. 5). For example, based on the mapping data 508 and the stack trace data 423, the SDK attribution module 214 may attribute the first SDK ("examplehttp") to the crash It may be identified as a possible cause of the crash. In addition, based on the stack trace data 423 and additional mapping data related to a second SDK ( "OS.app") not shown in FIG. 5, the SDK attribution module 214 may identify the second SDK as another possible cause of the crash. Thus, the SDK attribution module 214 identifies these first and second SDKs as SDK candidates 678 that may have caused the crash. In some examples, one or more of such SDK candidates may be involved in any given crash. Since the first SDK is located at the top of the stack trace data 423 shown in FIG. 4, the SDK attribution module 214 may, in some cases, rank the first SDK as the first, or potentially stronger, candidate related to the second SDK and involved in the crash. In some examples, one or more of such SDK candidates may be involved in any given crash. Since the first SDK is located at the top of the stack trace data 423 shown in FIG. 4, the SDK attribution module 214 may, in some cases, rank the first SDK as the first, or potentially stronger, candidate related to the second SDK and involved in the crash.

[0063] To identify these SDK candidates 678, in various examples, the SDK attribution module 214 may analyze the mapping data and attempt to match portions or patterns in such data (e.g., patterns 570, 572, 573, 574 in the mapping data 508) with source code included in frames or code positions of the stack trace data 423 received from the client computing device 240. Upon finding one or more matches, the SDK attribution module 214 may then identify a third - party library corresponding to the library identifier in the mapping data (e.g., the first SDK "examplehttp" associated with the SDK identifier 571 in the mapping data 508) as a library that may be a potential cause of the crash. It may be identified as a library that may be a potential cause of the crash.

[0064] In one or more specific non - limiting examples, the SDK attribution module 214 may perform such a process to identify the SDK candidates 678 based on the following pseudocode: set <library>matched_libraries = {}; for (string code_location in input_stack_trace.code_locations) { for (string library_pattern in mapping.library_patterns) { if (library_pattern.matches(code_location)) { matched_libraries.add(library_pattern.library); } } } In this pseudocode, input_stack_trace corresponds to stack trace data 423, and input_stack_trace.code_locations corresponds to each frame or code location of the stack trace data 423. The mapping.library_patterns in the pseudocode may correspond to patterns 570, 572, 573, 575 of the mapping data 508. As described above, the SDK attribution module 214 may analyze the mapping data and attempt to match parts or patterns in such data (e.g., patterns 570, 572, 573, 574 within the mapping data 508) with the source code included in the frames or code locations of the stack trace data 423 received from the client computing device 240. When such a match is found, the SDK attribution module 214 may add each identified library (e.g., library identifier) to the set of matched_libraries (e.g., SDK candidate 678).

[0065] After performing the matching operation, the SDK crash report generation unit 210 is associated with the first SDK ("examplehttp") and the second SDK ("OS.app"), and the SDK crash You may generate the stack trace data 625 shown in FIG. 6 included in the report data 224. The stack trace data 625 may include one or more portions of the stack trace data 423. The SDK crash reporting module 221 may send this SDK crash report data 224 including the stack trace data 625 to the SDK development system 232.

[0066] For example, as shown in FIG. 6, the stack trace data 625 includes an error identifier 660, library dependent source code 661 associated with the first SDK (“examplehttp”) of the SDK 238, and library dependent source code 663 associated with the second SDK (“OS.app”). The error identifier 660 corresponds to the error identifier 460 of the stack trace data 423 in FIG. 4, the library dependent source code 661 corresponds to the library dependent source code 461, and the library dependent source code 663 corresponds to the library dependent source code 463. However, as shown in FIG. 6, the stack trace data 625 does not include the application dependent source code 462. Instead, the stack trace data 625 includes a general or uniform placeholder 676. FIG. 6 shows one or more “-” or “_” characters as the placeholder 676, but in other examples, any other form of symbol or alphanumeric (e.g., “

[0067] <private>") may be used. As described above, in various cases when generating library error data such as the stack trace data 625 shown in FIG. 6, the scrubbing module 216 may be configured to remove specific information from the stack trace data 423.

[0068] For example, after identifying the SDK candidate 678, the scrubbing module 216 may wipe out confidential or specific information associated with the identity or internal details of the crashed application. Thus, in various cases, the scrubbing module 216 may be configured to remove all application-dependent information (e.g., the application-dependent source code 462) from the stack trace data 423 when generating the stack trace data 625 and replace this information with a placeholder 676. As a result, the SDK development system 232 does not receive any such application-dependent information within the SDK crash report data 224 for use or display by the SDK console interface 234.

[0069] The SDK crash report generation unit 210 (e.g., using the SDK attribution module 214 and / or the clustering module 218) may, in some cases, be configured to generate a unique fingerprint identifier (ID) 677, such as a unique hash identifier, associated with the content of the stack trace data 625. The fingerprint ID 677 may be used to aggregate and / or cluster the stack trace data 625 with other similar stack trace data that may be associated with other application crashes, as described in more detail below. The SDK crash reporting module 221 includes this SDK crash report data 224 including the stack trace data 625 in the first SDK ("examplehttp") included in the SDK 238 and the It may also be sent to the SDK development system 232 associated with the SDK of 2 (「OS.app」). Next, the SDK developers of these first and / or second SDKs may examine the SDK crash report data 224 using the SDK console interface 234.

[0070] However, in other cases, the scrubbing module 216 may be further configured to remove information associated with other SDKs from the stack trace data 625. Thus, in the example of FIG. 6 where the SDK attribution module 214 identifies the first and second SDKs as SDK candidates 678, the scrubbing module 216 may generate two separate instances of stack trace data associated with the first and second SDKs. The first stack trace data may include only the library-dependent source code associated with the first SDK, and the second stack trace data may include only the library-dependent source code associated with the second SDK. Both the first and second stack trace data may be stored in the SDK crash report data 224. The SDK crash reporting module 221 may send the portion of the SDK crash report data 224 that includes the first stack trace data to one or more of the SDK development systems 232 that develop the first SDK of the SDK 238, and the SDK crash reporting module 221 may send the portion of the SDK crash report data 224 that includes the second stack trace data to one or more of the SDK development systems 232 that develop the second SDK This is also acceptable. Next, each of the SDK developers of these first and second SDKs may examine the respective first and second stack trace data associated with the application crash. FIGS. 7-8 provide examples of such first and second stack trace data. By using this approach, the scrubbing module 216 is configured to remove SDK information regarding the second SDK when providing the first stack trace data to the developer of the first SDK, and to remove SDK information regarding the first SDK when providing the second stack trace data to the developer of the second SDK.

[0071] FIG. 7 is a conceptual diagram showing another example of stack trace data 725 included in an SDK crash report (e.g., SDK crash report data 224) provided to one or more SDK development systems 232 according to one or more aspects of the present disclosure. If the stack trace data 423 of FIG. 4 includes library dependency source code for a plurality of different third-party libraries among the third-party libraries 238 developed by the third-party library development system 232 (FIG. 2), the scrubbing module 216 may be configured to remove one or more portions of such library dependency source code, such as any library dependency source code not associated with the identified third-party library to which the library attribution module 214 attributes the current error, when generating the stack trace data 725.

[0072] Thus, in the example of FIG. 7, the scrubbing module 216 includes the application-dependent source code 462 and the first identified SDK (library) candidate "examplehttp" Remove all frames of the stack trace data 423 provided by the client computing device 240, including any library-dependent source code not associated therewith. Thus, when generating or updating the stack trace data 725, the scrubbing module 216 removes the library-dependent source code 463 associated with the second identified SDK (library) candidate "OS.app". As shown in FIG. 7, the stack trace data 725 includes an error identifier 760 and library-dependent source code 761 associated with the first SDK candidate "examplehttp". Class The clustering module 218 may also be used to cluster the stack trace data 725 with other similar stack trace data associated with other application crashes, as described in more detail below, and may generate a unique fingerprint identifier 777 for the stack trace data 725.

[0073] FIG. 8 is a conceptual diagram showing another example of stack trace data 825 included in an SDK crash report (e.g., SDK crash report data 224) provided to one or more SDK development systems 232 according to one or more aspects of the present disclosure. If the stack trace data 423 of FIG. 4 includes library-dependent source code for a plurality of different third-party libraries of the third-party library 238 developed by the third-party library development system 232 (FIG. 2), the scrubbing module 216, when generating the stack trace data 825, may also be configured to remove one or more portions of such library-dependent source code, such as any library-dependent source code not associated with the identified third-party library to which the library attribution module 214 attributes the current error.

[0074] Accordingly, in the example of FIG. 8, the scrubbing module 216 removes all frames of the stack trace data 423 provided by the client computing device 240 that include the application-dependent source code 462 and any library-dependent source code not associated with the second identified SDK (library) candidate "OS.app". Accordingly, when generating or updating the stack trace data 825, the scrubbing module 216 removes the library-dependent source code 463 associated with the first identified SDK (library) candidate "ex amplehttp". As shown in FIG. 8, the stack trace data 825 includes the error identifier 860 and the library-dependent source code 863 associated with the second SDK candidate "OS.app". The clustering module 218 may also generate a unique fingerprint identifier 877 for the stack trace data 825, which may be used to cluster the stack trace data 825 with other similar stack trace data associated with other application crashes. As a result, the scrubbing module 216 may generate two separate instances of stack trace data associated with these first and second SDKs. The first stack trace data may include only the library-dependent source code associated with the first SDK, and the second stack trace data may include only the library-dependent source code associated with the second SDK. In one or more specific non-limiting examples, the scrubbing module 216 may execute a process for generating such stack trace data for different candidate SDKs for inclusion in the SDK crash report data 224 sent to the SDK development system 232, similar to or based on the following pseudocode:

[0075] list<stack_trace> library_stack_traces = []; ​for (library in matched_libraries) { library_stack_trace = library_stack_traces.add_new(); last_matched = false; for (code_location in input_stack_trace.code_locations) { if (library.matches(code_location) { last_matched = true; library_stack_trace.append(code_location) } else if (last_matched) { last_matched = false; library_stack_trace.append('_') } } } In this pseudocode, matched_libraries is a list of those SDK candidates, namely, The first SDK ("examplehttp") and the second SDK ("OS.app"). The code is listed in library_stack_trace as follows: For example, the scrubbing module 216 creates a first library stack trace data 725 (library_stack_trace) and adds it to a list of library_stack_trace that includes the library-dependent source code 761 for the first SDK in the stack trace data 725 generated from the input stack trace data 423 (input_stack_trace). The mangling module 216 traverses each frame or code_location of the stack trace data 423 to identify the library-dependent source code 461 of the first SDK. Upon identifying such code, the scrubbing module 216 adds or appends this code as library-dependent source code 761 to the first library stack trace data 725 (library_stack_trace) in the list. For any other library-dependent source code (e.g., library-dependent source code 463) or any application-dependent source code (e.g., application-dependent source code 462), the scrubbing module 216 either completely removes this code within the stack trace data 725 as shown in FIG. 7, or replaces such code with one or more uniform placeholders (e.g., the "_" character) (not shown in FIG. 7).

[0076] Similarly, the scrubbing module 216 creates a second library stack trace data 825 (library_stack_trace) and appends it to the list of library_stack_trace that includes the library-dependent source code 863 for the second SDK in the stack trace data 825 generated from the input stack trace data 423 (input_stack_trace). The scrubbing module 216 traverses each frame or code_location of the stack trace data 423 to identify the library-dependent source code 4 63 of the first SDK. Upon identifying such code, the scrubbing module 216 adds or appends this code as library-dependent source code 863 to the first library stack trace data 825 (library_stack_trace) in the list. For any other la brary-dependent source code (e.g., library-dependent source code 463) or any application-dependent source code (e.g., application-dependent source code 462), the scrubbing module 216 either completely removes this code within the stack trace data 725 as shown in FIG. 7, or replaces such code with one or more uniform placeholders (e.g., the "_" character) (not shown in FIG. 7). Upon identifying such code, the scrubbing module 216 adds or appends this code as library-dependent source code 863 to the first library stack trace data 825 (library_stack_trace) in the list. For any other la For library-dependent source code (e.g., library-dependent source code 461) or any application-dependent source code (e.g., application-dependent source code 462), the scrubbing module 216, as shown in FIG. 8, completely removes this code within the stack trace data 825 or replaces such code with one or more uniform placeholders (e.g., the "_" character) (not shown in FIG. 8).

[0077] As described above, the clustering module 218 may also generate a unique fingerprint identifier 777 for the stack trace data 725, which may be used to cluster the stack trace data 725 with other similar stack trace data associated with other application crashes. The clustering module 218 may similarly generate a unique fingerprint identifier 877 for the stack trace data 825, which may be used to cluster the stack trace data 825 with other similar stack trace data associated with other application crashes. The clustering module 218 may, in various cases, store the fingerprint identifiers 777 and 877 (e.g., in the SDK crash report data 224 or in an associated data store).

[0078] Each of the stack trace data 725 and 825 may be processed and clustered into groups of similar data or reports based on the identified fingerprint identifiers 777, 877 respectively. By using the clustering module 218 to perform one or more clustering operations, the SDK crash report generation unit 210 may process the application crash report data 222 associated with any number of different applications 204 developed by any number of different application developers on the application development system 228, and then may be configured to aggregate and cluster similar errors and crashes that occurred during the execution of these applications 204. Next, the SDK crash reporting module 221 may output the clustered SDK crash report data 224 associated with a specific identified SDK 238 to the SDK development system 232, and the data may then be displayed on the display device 230 via the SDK console interface 234 for consideration by the respective SDK developers.

[0079] In various cases, to perform such aggregation or clustering, the SDK crash report generation unit 210 and / or the SDK crash reporting module 221 may index or group and store the current stack trace data and historical stack trace data within the stack trace data 225 of the SDK crash report data 224 according to the fingerprint identifiers respectively associated with such stack trace data. Thus, as an example, the SDK crash report generation unit 210 may store the stack trace data 725 in the stack trace data 225. The clustering module 218 may also receive, within the SDK crash report data 224, an application from the application crash reporting module 242 associated with a specific crash that resulted in the stack trace data 725 One or more other portions of the crash report data 222 may be remembered. For example, if the application crash report data 222 includes the version number information of the operating system used by the application 204, the version number of any SDK included in the application 204, the device type of the client computing device 240, etc., the clustering module 218 may store such information within the SDK crash report data 224. In some cases, the clustering module 218 may also obtain SDK version information from the mapping data 208. All such information within the SDK crash report data 224 and the stack trace data 225 may be indexed or grouped according to the fingerprint identifier 777. In some cases, the clustering module 218 may also obtain SDK version information and / or SDK name information from the mapping data 208 and then store it in the SDK crash report data 224.

[0080] Similarly, the SDK crash report generation unit 210 may store the stack trace data 825 in the stack trace data 225. The clustering module 218 may also store, within the SDK crash report data 224, one or more other portions of the application crash report data 222 received from the application crash reporting module 242 that are associated with a particular crash that resulted in the generation of both the stack trace data 725 and the stack trace 825 (e.g., the version number of the operating system used by the application 204, the version number of any SDKs included in the application 204, the device type of the client computing device 240, etc.). This information within the SDK crash report data 224 and the stack trace data 225 may further be indexed or grouped according to the fingerprint identifier 877. Further, as described in more detail below, the clustering module 218 may also store, within the SDK crash report data 224, the number of applications 204 affected by at least one error over a period of time (e.g., 30 days, 60 days, all cumulative time up to the present, etc.), the number of users affected by at least one error over that period of time, or the number of occurrences of at least one error over that period of time, one or more of which may be included.

[0081] Over time, the SDK crash report generation unit 210 may process additional application crash report data 222 associated with further errors or crashes encountered during the execution of the application 204. These may include one or more of the applications 204 developed by one or more different application developers using the application development system 228. The SDK crash report generation unit 210 may generate additional stack trace data associated with these errors or crashes. The clustering module 218 may generate corresponding fingerprint identifiers for such stack trace data and compare these fingerprint identifiers with previously generated fingerprint identifiers associated with the stack trace data and other data previously stored in the SDK crash report data 224. If there is a match of the fingerprint identifiers, the clustering module 218 may perform one or more clustering operations to aggregate or cluster the current error or crash data with the previously collected data, and such clustering may indicate similarities between errors or crashes that occur over time in one or more of the applications 204 that utilize similar ones of the SDKs 238. The SDK crash reporting module 221 then outputs the SDK crash report data 224 to the SDK development system 232 so that individual SDK developers can view, via the SDK console interface 234, portions of the SDK crash report data 224 corresponding to one or more of the SDKs 238 that they developed and that may be the cause of one or more application errors.

[0082] Based on the executed clustering operation, the clustering module 218 may store various forms of clustering information in the SDK crash report data 234. For example, the clustering module 218 may associate one or more of the following information with the clustered data and include it in the SDK crash report data 234: the number of occurrences of similar errors (e.g., reports, problems, crashes) over a period of time, the number of applications (and / or users of the applications) that have experienced similar errors, the name of the library (e.g., SDK) associated with the error, the library version, the operating system and / or operating system version of the computing device where the error occurred, and / or the type of the client computing device where the error occurred. When receiving such aggregated and / or clustered data in the SDK crash report data 224, the SDK development system 232 may use the SDK console interface 234 to output one or more portions of such data to the display device 230, which may be reviewed by the respective SDK developers of the SDK in question. FIGS. 9-10 provide examples of such output.

[0083] As described above, in some cases, the clustering module 218 may execute one or more application anonymization functions with respect to the applications of the application development system 228 and the application developers. For example, when including error information (e.g., clustering information) in the SDK crash report data 224 associated with one or more of the applications 104, the clustering module 218 may refrain from including any specific or identifying information regarding those of the applications 204 that are associated with the SDK crash report data 224. Instead, the clustering module 218 may include only general information regarding such applications 204, such as the number of applications in general. In some cases, the clustering module 218 may include additional information, such as the type of application, information regarding the client computing device 140 (e.g., operating system or device type information), depending on the type and details of the information included in the application crash report data 222 provided to the application server system 202 by the client computing device 240 (e.g., by the application error reporting module 242).

[0084] In some cases, the clustering module 218 may execute one or more application thresholding functions, for example, as an additional form of anonymization. For example, the clustering module 218 may include error data affecting applications exceeding a threshold number within the SDK crash report data 224 to further potentially preserve the anonymity of one or more applications 204 experiencing errors. In some cases, only errors (e.g., crashes) observed across applications exceeding this threshold number may be reported to the SDK development system 232. These errors may be related to common problems or bugs that are of particular interest to the library developers of one or more SDKs 238 associated with the SDK crash report data 224. In some cases, this threshold may include a predetermined or default number of applications.

[0085] FIG. 9 is a screen diagram showing an exemplary graphical user interface 980 that may be displayed in one or more third-party library development systems based on data provided by an application server system according to one or more aspects of the present disclosure. For purposes of illustration only, FIG. 9 is described with reference to the SDK development system 232 and the application server system 202 shown in FIG. 2.

[0086] As described above, the SDK crash reporting module 221 is the SDK crash report Data 224 may be sent to the SDK development system 232. One or more SDK console interfaces 234 of the SDK development system 232 may output one or more portions of such SDK crash report data 224 (e.g., in one or more graphical user interfaces) for display on one or more display devices 230. As a result, one or more SDK library developers may examine the information output by the SDK console interface 234 to determine any issues regarding each of the SDKs 238 that they developed or maintain. The graphical user interface 980 shown in FIG. 9 is an example of such a graphical user interface.

[0087] The graphical user interface 980 includes various information provided by the clustering module 218. Also, as described above, the clustering module 118 may generate clustered library error data 124, and similarly, may generate clustered SDK crash report data 224. For example, when generating such clustered SDK crash report data 224 for one or more errors (e.g., crashes) that occur during the execution of one or more of the applications 204, the clustering module 118 may include various forms of clustered information along with the SDK crash report data 224. For example, the clustering module 118 may, depending on the type and details of the information included in the application crash report data 222 provided by the client computing device 240 to the application server system 202, include information such as the type of the application 204, at least one version number of at least one of the corresponding SDKs 238, at least one version number of at least one operating system executed by the client computing device 240, information regarding the client computing device 240 (e.g., operating system or device type information), etc. In addition, the clustering module 218 may also include, within the SDK crash report data 224, one or more of the number of applications 204 affected by at least one error over a certain period (e.g., 30 days, 60 days, all cumulative time up to the present, etc.), the number of users affected by at least one error over that period, and / or the number of occurrences of at least one error over that period.

[0088] The graphical user interface 980 shown in FIG. 9 includes various examples of such information. The upper part of the graphical user interface 980 includes graphs 981, 982, 983 associated with the occurrence of specific errors or crashes over the most recent 30 days. In some cases, the SDK console interface 234 may output these graphs based on input received from one or more SDK developers provided in the SDK development system 232. For example, an SDK developer may provide user input as the SDK console interface 234 and request information within the graphical user interface for the most recent 30 days. In other examples, the SDK developer may request information for other periods (e.g., 60 days, cumulative total time).

[0089] Graph 981 shows a graphical representation of the number of identified crash reports (e.g., occurrences) plotted over time. For graph 981, the x-axis shows time in days, and the left y-axis shows the number of reports. Graph 982 shows a graphical representation of the number of users affected by crashes over time. For graph 982, the x-axis shows time in days, and the left y-axis shows the number of users affected. Graph 983 shows a graphical representation of the number of applications affected by crashes over time for the crashes. For graph 983, the x-axis shows time in days, and the right y-axis shows the number of applications affected (e.g., the number of those among the applications 204 that were affected).

[0090] As shown in FIG. 9, the graphical user interface may also include additional crash details 984 that may include one or more outstanding issues. In the example of FIG. 9, the crash details 984 may include information about one or more different points in time or days. For example, for a particular crash that occurred a certain number of times over the most recent 30 days (represented by graph 981) that affects a particular number of users (represented by graph 982) and / or an application (represented by graph 983), each line of the crash details 984 may include detailed information about one point in time (e.g., one day) represented by graphs 981, 982, 983. For example, as shown in FIG. 9, for each respective row corresponding to a particular day, the crash details 984 may identify or indicate the name or identifier of the SDK associated with the crash, the version number of the SDK that was first affected, the number of users affected (for one point on graph 982), the number of applications affected (for one point on graph 983), and the number of reports of the occurrence of the crash (for one point on graph 981).

[0091] FIG. 10 is a screen diagram showing another exemplary graphical user interface 1090 that may be displayed in one or more third-party library development systems based on data provided by an application server system according to one or more aspects of the present disclosure. For purposes of illustration only, FIG. 10 is described with reference to the SDK development system 232 and the application server system 202 shown in FIG. 2.

[0092] The graphical user interface 1090 is another example of an interface that may be output for display on the display device 230 of the SDK development system 232 by the SDK console interface 234. The SDK console interface 234 may output the graphical user interface 1090 for display upon receipt of the SDK crash report data 224, which may include the clustered data generated by the clustering module 218 as described above.

[0093] The upper portion of the graphical user interface 1090 includes graphs 1091, 1092, 1093 associated with the occurrence of certain errors or crashes over the most recent 30 days. Different occurrences of this particular crash during one or more runtimes of the application 204 may be associated with one or more version numbers of a particular SDK included in or used by the application 204, one or more version numbers of an operating system (OS) executed by the client computing device 240 during the execution of the application 204, and / or one or more device types of the client computing device 240 (e.g., a device type based on the manufacturer and / or model, a device type based on the physical characteristics of the client computing device 240, etc.).

[0094] Graph 1091 is a bar graph that shows the percentage of occurrences of these specific crashes or types of crashes, clustered by SDK version, with each bar representing a different SDK version number. The SDK developer of this particular SDK may examine Graph 1091 to identify the breakdown of different versions of the SDK associated with different occurrences of the same or similar crashes over the most recent 30-day period. Graph 1092 is a bar graph that shows the percentage of occurrences of a specific crash or type of crash, clustered by OS version, with each bar representing a different OS version number. The SDK version may examine Graph 1092 to identify the breakdown of different versions of the OS executed by client computing device 240 associated with different occurrences of the same or similar crashes over the most recent 30-day period.

[0095] Graph 1093 is a bar graph that shows the percentage of occurrences of a specific crash or type of crash, clustered by the device type of client computing device 240, with each bar representing a different device type. The SDK version may examine Graph 1093 to identify the breakdown of different device types of client computing device 240 associated with different occurrences of the same or similar crashes over the most recent 30 days.

[0096] The graphical user interface 1090 also includes stack information 1094. The stack information 1094 may, in various cases, include a graphical representation of the stack trace data 225 included in the SDK crash report data 224. As shown in FIG. 10, the stack information 1094 may include identifiers of the OS version and SDK version associated with the displayed stack trace. For example, the stack trace may be associated with a specific version number of the SDK included in or used by the crashed application and a specific version number of the OS executed by one or more of the client computing devices 240 during the execution of the application. The stack information 1094 also includes a text representation of one or more portions of the stack trace data 225. For example, this text representation may be one of the error identifier 760 and the library dependent source code 761 included in the stack trace data 725, as shown in FIG. 7. In this example, the specific SDK among the SDKs 238 that may be associated with the occurrence of the crash is the "examplehttp" SDK.

[0097] FIG. 11 is a flow diagram showing an exemplary operation of a process 1100 executed by an application server system, such as any of the application server systems shown in FIGS. 1-3, according to one or more aspects of the present disclosure. For purposes of illustration only, the operations of FIG. 11 are described with reference to the application server system 102 shown in FIG. 1.

[0098] As shown in FIG. 11, process 1100 includes the application server system 102 receiving (e.g., using the application error handling module 120) application error data 122 associated with at least one error that occurred during the execution of at least one of the applications 104 on the client computing device 140 from the client computing device 140 (1102). Process 1100 includes the application server system 102 receiving mapping data 108 that provides a mapping between (i) library-dependent source code of at least one application and (ii) at least one third-party library (1104), where the library-dependent source code is loaded from at least one third-party library during the execution of at least one application. The at least one third-party library may be included in the third-party library 138.

[0099] Process 1100 also includes the application server system 102 determining a match between the library-dependent source code and at least one portion of the application error data 122 based on the application error data 122 and the mapping data 108 (1106), and in response to determining the match, the application server system 102 attributing (e.g., using the library attribution module 114 of the library error data generation unit 110 of the library error data generation unit 110) at least one error that occurred during the execution of at least one application to at least one third-party library (1108). Process 1100 further includes the application server system 102 (e.g., using the library attribution module 114 and / or the clustering module 118 of the library error data generation unit 110) including generating (1110) library error data 124 associated with at least one third-party library, the library error data 124 including at least one portion of the application error data 122, the process 1100 further including the application server system 102 transmitting (1112) the library error data 124 to one or more third-party library development systems 132 that develop at least one third-party library (e.g., using the library error reporting module 121).

[0100] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium including any medium that enables transfer of a computer program from one place to another, for example, according to a communication protocol. In this way, the computer-readable medium generally may correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0101] By way of example and not limitation, such computer-readable storage media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other storage medium that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection can be properly termed a computer-readable medium. For example, if the instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, microwave are included in the definition of the medium. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but are directed to non-transient tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc (registered trademark), optical disc, digital versatile disc (DVD), floppy (registered trademark) disk and Blu-ray (registered trademark) disc (disc), and disk typically magnetically reproduces data, while disc optically reproduces data with a laser. The above combinations should also be included within the scope of computer-readable media.

[0102] The commands may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated circuits or discrete logic circuits. Thus, the term "processor" as used herein may refer to either the foregoing structures, or any other structure suitable for implementing the techniques described herein. Further, in some aspects, the functions described herein may be provided within dedicated hardware and / or software modules. Also, the techniques may be fully implemented in one or more circuits or logic elements.

[0103] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chip sets). In the present disclosure, various components, modules, or units have been described in order to emphasize the functional aspects of the devices configured to execute the disclosed techniques, but those components, modules, or units do not necessarily have to be implemented by different hardware units. Rather, as described above, the various units may be combined in a hardware unit, or provided by a set of operating hardware units, including one or more processors as described above, in connection with appropriate software and / or firmware.

[0104] It should be recognized that, depending on the embodiment, any particular act or event of the methods described herein may be executed in a different sequence, may be added, merged, or completely excluded (e.g., not all of the recited acts or events are necessary for the implementation of the method). Further, in certain embodiments, the acts or events may be executed not sequentially, but, for example, concurrently through multi-threaded processing, interrupt processing, or through multiple processors.

[0105] In some examples, a computer-readable storage medium includes a non-transitory medium. The term "non-transitory" indicates that the storage medium is not embodied in a carrier wave or a propagated signal. In some examples, a non-transitory storage medium may store data that can change over time (e.g., in RAM or a cache).

[0106] Various examples have been described. These and other examples are within the scope of the following claims.< / private> < / library>

Claims

1. 1. A method comprising: an application server system including one or more processors receiving, from one or more client computing devices, application error data associated with at least one error that occurred during execution of at least one application on the one or more client computing devices; the application server system receiving mapping data providing a mapping between (i) library-dependent source code of the at least one application and (ii) at least one third party library, the library-dependent source code being loaded from the at least one third party library during execution of the at least one application, the method further comprising: determining, by the application server system, a match between the library-dependent source code and at least a portion of the application error data based on the application error data and the mapping data; in response to determining the match, the application server system attributing the at least one error that occurred during execution of the at least one application to the at least one third party library; and generating library error data associated with the at least one third party library, the library error data including the at least one portion of the application error data, the method further comprising: The method includes the application server system sending the library error data to at least one third party library development system that develops the at least one third party library.

2. the application error data includes application crash report data associated with the at least one error that caused the at least one application to crash on the one or more client computing devices; the at least one portion of the application error data includes at least one portion of the application crash report data; The method of claim 1 , wherein the library error data comprises library crash report data that includes the at least one portion of the application crash report data.

3. the application crash report data includes stack trace data representing nested function calls of the at least one application; The method of claim 2 , wherein the at least one portion of the application crash report data includes at least one portion of the stack trace data.

4. 4. The method of claim 3, wherein determining the match includes the application server system matching one or more patterns of the library-dependent source code to one or more code locations in the at least one portion of the stack trace data based on the stack trace data and the mapping data.

5. receiving, by the application server system, de-obfuscation data associated with the library-dependent source code from the at least one third party library development system that develops the at least one third party library; and the application server system deobfuscating the at least one portion of the stack trace data based on the deobfuscation data. The method according to claim 3.

6. The method of claim 3 , wherein generating the library error data includes the application server system removing application-dependent source code from the stack trace data to generate the library error data.

7. the at least one third party library includes at least a first third party library and a second third party library; the library-dependent source code includes first library-dependent source code associated with the first third party library and second library-dependent source code associated with the second third party library; the at least one portion of the application error data includes the first library dependent source code; 7. The method of claim 6, wherein generating the library error data further comprises the application server system removing the second library dependent source code from the stack trace data to generate the library error data.

8. 8. The method of claim 1, wherein the library error data further indicates one or more of: at least one version number of the at least one third party library, at least one version number of at least one operating system executed by the one or more client computing devices, at least one device type of the one or more client computing devices, a number of applications affected by the at least one error over a period of time, a number of users affected by the at least one error over the period of time, or a number of occurrences of the at least one error over the period of time.

9. The method further comprises: the application server system determining a first fingerprint identifier associated with the at least one portion of the application error data included in the library error data; and the application server system determining a match between the first fingerprint identifier and a second fingerprint identifier associated with previously generated library error data, the previously generated library error data being associated with the at least one third party library, the previously generated library error data including at least a portion of application error data associated with the at least one error that occurred during execution of the at least one application on the one or more client computing devices, the method further comprising: in response to determining the match between the first fingerprint identifier and the second fingerprint identifier, the application server system clusters the library error data with the previously generated library error data to generate clustered library error data associated with the at least one third party library, the clustered library error data indicating at least a number of applications affected by the at least one error; The method of any one of claims 1 to 8, wherein sending the library error data includes the application server system sending the clustered library error data to the at least one third party library development system.

10. The method further comprises: the application server system determining that a number of the applications affected by the at least one error exceeds a threshold number; 10. The method of claim 9, wherein transmitting the clustered library error data occurs in response to determining that a number of the applications affected by the at least one error exceeds the threshold number.

11. The method of any one of claims 1 to 10, wherein receiving the mapping data comprises the application server system receiving the mapping data from at least one application development system that develops the at least one application.

12. At least one processor; and at least one computer-readable storage device configured to store instructions executable by the at least one processor, the instructions comprising: receiving application error data from one or more client computing devices, the application error data being associated with at least one error that occurred during execution of at least one application on the one or more client computing devices; and (ii) at least one third party library, the library-dependent source code being loaded from the at least one third party library during execution of the at least one application; and the instructions are executable by the at least one processor to receive mapping data providing a mapping between (i) library-dependent source code of the at least one application and (ii) at least one third party library, the library-dependent source code being loaded from the at least one third party library during execution of the at least one application, the instructions further comprising: determining a match between the library-dependent source code and at least a portion of the application error data based on the application error data and the mapping data; in response to determining the match, attributing the at least one error that occurred during execution of the at least one application to the at least one third party library; and executing instructions for generating library error data associated with the at least one third party library, the library error data including the at least one portion of the application error data, the instructions further comprising: An application server system executable by the at least one processor to send the library error data to at least one third party library development system that develops the at least one third party library.

13. The application server system of claim 12, wherein the instructions stored in the at least one computer readable storage device are further executable by the at least one processor to perform the method of any one of claims 2 to 11.

14. A computer-readable storage device storing instructions that, when executed, cause at least one processor to perform operations, including: receiving application error data from one or more client computing devices, the application error data being associated with at least one error that occurred during execution of at least one application on the one or more client computing devices; and (ii) receiving mapping data providing a mapping between at least one third party library and (i) library-dependent source code of the at least one application, the library-dependent source code being included in the at least one application. and loading from the at least one third party library during execution of the application, the operation further comprising: determining a match between the library-dependent source code and at least a portion of the application error data based on the application error data and the mapping data; in response to determining the match, attributing the at least one error that occurred during execution of the at least one application to the at least one third party library; and generating library error data associated with the at least one third party library, the library error data including the at least one portion of the application error data, the operations further comprising: transmitting said library error data to at least one third party library development system that develops said at least one third party library.

15. The computer readable storage device of claim 14, wherein the instructions further cause the at least one processor to perform the method of any one of claims 2 to 11.

Images