App performance acquisition and analysis scheme

Abstract

The invention belongs to the technical field of mobile application development and performance management, and discloses an App performance acquisition and analysis scheme, and the specific implementation mode of the App performance acquisition and analysis scheme is as follows: S1, a client (Android, IOS, MAC) is integrated with a Matrix SDK, and WIndown is integrated with a custom SDK; s2, index data are collected through the SDK, and the collected data are sorted and reported to a file server; and S3, the server side asynchronously pulls the index file, and performs modeling, storage and mining analysis on the data. According to the method, automatic analysis of dSYM data is particularly emphasized for Mac and iOS platforms, and a dSYM file is a symbol file of iOS and Mac application debugging information, contains symbol information generated during application compiling and is used for symbolic analysis in a crash log, so that a developer is helped to locate problems, and the development efficiency is improved by automatically analyzing the dSYM data. According to the scheme, crash and performance problems in the application can be quickly positioned and reported, and the efficiency of developers is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile application development and performance management, and specifically is an App application performance collection and analysis solution. Background Art

[0002] The importance of App performance collection in mobile application development is self-evident. With the popularity of mobile applications and the complexity of their functions, the performance of applications is directly related to user experience and the commercial interests of enterprises. However, traditional operation and maintenance methods often seem to be powerless when facing modern mobile applications, and there are many disadvantages. First of all, the large heterogeneity of the system is a major challenge faced by traditional operation and maintenance. Different mobile devices, operating systems, and network environments may have an impact on the performance of applications. Especially for Mac and iOS platforms, due to their unique system architecture and user experience design, traditional operation and maintenance tools and methods are not friendly in supporting the analysis of jams and crash data, and it is often difficult to accurately capture and diagnose problems. When an application fails, whether it is jams, crashes or other performance problems, it may bring a very poor experience to users, and even lead to user loss and damage to the brand image. This loss is immeasurable, especially in the highly competitive mobile application market, where a serious failure may cause the application to lose a large number of users and market share. In addition, traditional operation and maintenance also has shortcomings in data display and alarm rules. The large-scale display is often not flexible enough to meet the developer's personalized needs for data. However, if the alarm rules are not comprehensive, some important performance issues may be ignored, thus missing the best time to solve the problems. Summary of the invention

[0003] The purpose of the present invention is to provide an App application performance collection and analysis solution to solve the problems raised in the above background technology.

[0004] In order to achieve the above object, the present invention provides the following technical solution: App application performance collection and analysis solution, the specific implementation steps of the App application performance collection and analysis solution are as follows:

[0005] S1, the client (Android, IOS, MAC) integrates Matrix SDK, and WIndows integrates custom SDK;

[0006] S2, collects indicator data through SDK, organizes and reports the collected data to the file server;

[0007] S3, the server asynchronously pulls the indicator file to model, store, mine and analyze the data;

[0008] S4, the dsym data uploaded by Mac needs to be specially symbolized and analyzed;

[0009] S5, then display the overall market and indicator alarm according to the customization of each end.

[0010] Preferably, the Mac dsym symbolic analysis flowchart step in S4 consists of two parts: a MAC server and a K8S server.

[0011] Preferably, the MAC server flow chart steps are as follows:

[0012] A1, pull dSYM files from COS server and unzip them to mac server: First, you need to pull dSYM files related to the application from COS (object storage service) server. dSYM files are debug symbol files generated by Xcode when compiling iOS or macOS applications. They are essential for resolving crash addresses in crash logs. These dSYM files will be downloaded from COS server through a secure network connection (such as SSH or SFTP). Once the dSYM files are successfully downloaded to the MAC server, they need to be unzipped to the appropriate location on the server. The unzipping process can be automated through command line tools (such as unzip or tar) or scripts. The unzipped dSYM files will contain the debug symbol information of the application, which will be used to parse crash data in subsequent steps;

[0013] A2, then parse the crash data: After obtaining and decompressing the dSYM file, the next step is to parse the crash data. Crash data usually contains stack traces, device information, and key information about the operating system version when the application crashes. Special tools or scripts are needed to parse these crash data in order to extract useful information from them. When parsing the crash data, the debugging symbol information in the previously decompressed dSYM file will be used to convert the crash address in the crash log into a readable function name and line number. In this way, the code location that caused the crash can be accurately located, providing valuable clues for subsequent troubleshooting and repair;

[0014] A3. Finally, insert the parsed data into x-apm-matrix-mac-anr-symbol: After parsing the crash data, you need to insert the parsed results into x-apm-matrix-mac-anr-symbol (which may be a database, data storage service or data analysis platform). This is done to preserve and analyze the data for a long time so that you can track the performance and stability issues of the application and take appropriate measures to improve them. When inserting data, you need to ensure the accuracy and completeness of the data and follow the relevant data format and naming specifications. In addition, you can set appropriate data indexes and query functions so that you can quickly retrieve and analyze the data.

[0015] Preferably, the steps of the K8S server flow chart are as follows:

[0016] B1. Pull unanalyzed x-apm-matrix-mac-anr-symbol records: In a Kubernetes cluster, one or more Pods are usually deployed to perform data processing and analysis tasks. In this step, one or more Pods are responsible for pulling unanalyzed records from the x-apm-matrix-mac-anr-symbol data source (which may be a database, message queue, or data storage service). In order to ensure the integrity and consistency of the data, some mechanisms may need to be implemented to track which records have been analyzed. This can be achieved by maintaining a status field in the database, using the confirmation mechanism of the message queue, or using the distributed lock in Kubernetes. The Pod that pulls data may use Kubernetes's persistent volume (PersistentVolume) or ConfigMap to store configuration information, such as data source connection details and query conditions. In addition, the Pod can also use Kubernetes's automatic expansion function (such as Horizontal PodAutoscaler) to dynamically adjust the number of Pods according to the workload to ensure efficient data processing;

[0017] B2, insert the analyzed information into x-apm-matrix-mac-anr-symbol-analyse: Once the Pod has pulled records from the x-apm-matrix-mac-anr-symbol data source and performed the necessary analysis, the next step is to insert the analyzed information into the x-apm-matrix-mac-anr-symbol-analyse data source. This data source may be a database, data storage service, or data warehouse for long-term storage and analysis results. Before inserting the data, the data may need to be cleaned, converted, or aggregated to meet the x-apm-matrix Requirements for the x-mac-anr-symbol-analyse data source, which may include converting data in JSON or XML format into a table structure in a relational database, or aggregating and summarizing the data to generate reports or dashboards. In order to ensure the reliability and accuracy of the data, some validation and error handling mechanisms may need to be implemented. Finally, once the data is successfully inserted into the x-apm-matrix-mac-anr-symbol-analyse data source, Kubernetes' monitoring and logging capabilities can be used to track the processing process and monitor any potential problems. This can be achieved through Kubernetes' Metrics Server, Prometheus monitoring tools, and using Kubernetes' log collection tools (such as Fluentd, Logstash) to collect and analyze Pod logs.

[0018] Preferably, the task retry mechanism steps are as follows:

[0019] C1, set "error_count" in the task record in the database to error_count++;

[0020] C2, if the number of retries "error_count" > 3, set the task status to processing failure;

[0021] C3, put the task to be retried back into the work queue.

[0022] Preferably, the data archiving steps are as follows:

[0023] D1, set the task processing record status to "processed": before archiving data, you first need to update the status of the relevant task processing record to "processed";

[0024] D2, delete task records from the database: Once a task is marked as "processed", the next step is to delete the relevant records of the task from the database;

[0025] D3, add task history to ES: In order to retain the task history information for subsequent query and analysis, the task history needs to be added to the Elasticsearch search engine;

[0026] D4, move the cos file to the history directory: In addition to the records in the database and search engine, there may be task-related files (such as cos files) that need to be archived.

[0027] APM: Application Performance Management (APM) is a monitoring system that can track the operation of applications and help development teams identify and resolve potential problems.

[0028] Matrix: Matrix is ​​an application performance access framework developed and used daily by WeChat, supporting iOS, macOS and Android.

[0029] Runner: The core driver module of the architecture, which processes core tasks regularly (Scheduled);

[0030] Queue: The work queue of each terminal that needs to process tasks, using the producer-consumer model;

[0031] Handler: Each terminal is assigned a corresponding handler, which mainly parses the cos files uploaded by each terminal;

[0032] event: event monitoring at each stage of task processing;

[0033] Mapdb: A local database storage solution based on local memory (files), which mainly stores parsed JSON files and supports retrying tasks when task processing fails;

[0034] Metrics: monitors APM server applications and exposes the metrics information parsed from files on each end;

[0035] Reaper: Provides backup for tasks that have not been processed for a long time or tasks that failed to be archived.

[0036] CosScheduledService: Pull the cos files of each end regularly and put them into the work queue that responds to interrupts. This task can be started in local task mode or LTS scheduling mode through the configuration file;

[0037] MetricsScheduledService: Gets the real-time working status regularly, including: the number of historical files (historical processing + unprocessed), the number of current files, the number to be processed, the number currently being processed, and the number of failed processing;

[0038] ReaperScheduledService: The main tasks of the harvesting task include two:

[0039] E1, regularly harvests long-unprocessed work tasks and handles the situation where tasks cannot be adapted to the OWNER due to the host name being changed due to POD restart;

[0040] E2, the processing task is completed, but the archiving task fails.

[0041] The beneficial effects of the present invention are as follows:

[0042] The present invention, for Mac and iOS platforms, this type of solution particularly emphasizes the automatic analysis of dSYM data. The dSYM file is a symbol file of iOS and Mac application debugging information. It contains symbol information generated when the application is compiled, which is used for symbolic analysis in the crash log, thereby helping developers locate problems. By automatically analyzing dSYM data, this type of solution can quickly locate and report crashes and performance problems in the application, greatly improving the efficiency of developers. In addition, this type of solution also provides users with a flexible large-scale display function. Through a visual interface, users can view the performance data of the application in real time, including response time, crash rate, and key indicators of user behavior, which not only helps users quickly understand the performance status of the application, but also helps users discover potential problems and optimize them. In terms of alarm rules and channels, this type of solution also provides highly customized support. Users can set alarm rules according to their needs. When the performance data of the application exceeds the preset threshold, the system will automatically trigger an alarm and notify relevant personnel. At the same time, users can also choose their favorite alarm channels, such as email, SMS, and corporate WeChat, to ensure that the alarm information is received at the first time. This highly customized support makes application performance management more flexible and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the overall architecture structure of the present invention;

[0044] Figure 2 This is a schematic diagram of the structure of the Mac dsym symbolic analysis process of the present invention;

[0045] Figure 3 It is a schematic diagram of the back-end data processing architecture structure of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] like Figures 1 to 3 As shown, the embodiment of the present invention provides an App application performance collection and analysis solution, and the specific implementation steps of the App application performance collection and analysis solution are as follows:

[0048] S1, the client (Android, IOS, MAC) integrates Matrix SDK, and WIndows integrates custom SDK;

[0049] S2, collects indicator data through SDK, organizes and reports the collected data to the file server;

[0050] S3, the server asynchronously pulls the indicator file to model, store, mine and analyze the data;

[0051] S4, the dsym data uploaded by Mac needs to be specially symbolized and analyzed;

[0052] S5, then display the overall market and indicator alarm according to the customization of each end.

[0053] Among them, the Mac dsym symbolic analysis flowchart steps in S4 are composed of two parts: MAC server and K8S server.

[0054] The steps of the MAC server flow chart are as follows:

[0055] A1, pull dSYM files from COS server and unzip them to mac server: First, you need to pull dSYM files related to the application from COS (object storage service) server. dSYM files are debug symbol files generated by Xcode when compiling iOS or macOS applications. They are essential for resolving crash addresses in crash logs. These dSYM files will be downloaded from COS server through a secure network connection (such as SSH or SFTP). Once the dSYM files are successfully downloaded to the MAC server, they need to be unzipped to the appropriate location on the server. The unzipping process can be automated through command line tools (such as unzip or tar) or scripts. The unzipped dSYM files will contain the debug symbol information of the application, which will be used to parse crash data in subsequent steps;

[0056] A2, then parse the crash data: After obtaining and decompressing the dSYM file, the next step is to parse the crash data. Crash data usually contains stack traces, device information, and key information about the operating system version when the application crashes. Special tools or scripts are needed to parse these crash data in order to extract useful information from them. When parsing the crash data, the debugging symbol information in the previously decompressed dSYM file will be used to convert the crash address in the crash log into a readable function name and line number. In this way, the code location that caused the crash can be accurately located, providing valuable clues for subsequent troubleshooting and repair;

[0057] A3. Finally, insert the parsed data into x-apm-matrix-mac-anr-symbol: After parsing the crash data, you need to insert the parsed results into x-apm-matrix-mac-anr-symbol (which may be a database, data storage service or data analysis platform). This is done to preserve and analyze the data for a long time so that you can track the performance and stability issues of the application and take appropriate measures to improve them. When inserting data, you need to ensure the accuracy and completeness of the data and follow the relevant data format and naming specifications. In addition, you can set appropriate data indexes and query functions so that you can quickly retrieve and analyze the data.

[0058] Among them, the steps of the K8S server flowchart are as follows:

[0059] B1. Pull unanalyzed x-apm-matrix-mac-anr-symbol records: In a Kubernetes cluster, one or more Pods are usually deployed to perform data processing and analysis tasks. In this step, one or more Pods are responsible for pulling unanalyzed records from the x-apm-matrix-mac-anr-symbol data source (which may be a database, message queue, or data storage service). In order to ensure the integrity and consistency of the data, some mechanisms may need to be implemented to track which records have been analyzed. This can be achieved by maintaining a status field in the database, using the confirmation mechanism of the message queue, or using the distributed lock in Kubernetes. The Pod that pulls data may use Kubernetes's persistent volume (PersistentVolume) or ConfigMap to store configuration information, such as data source connection details and query conditions. In addition, the Pod can also use Kubernetes's automatic expansion function (such as Horizontal PodAutoscaler) to dynamically adjust the number of Pods according to the workload to ensure efficient data processing;

[0060] B2, insert the analyzed information into x-apm-matrix-mac-anr-symbol-analyse: Once the Pod has pulled records from the x-apm-matrix-mac-anr-symbol data source and performed the necessary analysis, the next step is to insert the analyzed information into the x-apm-matrix-mac-anr-symbol-analyse data source. This data source may be a database, data storage service, or data warehouse for long-term storage and analysis results. Before inserting the data, the data may need to be cleaned, converted, or aggregated to meet the x-apm-matrix Requirements for the x-mac-anr-symbol-analyse data source, which may include converting data in JSON or XML format into a table structure in a relational database, or aggregating and summarizing the data to generate reports or dashboards. In order to ensure the reliability and accuracy of the data, some validation and error handling mechanisms may need to be implemented. Finally, once the data is successfully inserted into the x-apm-matrix-mac-anr-symbol-analyse data source, Kubernetes' monitoring and logging capabilities can be used to track the processing process and monitor any potential problems. This can be achieved through Kubernetes' Metrics Server, Prometheus monitoring tools, and using Kubernetes' log collection tools (such as Fluentd, Logstash) to collect and analyze Pod logs.

[0061] Among them, the task retry mechanism steps are as follows:

[0062] C1, set "error_count" in the task record in the database to error_count++;

[0063] C2, if the number of retries "error_count" > 3, set the task status to processing failure;

[0064] C3, put the task to be retried back into the work queue.

[0065] The data archiving steps are as follows:

[0066] D1, set the task processing record status to "processed": before archiving data, you first need to update the status of the relevant task processing record to "processed";

[0067] D2, delete task records from the database: Once a task is marked as "processed", the next step is to delete the relevant records of the task from the database;

[0068] D3, add task history to ES: In order to retain the task history information for subsequent query and analysis, the task history needs to be added to the Elasticsearch search engine;

[0069] D4, move the cos file to the history directory: In addition to the records in the database and search engine, there may be task-related files (such as cos files) that need to be archived.

[0070] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0071] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. App application performance collection and analysis solution, characterized by: The specific implementation steps of the App application performance collection and analysis solution are as follows: S1, the client integrates Matrix SDK, and WIndows integrates custom SDK; S2, collects indicator data through SDK, organizes and reports the collected data to the file server; S3, the server asynchronously pulls the indicator file to model, store, mine and analyze the data; S4, the dsym data uploaded by Mac needs to be specially symbolized and analyzed; S5, displays the market and issues indicator alarms based on the customization of each terminal.

2. The App application performance collection and analysis solution according to claim 1 is characterized by: The Mac dsym symbolic analysis flowchart steps in S4 are composed of two parts: MAC server and K8S server.

3. The App application performance collection and analysis solution according to claim 2 is characterized by: The steps of the MAC server flow chart are as follows: A1, pull the dSYM file from the COS server and decompress it to the mac server; A2, then parses the crash data; A3, finally insert the parsed data into x-apm-matrix-mac-anr-symbol.

4. The App application performance collection and analysis solution according to claim 1 is characterized by: The steps of the K8S server flow chart are as follows: B1, pull the unanalyzed x-apm-matrix-mac-anr-symbol record; B2, insert the analyzed information into x-apm-matrix-mac-anr-symbol-analyse.

5. The App application performance collection and analysis solution according to claim 1 is characterized by: The task retry mechanism steps are as follows: C1, set "error_count" in the task record in the database to error_count++; C2, if the number of retries "error_count" > 3, set the task status to processing failure; C3, put the task to be retried back into the work queue.

6. The App application performance collection and analysis solution according to claim 1 is characterized by: The steps for data archiving are as follows: D1, set the task processing record status to "processed": before archiving data, you first need to update the status of the relevant task processing record to "processed"; D2, delete task records from the database: Once a task is marked as "processed", the next step is to delete the relevant records of the task from the database; D3, add task history to ES: In order to retain the task history information for subsequent query and analysis, the task history needs to be added to the Elasticsearch search engine; D4, move the cos files to the history directory: In addition to the records in the database and search engine, the files related to the task need to be archived.

Images