A high-concurrency data link buried point log analysis method, device and system

Abstract

The present invention discloses a method, device and system for analyzing buried point logs of high-concurrency data links, which determines whether the trigger conditions for uploading are met, and if so, uploads buried point logs, and calculates the number of buried point logs to be distributed for different business requests according to the fixed process non-blocking asynchronous log distribution algorithm, and distributes the corresponding number of buried point logs to different businesses according to the calculation results; based on the basic data and business data of the buried point logs distributed to different businesses in parallel, fault analysis and prediction are performed based on the cluster analysis algorithm, and the basic data and business data that are prone to failure in the buried point logs are obtained, and business updates and business recommendation solutions are output according to the results. Under the same hardware configuration, the single-machine concurrency is greatly improved, effectively saving hardware costs; the buried point data adopts an elastic data structure, which can flexibly respond to business changes; the buried point data adopts a basic data plus multiple business data mode to reduce communication bandwidth consumption.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a method, device and system for analyzing high-concurrency data link buried point logs. Background Art

[0002] With the development of cloud computing, more and more enterprises and individuals need to process their lives, studies, and work online. At the same time, with the rise of intelligent office, relevant business systems serve everyone in the form of mobile terminals or servers. However, when some business systems fail, it may affect a large number of users. Therefore, how to accurately maintain business health and recover from failures has become the main research direction of log tracking in data links. Data link analysis obtains business data and device data of applications, and through analysis, it ensures that the use is in the best state to the greatest extent. In data link log tracking, service request concurrency and log analysis are the most core technologies.

[0003] At present, the traditional method is to maintain in the form of microservice log tracking, and a single machine supports about 3,000 concurrent users. When the number of customers' application users exceeds 10 million, the log concurrency in the tracking service becomes the biggest challenge for data services and storage services. The main problems of traditional log tracking services are as follows: (1) Separate analysis based on business identification and fault identification; (2) Small single-machine concurrency and high hardware investment costs; (3) The adaptability and flexibility of the traditional log tracking data structure are poor; (4) The application-side uploads a large amount of basic data on the attributes of the terminal device to the tracking service, and the basic data is the same, so some data is transmitted repeatedly and occupies bandwidth. Summary of the invention

[0004] To this end, the present invention provides a high-concurrency data link tracking log analysis method, device and system to solve the above technical problems existing in traditional log tracking services.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] According to a first aspect of an embodiment of the present invention, a high-concurrency data link tracking log analysis method is proposed, the method comprising:

[0007] Obtaining an upload request for the collected data link tracking log, and determining whether the triggering condition for uploading is met, wherein the tracking log data includes basic data and business data;

[0008] If the trigger condition is met, the tracking log is uploaded, and the number of tracking logs to be distributed for different business requests is calculated according to the fixed process non-blocking asynchronous log distribution algorithm. According to the calculation results, the corresponding number of tracking logs are distributed to different businesses according to the parallel distribution rules;

[0009] Based on the basic data and business data of the tracking logs distributed in parallel to different businesses, fault analysis and prediction are performed based on the cluster analysis algorithm. The basic data and business data that are prone to failures in the tracking logs are obtained, and business updates and business recommendation plans are output based on the results.

[0010] Furthermore, the upload request of the collected data link tracking log is obtained, which specifically includes:

[0011] The tracking log adopts a fixed process method to write the basic data of the tracking log of the data link and the business data into a string file to realize data collection, and the string file sends an upload request.

[0012] Furthermore, the basic data includes user ID, application coding identification ID, application version identification ID, operating system type identification ID, operating system version identification ID, device type ID, device name ID, communication address ID, and download channel ID, and the business data includes log generation date ID, timestamp ID, log type ID, business embedding point number ID, and elastic business number ID.

[0013] Further, it is determined whether the triggering conditions for uploading are met, specifically including:

[0014] Determine whether the number of tracking logs exceeds the preset number;

[0015] If it exceeds, it is determined whether the time interval from the last upload exceeds the preset time interval. If so, the trigger condition for upload is met.

[0016] Furthermore, the number of embedded logs to be distributed for different business requests is calculated according to the fixed process non-blocking asynchronous log distribution algorithm, including:

[0017] The services include real-time detailed query, offline data warehouse analysis, and real-time recommendation services;

[0018] The number of uploaded tracking log data is x, i is the business type number of m, and the number of real-time detail query requests m1, real-time recommendation requests m2, and offline data warehouse analysis requests m3 are calculated according to the fixed process non-blocking asynchronous log concurrency algorithm f(x) of the tracking log. Specifically,

[0019] m i =f(x)

[0020]

[0021]

[0022]

[0023] f(x)T =[m1,m2,m3]

[0024]

[0025] Furthermore, fault analysis and prediction are performed based on cluster analysis algorithms, including:

[0026] The first step is to parse all the tracking logs contained in m3 according to the number of tracking logs distributed in parallel to the offline data warehouse, and calculate the total number of upload requests n for the tracking logs analyzed by the offline data warehouse based on the number of original tracking log requests x:

[0027] m3÷x=n

[0028] In the second step, based on the calculated total number of upload requests n, let Δn be the manually set clustering analysis algorithm trigger condition parameter. When n meets the following conditions, start the offline data warehouse tracking log analysis:

[0029] n ≥ Δn;

[0030] In the third step, the point-of-use log analysis algorithm is as follows:

[0031] The basic data frequency of the tracking log is defined as θ = (θ1, θ2, θ3, θ4, θ5, θ6, θ7, θ8, θ9), where θ i is the number of occurrences of each ID in the basic data. The business data frequency is defined as in is the number of occurrences of each ID in the business data. The business data failure frequency type distance value d corresponding to each basic data in the uploaded tracking log x is:

[0032]

[0033] in Indicates the average number of occurrences of each ID in the basic data in one upload. Indicates the average number of occurrences of each ID in the business data; M θ Indicates the frequency mode of each ID in the basic data. Indicates the frequency mode of each ID in the business data. The mode indicates the number with the highest frequency in a group of numbers.

[0034] The fourth step is to calculate the cosine similarity distance value cos(d) of all basic data and business data in a batch:

[0035]

[0036] Where i is the value sequence range, which is (1 to 9) in the embedding log;

[0037] Step 5: Based on the results of steps 4 and 3, calculate the fault distance value of the business data corresponding to the basic data in the tracking log within the upload period of this batch through clustering.

[0038]

[0039] Step 6: Store the calculated results in a list. When the calculated value is obviously abnormal or exceeds the set threshold, it indicates a fault. The fault can be located based on the D value. The corresponding θ fault frequency can be located through the business data fault distance and basic data relationship cluster analysis to locate the basic data records associated with a specific business data fault. According to the cluster analysis algorithm, business updates, business recommendations, fault analysis, and fault prediction can be achieved to complete system upgrades and updates.

[0040] Furthermore, the method specifically includes:

[0041] According to the calculated basic data fault distance value cos(d), determine which basic data are prone to failure, and trigger system update when the calculated value exceeds the set value;

[0042] Then, according to the business data fault distance value corresponding to the calculated basic data Determine which business data is prone to failure based on basic data, and trigger a targeted update of the business data when the calculated value exceeds the set value;

[0043] Regularly push targeted update services and message notifications to business data and basic data that are prone to failures based on the calculation results of D.

[0044] According to a second aspect of an embodiment of the present invention, a high-concurrency data link tracking log analysis device is provided, the device comprising:

[0045] The tracking log upload trigger module is used to obtain the upload request of the collected data link tracking log and determine whether the trigger condition for upload is met. The tracking log data includes basic data and business data.

[0046] The tracking point log distribution module is used to upload the tracking point log if the trigger condition is met, and calculate the number of tracking point logs to be distributed for different business requests according to the fixed process non-blocking asynchronous log distribution algorithm, and distribute the corresponding number of tracking point logs to different businesses according to the calculation results and the parallel distribution rules;

[0047] The tracking point log analysis module is used to perform fault analysis and prediction based on the basic data and business data of the tracking point logs distributed in parallel to different businesses, based on the cluster analysis algorithm, to obtain the basic data and business data in the tracking point logs that are prone to faults, and to output business updates and business recommendation plans based on the results.

[0048] According to a third aspect of an embodiment of the present invention, a high-concurrency data link tracking log analysis system is provided, characterized in that the system includes: a processor and a memory;

[0049] The memory is used to store one or more program instructions;

[0050] The processor is used to run one or more program instructions to execute any of the methods described above.

[0051] According to a fourth aspect of an embodiment of the present invention, a computer storage medium is proposed, wherein the computer storage medium contains one or more program instructions, and the one or more program instructions are used to be executed by a high-concurrency data link tracking log analysis system as described in any of the above methods.

[0052] The present invention has the following advantages:

[0053] The present invention proposes a high-concurrency data link tracking log analysis method, device and system, which obtain an upload request for a collected data link tracking log and determine whether a trigger condition for upload is met, wherein the tracking log data includes basic data and business data; if the trigger condition is met, the tracking log is uploaded, and the number of tracking logs to be distributed for different business requests is calculated according to a fixed process non-blocking asynchronous log distribution algorithm, and the corresponding number of tracking logs are distributed to different businesses according to the calculation results and parallel distribution rules; based on the basic data and business data of the tracking logs distributed in parallel to different businesses, fault analysis and prediction are performed based on a clustering analysis algorithm, basic data and business data that are prone to failure in the tracking logs are obtained, and business update and business recommendation plans are output according to the results. With this solution, the concurrent capacity of a single machine can be increased by 5 times with the same hardware configuration, saving 80% of the hardware cost; the embedded data adopts an elastic data structure, which can flexibly respond to business changes, increase service flexibility by 10 times, and improve computing efficiency by 80%; the embedded data adopts a basic data plus multiple business data, reducing communication bandwidth consumption by 40%; clustering analysis and optimization of system logs can save 30% of application promotion costs; analyzing the application frequency and the adaptability of the application to the model can collect personalized features through embedded data to improve the overall operation service and smooth operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0055] Figure 1 A flowchart of a high-concurrency data link tracking log analysis method provided in Example 1 of the present invention;

[0056] Figure 2 A specific implementation flow chart of a high-concurrency data link tracking log analysis method provided in Example 1 of the present invention;

[0057] Figure 3 A flow chart of a fixed-process non-blocking asynchronous log distribution algorithm for a high-concurrency data link tracking log analysis method provided in Example 1 of the present invention;

[0058] Figure 4 A fault log clustering analysis flow chart of a high-concurrency data link tracking log analysis method provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0059] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Example 1

[0061] like Figure 1 and Figure 2 As shown, this embodiment proposes a high-concurrency data link tracking log analysis method, the method comprising:

[0062] S100: Obtain an upload request for the collected data link tracking log, and determine whether the triggering condition for uploading is met. The tracking log data includes basic data and business data.

[0063] In this embodiment, the tracking log adopts a fixed process method to write the basic data of the data link's tracking log (user ID, application coding identification ID, application version identification ID, operating system type identification ID, operating system version identification ID, device type ID, device name ID, communication address ID, download channel ID) and business data (log generation date ID, timestamp ID, log type ID, business tracking number ID, elastic business number ID) into a string file to realize data collection, and the string file sends an upload request.

[0064] S200. If the triggering conditions are met, upload the tracking logs, and calculate the number of tracking logs to be distributed for different business requests according to the fixed process non-blocking asynchronous log distribution algorithm. According to the calculation results, the corresponding number of tracking logs are distributed to different businesses according to the parallel distribution rules.

[0065] (1) A load balancing method for parallel distribution of embedded logs based on a fixed process non-blocking asynchronous distribution algorithm, such as Figure 3 Shown

[0066] Input: The tracking log information triggers the triggering conditions of the upload request and then uploads the tracking log.

[0067] Method: Fixed process non-blocking asynchronous log distribution algorithm on-demand elastic log distribution implementation

[0068] Concurrent load balancing

[0069] Output: number of real-time detailed query logs, number of offline data warehouse analysis logs, number of embedded logs pushed by real-time business

[0070] The details are as follows:

[0071] The string file sends an upload request. According to the trigger conditions (30 accumulated tracking logs, 20 seconds since the last upload, and a system exception), the uploaded information is judged and fixed process non-blocking asynchronous log distribution is used to achieve service load balancing.

[0072] According to the number of uploaded tracking log data as x, i as the business type sequence number of m, and the fixed process non-blocking asynchronous log concurrency algorithm f(x) of the tracking log, the number of real-time detailed query requests m1, the number of real-time recommendation requests m2, and the number of offline data warehouse analysis requests m3 are calculated as follows:

[0073] m i =f(x)

[0074]

[0075]

[0076]

[0077] f(x) T =[m1,m2,m3]

[0078]

[0079] Using the above calculation, we generate f(x) T The number of requests required to implement real-time query, fault analysis, and message push i , and uniformly distributed to three business scenarios to realize concurrent service requests. The number of embedded logs that meet the trigger conditions can be uploaded when it exceeds 30, while the number of embedded logs for business requests may be tens of thousands. The needs can be met by copying a large number of original embedded logs.

[0080] According to the above results, the processed tracking logs are distributed to real-time detail query, offline data warehouse analysis, and real-time recommendation to complete real-time detail query based on business needs, offline data warehouse fault analysis, real-time business recommendation, and fault prediction.

[0081] The method of this embodiment includes a method for collecting and uploading a data link tracking log information; a concurrent load balancing method for tracking logs; real-time business detail query, real-time business recommendation calculation, and offline data warehouse fault log clustering analysis methods as main invention points, which solve the problems of parallel requests for edge device intelligence and business intelligence services, service abnormality recovery, business iteration planning, and other issues.

[0082] Through the above steps, we can output basic information portraits of problem types, business information portraits, and formulate system version maintenance and business upgrade services according to the concurrency method of the message request process of the embedded log and the fault clustering analysis method of the offline data warehouse. In this way, we can design application version iterations and personalized services that are more in line with the business. This ensures the stable operation of the business production process, and can also provide accurate services to different types of customers based on the subsequent business information.

[0083] S300. According to the basic data and business data of the tracking logs distributed in parallel to different businesses, fault analysis and prediction are performed based on the cluster analysis algorithm, the basic data and business data that are prone to failure in the tracking logs are obtained, and business updates and business recommendation plans are output based on the results.

[0084] (2) The detailed implementation process of fault log cluster analysis, such as Figure 4 As shown:

[0085] Input: Basic information and business information of tracking point logs distributed according to load balancing.

[0086] Method: 1. Read the basic information and business information ID of the data warehouse's tracking logs and use the clustering analysis algorithm to calculate the frequency of failures of the types of basic information in each tracking log. 2. Read the real-time query tracking logs and offline data warehouse tracking logs and calculate the business information failure frequency according to the fault clustering algorithm. When the failure frequency exceeds the manually set threshold, the business update plan is triggered. The business update plan starts. 3. Read the real-time computing point log and the offline data warehouse point log and calculate the business information failure frequency according to the fault clustering algorithm, such as Develop a business recommendation plan based on the frequency. 4. Read the real-time query tracking log, offline data warehouse business failure log, and real-time calculation tracking log. Clustering algorithms predict service failure frequencies.

[0087] Output: basic information fault type frequency cos(d), business information fault type frequency D corresponding to the basic information

[0088] The specific algorithm is as follows:

[0089] The first step is to parse all the tracking logs contained in the logs m3 distributed in parallel to the offline data warehouse, and calculate the total number of upload requests n for the tracking logs analyzed by the offline data warehouse based on the number of original tracking log requests x (that is, the number of tracking log data uploaded above):

[0090] m3÷x=n

[0091] In the second step, based on the calculated total number of upload requests n, let Δn be the manually set clustering analysis algorithm trigger condition parameter. When n meets the following conditions, start the offline data warehouse tracking log analysis:

[0092] n ≥ Δn;

[0093] In the third step, the tracking log analysis algorithm is as follows:

[0094] The basic data frequency of the tracking log is defined as θ = (θ1, θ2, θ3, θ4, θ5, θ6, θ7, θ8, θ9), where θ i is the number of occurrences of each ID in the basic data. The business data frequency is defined as in is the number of occurrences of each ID in the business data. The business data failure frequency type distance value d corresponding to each basic data in the uploaded tracking log x is:

[0095]

[0096] in Indicates the average number of occurrences of each ID in the basic data in one upload. Indicates the average number of occurrences of each ID in the business data; M θ Indicates the frequency mode of each ID in the basic data. Indicates the frequency mode of each ID in the business data. The mode indicates the number with the highest frequency in a group of numbers.

[0097] The fourth step is to calculate the cosine similarity distance value cos(d) of all basic data and business data in a batch:

[0098]

[0099] Where i is the value sequence range, which is (1 to 9) in the embedding log;

[0100] Step 5: Based on the results of steps 4 and 3, calculate the fault distance value of the business data corresponding to the basic data in the tracking log within the upload period of this batch through clustering.

[0101]

[0102] Step 6: Store the calculated results in a list. When the calculated value is obviously abnormal or exceeds the set threshold, it indicates a fault. The fault can be located based on the D value. The corresponding θ fault frequency can be located through the business data fault distance and basic data relationship cluster analysis to locate the basic data records associated with a specific business data fault. According to the cluster analysis algorithm, business updates, business recommendations, fault analysis, and fault prediction can be achieved to complete system upgrades and updates.

[0103] The overall picture can be summarized as:

[0104] Step 1: Calculate the basic information that is prone to failure, cluster the failure frequency to get a value, and trigger a system update when this value is exceeded.

[0105] Step 2: Determine which business information is prone to failure based on basic information, cluster the failure frequency to obtain a value, and trigger a targeted update of the business information when this value is exceeded.

[0106] Step 3: Regularly push qualitative update services and message notifications for business information and basic information that are prone to failure based on D's calculation basis.

[0107] The implementation cases are as follows:

[0108] 1. Upload tracking logs via mobile app

[0109] 2. Determine whether the requested data format is complete, including basic information and business information.

[0110] 3. Verify the transmission request by fixing the process communication address

[0111] 4. Upload condition satisfaction judgment, upload after the condition is triggered

[0112] 5. Logs are distributed to real-time queries, data warehouse analysis, and real-time calculations according to a master process and multiple worker threads according to parallel distribution rules.

[0113] 6. In the data warehouse analysis, all offline logs are cyclically read for business information corresponding to the basic information, and the number of offline requests calculated exceeds the manually set threshold.

[0114] 7. Trigger the log clustering analysis algorithm to cluster the basic information by frequency and the business information by type, and find the basic information and business information with the maximum distance from the sample, which are defined as fault logs.

[0115] 8. Distribute iterative upgrade plans and recommended messages according to basic information and business information.

[0116] 9. Repeat the above operations for the business result log. Through the above operations, the fault sorting of the data link tracking log is realized for real-time recommendation and offline calculation update iteration and message recommendation feedback scheme to ensure the optimal business status.

[0117] Example 2

[0118] Corresponding to the above-mentioned embodiment 1, this embodiment proposes a high-concurrency data link embedding log analysis device, the device comprising:

[0119] The tracking log upload trigger module is used to obtain the upload request of the collected data link tracking log and determine whether the upload trigger conditions are met. The tracking log data includes basic data and business data;

[0120] The tracking point log distribution module is used to upload the tracking point log if the trigger condition is met, and calculate the number of tracking point logs to be distributed for different business requests according to the fixed process non-blocking asynchronous log distribution algorithm, and distribute the corresponding number of tracking point logs to different businesses according to the calculation results and the parallel distribution rules;

[0121] The tracking point log analysis module is used to perform fault analysis and prediction based on the basic data and business data of the tracking point logs distributed in parallel to different businesses, based on the cluster analysis algorithm, to obtain the basic data and business data in the tracking point logs that are prone to faults, and to output business updates and business recommendation plans based on the results.

[0122] The functions performed by each component in a high-concurrency data link tracking log analysis device provided by an embodiment of the present invention have been described in detail in the above-mentioned embodiment 1, so no further details will be given here.

[0123] Example 3

[0124] Corresponding to the above embodiment, the embodiment of the present invention proposes a high-concurrency data link tracking log analysis system, the system comprising: a processor and a memory;

[0125] The memory is used to store one or more program instructions;

[0126] The processor is used to run one or more program instructions to execute the method of embodiment 1.

[0127] Example 4

[0128] Corresponding to the above-mentioned embodiments, this embodiment proposes a computer storage medium, which contains one or more program instructions, and the one or more program instructions are used to be executed by a high-concurrency data link point log analysis system as the method of embodiment 1.

[0129] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A high-concurrency data link embedding log analysis method, characterized in that: The method comprises: Obtaining an upload request for the collected data link tracking log, and determining whether the triggering condition for uploading is met, wherein the tracking log data includes basic data and business data; If the trigger condition is met, the tracking log is uploaded, and the number of tracking logs to be distributed for different business requests is calculated according to the fixed process non-blocking asynchronous log distribution algorithm. According to the calculation results, the corresponding number of tracking logs are distributed to different businesses according to the parallel distribution rules; Based on the basic data and business data of the tracking logs distributed in parallel to different businesses, fault analysis and prediction are performed based on the cluster analysis algorithm. The basic data and business data that are prone to failures in the tracking logs are obtained, and business updates and business recommendation solutions are output based on the results. According to the fixed process non-blocking asynchronous log distribution algorithm, the number of embedded logs to be distributed for different business requests is calculated, including: The services include real-time detailed query, offline data warehouse analysis, and real-time recommendation services; The number of uploaded tracking log data is x, i is the value sequence number range, and the number of real-time detailed query requests m1, real-time recommendation requests m2, and offline data warehouse analysis requests m3 are calculated according to the fixed process non-blocking asynchronous log concurrency algorithm f(x) of the tracking log. Specifically, m i =f(x) f(x) T =[m1,m2,m3] Fault analysis and prediction based on cluster analysis algorithm, including: The first step is to parse all the tracking logs contained in m3 according to the number of tracking logs distributed in parallel to the offline data warehouse, and calculate the total number of upload requests n for the tracking logs analyzed by the offline data warehouse based on the number of original tracking log requests x: m3÷x=n In the second step, based on the calculated total number of upload requests n, let Δn be the manually set clustering analysis algorithm trigger condition parameter. When n meets the following conditions, start the offline data warehouse tracking log analysis: n ≥ Δn; In the third step, the tracking log analysis algorithm is as follows: The basic data frequency of the tracking log is defined as θ = (θ1, θ2, θ3, θ4, θ5, θ6, θ7, θ8, θ9), and the business data frequency is defined as The business data failure frequency type distance value d corresponding to each basic data in the uploaded tracking log x is: in Indicates the average number of occurrences of each ID in the basic data in one upload. Indicates the average number of occurrences of each ID in the business data; M θ Indicates the frequency mode of each ID in the basic data. Indicates the frequency mode of each ID in the business data. The mode indicates the number with the highest frequency in a group of numbers. The fourth step is to calculate the cosine similarity distance value cos(d) of all basic data and business data in a batch: Step 5: Based on the results of steps 4 and 3, calculate the fault distance value of the business data corresponding to the basic data in the tracking log within the upload period of this batch through clustering. The sixth step is to store the calculated results in a list. When the calculated value is obviously abnormal or exceeds the set threshold, it indicates a fault. The fault frequency of θ corresponding to θ can be located according to the D value, thereby locating the basic data record associated with a specific business data fault through cluster analysis of business data fault distance and basic data relationship. According to the cluster analysis algorithm, business updates, business recommendations, fault analysis, and fault prediction are realized to complete system upgrades and updates.

2. According to a high-concurrency data link embedding log analysis method according to claim 1, it is characterized in that: Obtain the upload request for the collected data link tracking log, including: The tracking log adopts a fixed process method to write the basic data of the tracking log of the data link and the business data into a string file to realize data collection, and the string file sends an upload request.

3. According to a high-concurrency data link embedding log analysis method according to claim 1, it is characterized in that: The basic data includes user ID, application coding identification ID, application version identification ID, operating system type identification ID, operating system version identification ID, device type ID, device name ID, communication address ID, and download channel ID; the business data includes log generation date ID, timestamp ID, log type ID, business embedding point number ID, and elastic business number ID.

4. According to a high-concurrency data link tracking log analysis method according to claim 1, it is characterized in that: Determine whether the trigger conditions for uploading are met, including: Determine whether the number of tracking logs exceeds the preset number; If it exceeds, it is determined whether the time interval from the last upload exceeds the preset time interval. If so, the trigger condition for upload is met.

5. According to a high-concurrency data link tracking log analysis method according to claim 1, it is characterized in that: The method specifically also includes: According to the calculated basic data fault distance value cos(d), determine which basic data are prone to failure, and trigger system update when the calculated value exceeds the set value; Then, according to the business data fault distance value corresponding to the calculated basic data Determine which business data is prone to failure based on basic data, and trigger a targeted update of the business data when the calculated value exceeds the set value; Regularly push targeted update services and message notifications to business data and basic data that are prone to failures based on the calculation results of D.

6. A high-concurrency data link embedding log analysis device, characterized in that: The device comprises: The tracking log upload trigger module is used to obtain the upload request of the collected data link tracking log and determine whether the trigger condition for upload is met. The tracking log data includes basic data and business data. The tracking point log distribution module is used to upload the tracking point log if the trigger condition is met, and calculate the number of tracking point logs to be distributed for different business requests according to the fixed process non-blocking asynchronous log distribution algorithm, and distribute the corresponding number of tracking point logs to different businesses according to the calculation results and the parallel distribution rules; The tracking log analysis module is used to perform fault analysis and prediction based on the basic data and business data of the tracking logs distributed in parallel to different businesses, based on the cluster analysis algorithm, to obtain the basic data and business data that are prone to faults in the tracking logs, and to output business updates and business recommendation solutions based on the results; According to the fixed process non-blocking asynchronous log distribution algorithm, the number of embedded logs to be distributed for different business requests is calculated, including: The services include real-time detailed query, offline data warehouse analysis, and real-time recommendation services; The number of uploaded tracking log data is x, i is the value sequence number range, and the number of real-time detailed query requests m1, real-time recommendation requests m2, and offline data warehouse analysis requests m3 are calculated according to the fixed process non-blocking asynchronous log concurrency algorithm f(x) of the tracking log. Specifically, m i =f(x) f(x) T =[m1,m2,m3] Fault analysis and prediction based on cluster analysis algorithm, including: The first step is to parse all the tracking logs contained in m3 according to the number of tracking logs distributed in parallel to the offline data warehouse, and calculate the total number of upload requests n for the tracking logs analyzed by the offline data warehouse based on the number of original tracking log requests x: m3÷x=n In the second step, based on the calculated total number of upload requests n, let Δn be the manually set clustering analysis algorithm trigger condition parameter. When n meets the following conditions, start the offline data warehouse tracking log analysis: n ≥ Δn; In the third step, the tracking log analysis algorithm is as follows: The basic data frequency of the tracking log is defined as θ = (θ1, θ2, θ3, θ4, θ5, θ6, θ7, θ8, θ9), and the business data frequency is defined as The business data failure frequency type distance value d corresponding to each basic data in the uploaded tracking log x is: in Indicates the average number of occurrences of each ID in the basic data in one upload. Indicates the average number of occurrences of each ID in the business data; M θ Indicates the frequency mode of each ID in the basic data. Indicates the frequency mode of each ID in the business data. The mode indicates the number with the highest frequency in a group of numbers. The fourth step is to calculate the cosine similarity distance value cos(d) of all basic data and business data in a batch: Step 5: Based on the results of steps 4 and 3, calculate the business data fault distance value D(θ,θ) corresponding to the basic data in the tracking logs within the upload period of this batch by clustering: D(θ,θ)=d×cos(d) The sixth step is to store the calculated results in a list. When the calculated value is obviously abnormal or exceeds the set threshold, it indicates a fault. The fault frequency of θ corresponding to θ can be located according to the D value, thereby locating the basic data record associated with a specific business data fault through cluster analysis of business data fault distance and basic data relationship. According to the cluster analysis algorithm, business updates, business recommendations, fault analysis, and fault prediction are realized to complete system upgrades and updates.

7. A high-concurrency data link point log analysis system, characterized in that: The system includes: a processor and a memory; The memory is used to store one or more program instructions; The processor is used to run one or more program instructions to execute the method according to any one of claims 1-5.

8. A computer storage medium, characterized in that: The computer storage medium contains one or more program instructions, and the one or more program instructions are used to be executed by a high-concurrency data link tracking log analysis system as described in any one of claims 1-5.

Images