Link log acquisition method and device, electronic equipment and storage medium
By determining the root data and weight of the link log, efficiently collecting link logs related to the target service, solving the problem of useless sampling in the existing technology, and achieving rapid collection of key link logs and effective analysis of service failures.
Patent Information
- Application Number
- CN202411101844.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-22
AI Technical Summary
There is no useless sampling in the existing link log sampling method, and the link logs of focus cannot be effectively collected, resulting in the inability to quickly determine service failures.
By obtaining the original link log set stored in the memory, the root data of each original link log is determined, and the target link log set is determined based on the weight of the target root data, and the first link log related to the target service is stored, so as to achieve Efficient collection of link logs is focused on.
It realizes efficient collection of link logs and quickly determines the link logs of focus, providing a basis for subsequent determination of business failures, avoids useless collection and repeated collection, and improves troubleshooting efficiency.
Smart Images

Figure CN119011383A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a link log collection method, device, electronic device and storage medium. Background Art
[0002] When the target business implements the corresponding functions, the data interaction behind it is complex, the number of server nodes and services called is large, and there are complex networking and calling relationships between services. In order to analyze and solve the abnormal situations of service processing, it is necessary to determine the calling relationship between services and the response time of each service node through link tracking (link log) to help technicians solve the faults in the service system.
[0003] The so-called link log is a request that goes through the interface calls of multiple projects. It strings together all the requests called in this request through a global ID. Through the global ID, all the system logs involved can be quickly located, making it convenient to troubleshoot problems when they occur online.
[0004] At present, the sampling of link logs is usually carried out according to a set ratio, head sampling or tail sampling. These methods have the problem of useless sampling and cannot collect the key link logs. Summary of the invention
[0005] The present invention provides a link log collection method, device, electronic device and storage medium to solve the problem that the key link logs cannot be collected. The present invention can realize efficient collection of link logs, quickly determine the key link logs, and provide a basis for subsequent determination of business failures.
[0006] According to one aspect of the present invention, a link log collection method is provided, the method comprising:
[0007] In response to a link log collection instruction of a target service, acquiring an original link log set stored in a first memory;
[0008] Determining root data of each original link log in the original link log set;
[0009] Determine the weight of each of the root data respectively, and determine the target link log set of the target root data according to the weight of the target root data;
[0010] A first link log related to a target service is determined in the target link log set, and each of the first link logs is stored in a second memory.
[0011] According to another aspect of the present invention, a link log collection device is provided, the device comprising:
[0012] An acquisition module, configured to acquire an original link log set stored in the first memory in response to a collection instruction of a link log of a target service;
[0013] A root data determination module, used to determine the root data of each original link log in the original link log set;
[0014] A weight determination module, used to determine the weight of each of the root data respectively, and determine the target link log set of the target root data according to the weight of the target root data;
[0015] The link log determination module is used to determine the first link log related to the target service in the target link log set, and store each of the first link logs in the second memory.
[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
[0017] at least one processor; and
[0018] a memory communicatively connected to the at least one processor; wherein,
[0019] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the link log collection method described in any embodiment of the present invention.
[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the link log collection method described in any embodiment of the present invention when executed.
[0021] The technical solution of the embodiment of the present invention obtains an original link log set stored in a first memory by responding to a collection instruction of a link log of a target business; determines the root data of each original link log in the original link log set; respectively determines the weight of each root data, and determines a target link log set of the target root data according to the weight of the target root data; determines the first link log related to the target business in the target link log set, and stores each of the first link logs in a second memory, which solves the problem of being unable to collect key link logs, can achieve efficient collection of link logs, can quickly determine key link logs, and provide a basis for subsequent determination of business failures.
[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0024] Figure 1 is a flow chart of a link log collection method provided according to Embodiment 1 of the present invention;
[0025] Figure 2 This is a schematic diagram of the structure of a link log collection device provided in Embodiment 2 of the present invention;
[0026] Figure 3 It is a structural schematic diagram of an electronic device that implements the link log collection method of an embodiment of the present invention. DETAILED DESCRIPTION
[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0029] Embodiment 1
[0030] Figure 1This is a flow chart of a link log collection method provided according to the first embodiment of the present invention. This embodiment is applicable to the case of efficiently collecting relevant link logs of target services. The method can be executed by a link log collection device, which can be implemented in the form of hardware and / or software. The link log collection device can be configured in electronic devices such as computers, servers or tablet computers. Figure 1 As shown, the method includes:
[0031] Step 110: In response to a collection instruction of a link log of a target service, an original link log set stored in a first memory is acquired.
[0032] Among them, the first memory can be an independent memory, for example, a cloud database or a memory of a cloud server, etc.; it can also be a memory of any electronic device, which is not limited in this embodiment.
[0033] In an optional implementation of this embodiment, the link collector may collect link logs generated by each service in the target system according to a preset collection ratio; and store each link log in the first memory.
[0034] Among them, the target system can be any system, for example, a shopping system, a query system or a search engine, etc., which is not limited in this embodiment; the target system can include multiple businesses; illustratively, for a shopping system, it can include an order generation business, an order evaluation business or a payment business, etc., which is not limited in this embodiment.
[0035] The target business involved in this embodiment is any business in the target system, such as the order generation business, order evaluation business or payment business involved above, which is not limited in this embodiment.
[0036] The preset collection ratio may be any ratio, such as collecting 5,000 logs for every 100,000 link logs or collecting 2,000 logs for every 50,000 link logs, etc., which is not limited in this embodiment.
[0037] Optionally, in this embodiment, the link logs generated by each business in the target system can be collected by a link collector according to a preset collection ratio, and these link logs can be stored in a first memory. It can be understood that these link logs stored in the first memory are the original link log set involved in this embodiment.
[0038] It should be noted that in this embodiment, the link collector can collect the original link log once every set time interval (for example, five minutes, ten minutes or twenty minutes, which is not limited in this embodiment), and delete the currently stored original link log set before the next collection.
[0039] The advantage of such a setting is that the original link log set can be continuously updated, and the link log related to the target business can be continuously and non-repetitively determined to ensure the target business and determine the failure of the target business in time.
[0040] In an optional implementation of this embodiment, after receiving an instruction to collect link logs of a target service, a processor of an electronic device equipped with a link log collection device may obtain an original link log set stored in a first memory.
[0041] It should be noted that the first memory and the processor involved in this embodiment can be deployed in the same electronic device or in different devices, which is not limited in this embodiment.
[0042] Step 120: Determine the root data of each original link log in the original link log set.
[0043] Among them, the root data of an original link log may include a root service and a root interface; the root data of different original link logs may be the same or different, and this is not limited in this embodiment; illustratively, the root data of the original link log A is "root service 1-root interface 2", the root data of the original link log B is "root service 1-root interface 2", and the root data of the original link log C is "root service 2-root interface 2".
[0044] In an optional implementation of this embodiment, determining the root data of each original link log in the original link log set may include: traversing each of the original link logs to determine the root node and root interface of each of the original link logs; and combining the root node and the root interface into one root data.
[0045] Optionally, in this embodiment, after obtaining the original link log set, the processor can traverse each original link log and determine the root node based on the traversal results; further, the interface containing the original link log where the root node is located can be determined, and the interface can be determined as the root interface; further, the root node and the root interface can be determined as a root data.
[0046] In this embodiment, in the process of traversing each original link log, if the target node of the target original link log is traversed to "parentid=null", then the target node is the root node of the target original link log. Exemplarily, the root node of any original link log can be "parented=null; spanid=12345; service=service23; interface=interface433; error=true; error_message=502".
[0047] Step 130: determine the weight of each of the root data respectively, and determine the target link log set of the target root data according to the weight of the target root data.
[0048] In an optional implementation of this embodiment, after the root data of each original link log is determined, the weight of each root data can be further determined. Furthermore, the target link log set of the target root data can be determined according to the weight of the target root data.
[0049] Optionally, in this embodiment, the weight of each root data is determined separately, and the target link log set of the target root data is determined according to the weight of the target root data, which may include: separately determining the target parameters of each root data; separately determining the calculation parameters of each root data according to the target parameters; forming a target matrix matching each root data according to each calculation parameter, and determining the weight of each root data according to the target matrix; determining the collection quantity of target link logs according to the target weight and the number of original link logs in the original link log set; collecting the target link logs according to the collection quantity to obtain the target link log set.
[0050] The target parameter of the target root data may be the request volume and error volume of each original link log matching the target root data; the calculation parameter may include at least one of the following: request volume ratio, error ratio, request growth rate, and error growth rate;
[0051] In an optional implementation of the present embodiment, after obtaining each root data, the request amount and error amount of each root data in the original log link set can be determined respectively; exemplarily, the original log link set contains five root data in total, the request amount for the first root data is 500, and the error amount is 50; the request amount for the second root data is 30, and the error amount is 2; the request amount for the third root data is 200, and the error amount is 50; the request amount for the fourth root data is 400, and the error amount is 350; the request amount for the fifth root data is 100, and the error amount is 90.
[0052] Furthermore, the calculation parameters of each root data can be determined according to the target parameters. In an optional implementation of the present embodiment, the calculation parameters of each root data are determined according to the target parameters, which may include: determining the total request volume of each root data; determining the target request volume ratio of the target root data according to the total request volume and the request volume of each original link log matching the target root data; determining the target error ratio of the target root data according to the total request volume and the error volume of each original link log matching the target root data; determining the request growth rate of the target root data according to the target request volume at the current moment and the reference request volume at the previous moment; determining the error growth rate of the target root data according to the target error ratio at the current moment and the reference error ratio at the previous moment.
[0053] It should be noted that, in an example of this embodiment, a moment can be 5 minutes, then the target request volume at the current moment is the request volume of the target root data within 5 minutes; the reference request volume at the previous moment is the request volume of the target root data within 5-10 minutes; the target error ratio at the current moment can be the error ratio of the target root data within 5 minutes; the reference error ratio at the previous moment is the request volume ratio of the target root data within 5-10 minutes.
[0054] In an optional implementation of the present embodiment, the total request volume of each root data can be determined. It can be understood that the total request volume of each root data can be obtained by adding up the request volumes of all root data; further, the target request volume ratio of the target root data can be determined based on the total request volume and the request volume of each original link log that matches the target root data; illustratively, the target request volume ratio of the target root data can be obtained by dividing the request volume of each original link log that matches the target root data by the total request volume.
[0055] Furthermore, the target error ratio of the target root data can be determined based on the total request volume and the error volume of each original link log matching the target root data; illustratively, the target error ratio of the target root data can be obtained by dividing the error volume of each original link log matching the target root data by the total request volume.
[0056] Furthermore, the request growth rate of the target root data can be determined based on the target request amount at the current moment and the reference request amount at the previous moment; illustratively, the request growth rate of the target root data can be obtained by subtracting the result of the reference request amount at the previous moment from the target request amount at the current moment and dividing it by the reference request amount at the previous moment.
[0057] Furthermore, the error growth rate of the target root data can be determined based on the target error ratio at the current moment and the reference error ratio at the previous moment; illustratively, the error growth rate of the target root data can be obtained by subtracting the result of the reference error ratio at the previous moment from the target error ratio at the current moment and dividing it by the reference error ratio at the previous moment.
[0058] For example, Table 1 shows the target parameters of each root data at the current moment and the calculation results of the calculation parameters:
[0059] Table 1
[0060]
[0061]
[0062] Table 2 shows the target parameters of each root data at the last moment and the calculation results of the calculation parameters:
[0063] Table 2
[0064]
[0065] In the above table, "root service-root interface" constitutes one root data.
[0066] In an optional implementation of this embodiment, after the calculation parameters of each root data are determined according to the target parameters, a target matrix matching each root data can be further formed according to each calculation parameter, and the weight of each root data can be determined according to the target matrix.
[0067] Optionally, in this embodiment, a target matrix matching each of the root data is formed according to each of the calculation parameters, and the weight of each of the root data is determined according to the target matrix, which may include: quantizing each of the calculated calculation parameters according to a preset configuration rule to obtain the target matrix; and determining the weight of each of the root data according to the quantized value of each of the calculation parameters in the target matrix.
[0068] Among them, the preset configuration rules can be that the request proportion in the calculation results is less than or equal to 10% as low, greater than 10% and less than 30% as medium, and greater than or equal to 30% as high; the error proportion is less than or equal to 10% as low, greater than 10% and less than 40% as medium, and greater than or equal to 40% as high; the request growth rate is less than or equal to 20% as low, greater than 10% and less than 50% as medium, and greater than or equal to 50% as high; the error proportion growth rate is less than or equal to 0% as low, greater than 0% and less than 20% as medium, and greater than or equal to 20% as high; it should be noted that the preset configuration rules can also be other rules, which are not limited in this embodiment.
[0069] Furthermore, the calculation results of the calculation parameters may be quantified according to preset rules to obtain a target matrix. For example, for the above example, the target matrix may be as shown in Table 3:
[0070] Table 3
[0071]
[0072] Furthermore, the weight of each root data may be determined according to the quantized value of each calculation parameter in the target matrix.
[0073] In this embodiment, the quantization value "low" of the target matrix in Table 3 can be determined as "0", the quantization value "medium" can be determined as "1", and the quantization value "high" can be determined as "2"; it should be noted that in this embodiment, the target matrix can also be quantized in other ways, for example, the quantization value "low" can be determined as "1", the quantization value "medium" can be determined as "2", the quantization value "high" can be determined as "3", etc., which is not limited in this embodiment.
[0074] Exemplarily, in the above example, if the quantization value "low" of the target matrix is determined as "0", the quantization value "medium" is determined as "1", and the quantization value "high" is determined as "2", then the quantization result for the root data [service 1-interface 1] is 1, the quantization result for the root data [service 2-interface 2] is 2, the quantization result for the root data [service 1-interface 3] is 3, the quantization result for the root data [service 3-interface 4] is 5, and the quantization result for the root data [service 3-interface 5] is 4. The total number of these root data is 15, so the weights of these root data are 1 / 15, 2 / 15, 3 / 15, 5 / 15 and 4 / 15 respectively.
[0075] Optionally, in this embodiment, after the weight of each root data is determined, the number of target link logs to be collected may be further determined according to the target weight and the number of original link logs in the original link log set.
[0076] The target weight can be the weight of any root data, for example, it can be 1 / 15 of the weight of the root data [Service 1-Interface 1] in the above example, or it can be 4 / 15 of the weight of the root data [Service 3-Interface 5], which is not limited in this embodiment. For example, if the number of original link logs in the original link log set is 5000, then the number of collections for the root data [Service 3-Interface 4] is 5000*5 / 15, about 1333.
[0077] Furthermore, a target link log set can be obtained by collecting a target number of link logs according to the target weight of the target root data.
[0078] Step 140: Determine first link logs related to the target service in the target link log set, and store each of the first link logs in a second memory.
[0079] In an optional implementation of this embodiment, after obtaining the target link log set, first link logs related to the target service may be further determined in the target link log set, and each first link log may be stored in the second memory.
[0080] Optionally, in this embodiment, determining the first link log related to the target business in the target link log set may include: determining the business identifier of the target business; identifying the link identifier of each link log in the target link log set, and comparing each link identifier with the business identifier respectively; determining each reference link log that matches the business identifier based on the comparison result, and determining each reference link log as the first link log.
[0081] The service identifier of the target service may be the traceid of the target service or other identifiers, which are not limited in this embodiment. The link identifier of the link log may be the traceid of the link log or other identifiers, which are not limited in this embodiment. It should be noted that the service identifier of the target service selected in this embodiment is the same identifier as the link identifier of the link log.
[0082] In this embodiment, after identifying the service identifier traceid of the target service, the link identifier traceid of each link log in the target link log set can be further identified. Furthermore, the service identifier traceid can be compared with the link identifier traceid of each link log respectively. If the comparison result is consistent, that is, the service identifier traceid is consistent with the link identifier traceid of the target link log, the target link log is determined as the first link log, and the target link log is stored in the second memory.
[0083] It should be noted that the second memory involved in this embodiment can be in the same device as the first memory, but it represents a different storage area; it can also be in a different device from the first memory, so that the link log of the target service can be better distinguished from the original link log.
[0084] The technical solution of this embodiment obtains an original link log set stored in a first memory by responding to a collection instruction of a link log of a target business; determines the root data of each original link log in the original link log set; determines the weight of each root data respectively, and determines a target link log set of the target root data according to the weight of the target root data; determines the first link log related to the target business in the target link log set, and stores each of the first link logs in a second memory, which solves the problem of being unable to collect key link logs, can achieve efficient collection of link logs, can quickly determine key link logs, and provide a basis for subsequent determination of business failures.
[0085] In order to better understand the technical solution of the embodiment of the present invention, a specific example is used below to illustrate the embodiment of the present invention:
[0086] 1. The link collector collects the original logs of the request and stores them in storage 1. At the same time, the link collector can be configured with the minimum collection quantity and collection ratio (for example, 5,000 records are collected for 100,000 requests).
[0087] 2. [Processor 1] pulls the data from [Memory 1] for analysis, and analyzes the number of requests and errors of the root node within 10-5 minutes and within 5 minutes (the time here can be customized) (the root node is the service and interface, which is the field in the original log. Users can use other log fields in different scenarios. The root node is determined by parentid = null in the log.)
[0088] Among them, request volume ratio = root service interface request volume / total request volume; (total request volume = total request volume); error ratio = root service interface error volume / root service interface request volume; request volume growth rate = (requests within 5 minutes - requests 10 - 5 minutes ago) / requests 10 - 5 branches ago; error ratio growth rate = (error ratio within 5 minutes - error ratio 10 - 5 branches ago) / error ratio 10 - 5 branches ago.
[0089] Furthermore, the matrix can be divided according to certain rules (configurable below) such as:
[0090] Request ratio: <=10% is low, 10%-30% is medium, >30% is high; Error ratio: <=10% is low, 10%-40% is medium, >40% is high; Request growth rate: <=20% is low, 20%-50% is medium, >50% is high; Error ratio growth rate: <0% is low, 0%-20% is medium, >20% is high.
[0091] 3. Low is 0 points, medium is 1 point, and high is 2 points; therefore, [Service 1 Interface 1] is 1 point, [Service 2 Interface 2] is 2 points, [Service 2 Interface 3] is 3 points, [Service 3 Interface 4] is 5 points, and [Service 3 Interface 5] is 4 points. The total score is 15 points. The weight coefficients are 1 / 15, 2 / 15, 3 / 15, 5 / 15, and 4 / 15 respectively.
[0092] 4. According to step 3, among the log data of the last 5 minutes, assuming that a fixed ratio of 100,000 is set to be retained, how to better store the key links? According to the above rules, the number of collection items retained for [Service 3-Interface 4] is 5000*5 / 15, which is about 1333 items.
[0093] 5. [Processor] obtains the retained root node log and obtains the relevant traceid
[0094] 6. The processor sends the traceid in the retained log to storage 1. Storage 1 obtains the relevant traceid and then sends all logs related to the traceid to storage 2 (there are multiple logs for the same traceid). Subsequently, all logs related to the traceid will be sent and saved in storage 2.
[0095] 7. [Memory 1] has determined that the log data sent to [Memory 2] will not be obtained by the [Processor] to avoid duplication.
[0096] 8. [Storage 1] Delete data every 20 minutes (configurable).
[0097] The solution of this embodiment can avoid the defects of fixed collection, head collection, tail collection and other collection methods through short-term analysis and matrix. It can find more key interfaces and links; dynamically adjust the interface services that need to be paid attention to according to the actual online services, set a larger priority for the normal weight of the newly launched services, and adjust the weight of the normal attention after stability; it can avoid excessive useless collection and loss of server performance and storage.
[0098] Embodiment 2
[0099] Figure 2 1 is a schematic diagram of the structure of a link log collection device provided according to Embodiment 2 of the present invention. Figure 2 As shown, the device includes: an acquisition module 210, a root data determination module 220, a weight determination module 230 and a link log determination module 240.
[0100] The acquisition module 210 is used to acquire the original link log set stored in the first memory in response to the collection instruction of the link log of the target service;
[0101] A root data determination module 220, configured to determine the root data of each original link log in the original link log set;
[0102] A weight determination module 230, used to determine the weight of each of the root data respectively, and determine the target link log set of the target root data according to the weight of the target root data;
[0103] The link log determination module 240 is configured to determine first link logs related to a target service in the target link log set, and store each of the first link logs in a second memory.
[0104] The solution of this embodiment is to obtain the original link log set stored in the first memory by the acquisition module in response to the collection instruction of the link log of the target business; determine the root data of each original link log in the original link log set by the root data determination module; determine the weight of each root data respectively by the weight determination module, and determine the target link log set of the target root data according to the weight of the target root data; determine the first link log related to the target business in the target link log set by the link log determination module, and store each of the first link logs in the second memory, which solves the problem of being unable to collect the link logs of key concern, can achieve efficient collection of link logs, can quickly determine the link logs of key concern, and provide a basis for subsequent determination of business failures.
[0105] In an optional implementation of this embodiment, the link log collection device further includes: a collection module, which is used to collect link logs generated by each service in the target system through a link collector according to a preset collection ratio;
[0106] Each of the link logs is stored in the first memory.
[0107] In an optional implementation of this embodiment, the root data determination module 220 is specifically used to traverse each of the original link logs to determine the root node and the root interface of each of the original link logs;
[0108] The root node and the root interface form a piece of root data.
[0109] In an optional implementation of this embodiment, the weight determination module 230 is specifically used to determine the target parameters of each of the root data respectively;
[0110] Determine the calculation parameters of each of the root data respectively according to the target parameters;
[0111] forming a target matrix matching each of the root data according to each of the calculation parameters, and determining a weight of each of the root data according to the target matrix;
[0112] Determine the collection quantity of the target link logs according to the target weight and the quantity of the original link logs in the original link log set;
[0113] The target link logs are collected according to the collection quantity to obtain a target link log set.
[0114] In an optional implementation of this embodiment, the target parameters of the target root data include the request amount and the error amount of each original link log matching the target root data;
[0115] The calculation parameters include at least one of the following: request volume ratio, error reporting ratio, request growth rate, and error reporting growth rate;
[0116] Accordingly, the weight determination module 230 is also used to determine the total request amount of each root data;
[0117] Determine a target request volume ratio of the target root data according to the total request volume and the request volume of each original link log matching the target root data;
[0118] Determine a target error ratio of the target root data according to the total request volume and the error volume of each original link log matching the target root data;
[0119] Determine the request growth rate of the target root data according to the target request volume at the current moment and the reference request volume at the previous moment;
[0120] The error growth rate of the target root data is determined according to the target error ratio at the current moment and the reference error ratio at the previous moment.
[0121] In an optional implementation of this embodiment, the weight determination module 230 is further configured to quantize each calculated calculation parameter according to a preset configuration rule to obtain the target matrix;
[0122] The weight of each root data is determined according to the quantized value of each calculation parameter in the target matrix.
[0123] In an optional implementation of this embodiment, the link log determination module 240 is specifically used to determine the service identifier of the target service;
[0124] Identify the link identifier of each link log in the target link log set, and compare each link identifier with the service identifier respectively;
[0125] The reference link logs matching the service identifier are determined according to the comparison result, and the reference link logs are determined as first link logs.
[0126] The link log collection device provided in the embodiment of the present invention can execute the link log collection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0127] In the technical solution of the embodiment of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of user personal information (such as link logs, etc.) are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0128] Embodiment 3
[0129] Figure 3 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0130] like Figure 3 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0131] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0132] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the link log collection method.
[0133] In some embodiments, the link log collection method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the link log collection method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the link log collection method in any other appropriate manner (for example, by means of firmware).
[0134] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0135] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0136] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0137] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0138] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0139] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0140] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0141] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A link log collection method, characterized in that: include: In response to a link log collection instruction of a target service, acquiring an original link log set stored in a first memory; Determining root data of each original link log in the original link log set; Determine the weight of each of the root data respectively, and determine the target link log set of the target root data according to the weight of the target root data; A first link log related to a target service is determined in the target link log set, and each of the first link logs is stored in a second memory.
2. The link log collection method according to claim 1, characterized in that: Before acquiring the original link log set stored in the first memory, the method further includes: The link collector collects link logs generated by each business in the target system according to the preset collection ratio; Each of the link logs is stored in the first memory.
3. The link log collection method according to claim 1, characterized in that: The step of determining the root data of each original link log in the original link log set includes: Traversing each of the original link logs to determine a root node and a root interface of each of the original link logs; The root node and the root interface form a piece of root data.
4. The link log collection method according to claim 1, characterized in that: The step of respectively determining the weight of each of the root data and determining the target link log set of the target root data according to the weight of the target root data includes: respectively determining target parameters of each of the root data; Determine the calculation parameters of each of the root data respectively according to the target parameters; forming a target matrix matching each of the root data according to each of the calculation parameters, and determining a weight of each of the root data according to the target matrix; Determine the collection quantity of the target link logs according to the target weight and the quantity of the original link logs in the original link log set; The target link logs are collected according to the collection quantity to obtain a target link log set.
5. The link log collection method according to claim 4, characterized in that: The target parameters of the target root data include the request amount and error amount of each original link log matching the target root data; The calculation parameters include at least one of the following: request volume ratio, error reporting ratio, request growth rate, and error reporting growth rate; Correspondingly, determining the calculation parameters of each root data according to the target parameter includes: Determine the total number of requests for each root data; Determine a target request volume ratio of the target root data according to the total request volume and the request volume of each original link log matching the target root data; Determine a target error ratio of the target root data according to the total request volume and the error volume of each original link log matching the target root data; Determine the request growth rate of the target root data according to the target request volume at the current moment and the reference request volume at the previous moment; The error growth rate of the target root data is determined according to the target error ratio at the current moment and the reference error ratio at the previous moment.
6. The link log collection method according to claim 5, characterized in that: The forming a target matrix matching each of the root data according to each of the calculation parameters, and determining a weight of each of the root data according to the target matrix, comprises: According to a preset configuration rule, each calculated calculation parameter is quantified to obtain the target matrix; The weight of each root data is determined according to the quantized value of each calculation parameter in the target matrix.
7. The link log collection method according to claim 1, characterized in that: Determining a first link log related to a target service in the target link log set includes: Determine the business identity of the target business; Identify the link identifier of each link log in the target link log set, and compare each link identifier with the service identifier respectively; The reference link logs matching the service identifier are determined according to the comparison result, and the reference link logs are determined as first link logs.
8. A link log collection device, characterized in that: include: An acquisition module, configured to acquire an original link log set stored in the first memory in response to a collection instruction of a link log of a target service; A root data determination module, used to determine the root data of each original link log in the original link log set; A weight determination module, used to determine the weight of each of the root data respectively, and determine the target link log set of the target root data according to the weight of the target root data; The link log determination module is used to determine the first link log related to the target service in the target link log set, and store each of the first link logs in the second memory.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the link log collection method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the link log collection method according to any one of claims 1 to 7 when executed.
