Link data processing method and device, equipment and storage medium

Abstract

The invention provides a link data processing method and device, equipment and a storage medium. The link data processing method comprises the following steps: collecting link data under a plurality of service requests; matching the plurality of link data according to at least one of the following items: preset sampling configuration information, collection times of the plurality of service requests, and anomaly detection results of the plurality of service requests to obtain a matching result; and the matching result is sent to the server, so that the server stores or discards the multiple pieces of link data according to the matching result, the storage cost of the link data can be reduced, the quality of the collected link data can be improved, and the use effect of the link data can be ensured.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to a link data processing method, apparatus, device, and storage medium. Background Art

[0002] Link data refers to the path and related information formed when a business request is transmitted between multiple microservices in a microservice architecture, such as microservice name, service call time, service call sequence, request response status, etc. By collecting and analyzing link information, you can understand the flow process of business requests, the interaction relationship between services, and performance bottlenecks and failure points. Currently, all collected link data is stored, so there is a high storage cost. Summary of the invention

[0003] The present application provides a link data processing method, device, equipment and storage medium, which can reduce the storage cost of link data, improve the quality of collected link data, and ensure the use effect of link data.

[0004] In a first aspect, a link data processing method is provided, comprising: collecting link data under multiple business requests; matching the multiple link data according to at least one of the following: preset sampling configuration information, the number of collection times of multiple business requests, and the abnormal detection results of multiple business requests to obtain matching results; sending the matching results to a server, so that the server stores or discards the multiple link data according to the matching results.

[0005] In a second aspect, a link data processing method is provided, comprising: receiving a matching result sent by a client; storing or discarding link data under multiple service requests based on the matching result; wherein the matching result is obtained by matching link data under multiple service requests based on at least one of the following: preset sampling configuration information, the number of collection times of multiple service requests, and anomaly detection results of multiple service requests.

[0006] In a third aspect, a link data processing device is provided, including: a first acquisition module, used to collect link data under multiple business requests; a first matching module, used to match multiple link data according to at least one of the following: preset sampling configuration information, the number of collection times of multiple business requests, and the abnormal detection results of multiple business requests to obtain matching results; a result sending module, used to send the matching results to a server, so that the server stores or discards the multiple link data according to the matching results.

[0007] In a fourth aspect, a link data processing device is provided, comprising: a result receiving module for receiving a matching result sent by a client; a result processing module for storing or discarding link data under multiple service requests according to the matching result; wherein the matching result is obtained by matching link data under multiple service requests according to at least one of the following: preset sampling configuration information, the number of collection times of multiple service requests, and the anomaly detection results of multiple service requests.

[0008] In a fifth aspect, the present application provides an electronic device, comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing the method in the first aspect, the second aspect or their respective implementations.

[0009] In a sixth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute a method as in the first aspect, the second aspect, or any implementation thereof.

[0010] In a seventh aspect, the present application provides a computer program product, comprising computer program instructions, which enable a computer to execute the method in the first aspect, the second aspect or their respective implementations.

[0011] In an eighth aspect, the present application provides a computer program, which enables a computer to execute the method in the first aspect, the second aspect, or each implementation thereof.

[0012] In the technical solution of the present application, the link data under multiple business requests can be matched according to at least one of the preset sampling configuration information, the number of collection times of multiple business requests, and the abnormal detection results of multiple business requests, so as to determine whether it needs to be stored. Therefore, not only the storage cost of the link data can be reduced, but also the quality of the collected link data can be improved to ensure the use effect of the link data. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The following is an introduction to the drawings required for describing the embodiments.

[0014] Figure 1 A flowchart of a link data processing method provided in an embodiment of the present application;

[0015] Figure 2 A flowchart of another link data processing method provided in an embodiment of the present application;

[0016] Figure 3 A schematic diagram of a link data processing method provided in an embodiment of the present application;

[0017] Figure 4A schematic diagram of a link data processing device 400 provided in an embodiment of the present application;

[0018] Figure 5 A schematic diagram of a link data processing device 500 provided in an embodiment of the present application;

[0019] Figure 6 It is a schematic diagram of an electronic device 600 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The technical solution of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0021] It should be noted that the information, data (including, but not limited to: data for analysis, data for storage, data for display, etc., such as link data, sampling configuration information, the number of collection times of business requests, abnormal detection results of business requests, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the link data involved in this application and the operations performed on the link data are all obtained with full authorization.

[0022] In one embodiment, the technical solution of the present application can be used in monitoring and sampling scenarios of link data. Specifically, the link data can be data under any service request, for example, the service request can be a request under a transfer service or a login service, but is not limited thereto.

[0023] In one embodiment, the solution provided by the present application can be executed by any terminal device and server with data processing capabilities, for example: the server, which can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services; the terminal device, which can be a tablet computer, a laptop computer or a desktop computer, etc.

[0024] Figure 1 A flowchart of a link data processing method provided in an embodiment of the present application, which method can be executed by the terminal device in the above embodiment, such as Figure 1 As shown, the method includes:

[0025] S110: collecting link data under multiple service requests;

[0026] S120: Matching the multiple link data to obtain a matching result according to at least one of the following: preset sampling configuration information, the number of collection times of the multiple service requests, and the abnormality detection results of the multiple service requests;

[0027] S130: Send the matching result to the server, so that the server stores or discards the multiple link data according to the matching result.

[0028] In one embodiment, a client can be installed on a terminal device, and the client can include an Agent. The Agent is generally a software component or program that can collect, process and report link data such as request links and performance of applications and business services. Specifically, the Agent can be deployed together with multiple business requests (for example, the Agent can be integrated in the execution environment of the business request to capture link data in real time), and the link data under multiple business requests can be collected by the Agent, and subsequent link data matching is performed and the matching results are sent to the server. Of course, the matching of link data and the sending of matching results to the server can also be performed by other modules or units of the client, and this application does not limit this.

[0029] Before matching the link data, the client can first display the sampling configuration page; the user, for example, the link monitoring personnel can select or input the preset interface information in the sampling configuration page based on the client; then, the client can obtain the preset interface information and send it to the server, so that the server determines the sampling configuration information according to the preset interface information; thereafter, the server can send the sampling configuration information to the client, and the client can receive the sampling configuration information sent by the server.

[0030] Exemplarily, the sampling configuration information may include at least one of the following: preset interface information, sampling setting information of multiple service requests, and server load information.

[0031] Among them, the preset interface information may include at least one of the following: interface characters and specific attribute parameters corresponding to the preset interface. The preset interface may be at least one of the following interfaces, but is not limited to: an interface whose relevance to the business request is greater than the preset relevance threshold (for example, it may be an interface for obtaining the transfer amount in a transfer business request), any or all interfaces involved in a specific business request (a specific business request may be a core key business request among multiple business requests, that is, its corresponding business importance is greater than the importance threshold); the interface character may be the interface name or interface identifier of the preset interface, etc.; the specific attribute parameter may be data based on the key-value method, and may include the key feature code of the preset interface, which is a character sequence that uniquely identifies the interface, and may include key information such as the name of the interface, parameter type, return type, version information, etc. For example, the interface name may correspond to key, and the parameter type, return type, version information, etc. may correspond to value.

[0032] The sampling setting information of the service request may include at least one of the following: the maximum sampling rate and sampling weight of the service request. The sampling setting information may be selected or output by the user based on the client, and the client may send it to the server so that the server determines the sampling configuration information according to the sampling setting information.

[0033] The server load information is information used to describe the current operating status and performance of the server. It can be obtained locally by the server and may include at least one of the following: network status information (for example, network bandwidth information such as upload and download speeds, packet loss rate, etc.), process and thread operating status, disk read and write operation data, memory usage, central processing unit (CPU) usage, and application performance indicators (for example, database query response time).

[0034] After obtaining the sampling configuration information, the client can use the sampling configuration information and the number of collection times and anomaly detection results of the service request to match multiple link data to obtain a matching result, which is introduced below.

[0035] In one embodiment, the client may first match multiple link data according to at least one of the sampling configuration information, the number of collection times of multiple business requests, and the abnormality detection results of multiple business requests, and determine the first link data that successfully matches among the multiple link data; then, the matching result may be determined according to the first link data and the second link data among the multiple link data except the first link data.

[0036] Among them, the second link data can be understood as link data that has not been successfully matched among multiple link data. For example, it can be link data that does not match any of the sampling configuration information, the number of collection times of multiple business requests, and the abnormal detection results of multiple business requests.

[0037] Correspondingly, the above-mentioned successful matching may refer to a successful matching with at least one of the sampling configuration information, the number of collection times of multiple service requests, and the abnormality detection results of multiple service requests.

[0038] Exemplarily, the matching result may be a combination result of the first link data and the second link data.

[0039] For example, a sampling tag may be set for the first link data; then, the matching result may be determined based on the first link data with the sampling tag and the second link data other than the first link data in the plurality of link data. Specifically, for the plurality of link data, a sampling tag may be set for each first link data that is determined to be successfully matched, and the first link data with the sampling tag may be added to the matching result set, and each second link data that is determined to be unsuccessful in matching may be directly added to the matching result set without the sampling tag, and finally, the matching result set may be determined as the matching result.

[0040] For example, the first link data may be combined into a first link data set, and the second link data may be combined into a second link data set; then, the first link data set and the second link data set are combined to obtain a matching result. In this case, no label may be set for the first link data, and the first link data and the second link data may be distinguished through two different sets, so that the subsequent server may determine whether to store them accordingly.

[0041] For example, in combination with the above embodiment, different sampling tags may be set for the first link data and the second link data, and then the first link data and the second link data with different sampling tags are combined to obtain a matching result.

[0042] Specifically, setting a sampling tag for the link data may include: marking the link data with a sampling tag, that is, carrying the sampling tag in the link data. For example, a new field or attribute may be added to the data structure of the link data, and the sampling tag may be added to the field or attribute. Alternatively, the sampling tag may be directly spliced ​​at the data header or the data tail of the link data.

[0043] In the above content, the first link data and the second link data can be distinguished in the matching results by different sets, whether the same set carries a label, or different labels are carried in the same set, so that the server can distinguish the first link data and the second link data, thereby storing the first link data process and discarding the second link data (or further screening to determine whether to store it), that is, selective storage is performed, which not only reduces the storage cost, but also ensures that the stored link data is of higher quality and usage effect.

[0044] The specific matching process is introduced below:

[0045] In one embodiment, for any target link data among the multiple link data, according to at least one of the following: sampling configuration information (including at least one of the following: preset interface information, sampling setting information of multiple service requests, load information of the server), the number of collection times of multiple service requests, and the abnormality detection results of multiple service requests, the target link data is matched, and the first link data that successfully matches among the multiple link data is determined, including at least one corresponding situation of the following situations:

[0046] In case 1, the target interface information in the target link data is matched with the preset interface information, and in response to the target interface information matching with the preset interface information, the target link data is determined to be the first link data.

[0047] Exemplarily, all characters (for example, all characters corresponding to the interface name), partial characters (for example, characters corresponding to the interface prefix (which can be a part of the interface name)) or specific attribute parameters corresponding to the target interface information and the preset interface information can be matched, and in response to at least one of all characters, partial characters or specific attribute parameters corresponding to the target interface information and the preset interface information successfully matching, the target link data is determined to be the first link data.

[0048] For example, the matching of some of the above characters can be achieved according to a regular expression, and the regular expression can be "(^ / ?(? <prefix> / [^ / \\? #]+)+ / ? $)". When matching only according to specific attribute parameters, the matching method can be based on key-value. For example, the characters corresponding to the target interface information, such as the interface name, can be searched in the key of the specific attribute parameter. If not found, it is determined that the match fails. If found, it is determined that the match succeeds. Alternatively, the attribute parameters of the target interface information, such as parameter type, return type, version information, etc., are continued to be searched in the value of the specific attribute parameter. If found, the match succeeds, otherwise, the match fails.

[0049] In case one, key feature matching of link data can be performed through full matching, partial matching, and key feature code matching, so that link data that meets the preset interface information, that is, matches the preset interface information, can be screened out, thereby realizing the collection of link data that has key value for business services and meets business needs.

[0050] Case 2: determine whether the number of collection times of the target service request corresponding to the target link data within the preset time is less than the first preset number. In response to the number of collection times within the preset time being less than the first preset number, determine that the target link data is the first link data.

[0051] Exemplarily, the preset time may be 1 minute, and the first preset number may be 1, but is not limited thereto.

[0052] Exemplarily, a sliding window with a corresponding time of 1 minute can be set. If the target link data is sampled for the first time during the link data collection process for the target service request within the sliding window, that is, the target link data is collected once, then the target link data can be determined to be the first link data.

[0053] In the second case, the number of times the link data is collected is taken into consideration, so that the link data with fewer collection times can be retained, thereby ensuring that at least one sample is captured for each interface within the preset time, thereby achieving the integrity and comprehensiveness of the sampled data.

[0054] Case three: determine whether the abnormality detection result of the target business request is a request abnormality. In response to the abnormality detection result of the target business request being a request abnormality, determine that the target link data is the first link data.

[0055] Exemplarily, the anomaly detection result of the target business request includes at least one of the following: the response time of the target business request, the response status code of the target business request (used to identify whether the response result corresponding to the target business request is abnormal, including an abnormal status code that identifies the response result as abnormal and a normal status code that identifies the response result as normal), and an identification result of identifying the target business request based on the first anomaly detection algorithm and the first business request data corresponding to the target business request.

[0056] Correspondingly, the anomaly detection result of the target business request is a request anomaly, including at least one of the following: the response duration of the target business request is greater than the duration threshold, the response status code of the target business request is an abnormal status code, and the identification result is an abnormal identification result.

[0057] For example, the first service request data may include at least one of the following: a request timestamp of the target service request, a Uniform Resource Locator (URL) of the request, request parameters, a response status code, a response time, etc. The first anomaly detection algorithm may be an anomaly detection algorithm such as a Z-score or an IQR (interquartile range).

[0058] In situation three, errors and slow requests in the link data with characteristics such as too high request response time and returned abnormal status code can be screened out, and abnormal requests can be identified, so as to realize the collection of data with high value for analyzing link information and business flow, and improve the value and effectiveness of the stored link data.

[0059] Case 4: determining the remaining number of samples of the target service request according to the sampling setting information of the target service request and the load information of the server, and in response to the remaining number of samples being greater than 0, determining that the target link data is the first link data.

[0060] Exemplarily, the target sampling rate can be determined based on the server load information and the sampling weight of the target service request, and the target sampling rate is less than or equal to the maximum sampling rate of the target service request; then, the existing sampling number of the first link data corresponding to the target service request is determined among the multiple link data; finally, the remaining sampling number can be determined based on the existing sampling number and the target sampling rate. Specifically, the maximum sampling number corresponding to the target service request can be determined based on the target sampling rate (specifically, it can be the product of the target sampling rate and the total data number corresponding to the target service request), and the difference between the maximum sampling number and the existing sampling number is determined as the remaining sampling number.

[0061] For example, the sampling weight "Service Weight" of the target business request can be determined first, which can be determined specifically according to the weight of the business service corresponding to the target business request among multiple business services; at the same time, the load information "Host Load" of the server can be obtained, and then the target sampling rate Sampling Rate = g (Service Weight, HostLoad) can be determined according to the weight function g (a function that can map the sampling weight and load information to the sampling rate) and the sampling weight and load information. For example, the weight function can be a linear function, Sampling Rate = a*Service Weight-b*HostLoad+c, where a, b and c are the coefficients of the weight function, which can be determined based on the application scenario of the business request and the corresponding business needs.

[0062] In situation four, the sampling rate of business requests can be adaptively adjusted according to the actual load of the server's network and storage and the changes in business traffic, that is, the sampling of business requests. When the sampling quota is available, that is, the number of remaining samples is greater than 0, the link data under the business request is sampled and stored to achieve an effective balance between link data integrity and storage cost.

[0063] In one embodiment, multiple link data can be matched in a preset matching order based on the sampling setting information and the server load information, and at least one of the following: preset interface information, the number of collection times of multiple business requests, and the abnormal detection results of multiple business requests to determine the first link data; wherein the matching order corresponding to the sampling setting information and the server load information is at the end of the preset matching order.

[0064] It is understandable that, since the number of existing samples of the determined first link data needs to be determined when matching the sampling setting information with the server load information, the matching order corresponding to the sampling setting information and the server load information needs to be set last.

[0065] In the above process, matching is performed in sequence according to a preset matching order, thereby avoiding repeated attempts and invalid operations that may be caused by disordered matching, thereby improving matching accuracy and efficiency.

[0066] The link data processing method involved in the present application is introduced below from the server perspective, wherein the embodiments corresponding to the server side and the embodiments corresponding to the client side can refer to each other.

[0067] Figure 2 This is a flowchart of another link data processing method provided in an embodiment of the present application. The method can be executed by the server in the above embodiment, such as Figure 2 As shown, the method includes:

[0068] S210: receiving the matching result sent by the client;

[0069] S220: According to the matching result, the link data under the multiple service requests are stored or discarded.

[0070] The matching result is obtained by matching link data under multiple service requests according to at least one of the following: preset sampling configuration information, collection times of multiple service requests, and abnormality detection results of multiple service requests.

[0071] In one embodiment, before receiving the matching result sent by the client, the server may also receive preset interface information sent by the client; then, determine sampling configuration information according to the preset interface information; and send the sampling configuration information to the client, so that the client determines the matching result according to the sampling configuration information.

[0072] Exemplarily, in combination with the above embodiment, the server may combine at least one of the preset interface information, the sampling setting information of the multiple service requests, and the load information of the server to obtain the sampling configuration information.

[0073] In one embodiment, in combination with the above embodiment, the matching result may include: first link data that is successfully matched and second link data that is not successfully matched.

[0074] If the first link data in the matching result is provided with a sampling tag and the second link data is not provided with a sampling tag, the server may first identify the matching result according to the sampling tag to determine the first link data and the second link data.

[0075] If neither the first link data nor the second link data in the matching result is set with a sampling tag and is in different link data sets, the server may determine the first link data and the second link data based on different link data sets (ie, the first link data set and the second link data set).

[0076] If the first link data and the second link data in the matching result are set with different sampling tags, the server may identify the matching result according to the different sampling tags to determine the first link data and the second link data.

[0077] In one embodiment, the above S220 may include:

[0078] S220-1: Store the first link data.

[0079] Exemplarily, the server may perform persistent storage on the first link data.

[0080] It can be understood that the first link data is link data that matches at least one of the sampling setting information and the server load information, preset interface information, the number of collection times of multiple business requests, and the anomaly detection results of multiple business requests. Therefore, it is data with key value for analyzing business services and business requests. Storing it can ensure that the collected link data is valid and of high quality.

[0081] S220-2: Store or discard the second link data according to the abnormality detection result of the second link data and / or the number of collection times of the service request corresponding to the second link data.

[0082] Exemplarily, in response to the abnormality detection result of the second link data being a request abnormality, the second link data can be stored; in response to the abnormality detection result of the second link data being a request normal, it is determined whether the number of collection times of the business request corresponding to the second link data within the preset time is less than the second preset number; if the number of collection times within the preset time is less than the second preset number, the second link data is stored; if the number of collection times within the preset time is greater than or equal to the second preset number, the second link data is discarded.

[0083] Among them, the anomaly detection result of the second link data is: based on the second anomaly detection algorithm and the second business request data corresponding to the business request corresponding to the second link data, an identification result of the business request corresponding to the second link data is identified, and the data volume of the second business request data is greater than the data volume of the first business request.

[0084] For example, the second service request data includes at least one of the following: a request timestamp of the target service request, a requested URL, requested parameters, a response status code, a response time, a log file, user feedback information, etc. The second anomaly detection algorithm may be the same as or different from the first anomaly detection algorithm, for example, an anomaly detection algorithm such as an isolation forest or a support vector machine (SVM).

[0085] For example, it can be determined based on the sliding window and the number of transactions per second (TPS) whether the number of collection times of the service request corresponding to the second link data within the preset time is less than the second preset number, but is not limited thereto. TPS is the ratio of the total number of transactions to the measurement time, wherein the measurement time is consistent with the time corresponding to the sliding window and the preset time, and the total number of transactions corresponds to the number of collection times. The server can count the TPS of the service request through the sliding window, thereby determining the number of collection times of the corresponding service request within the preset time.

[0086] In the above content, the server can use the request data with a larger data volume to perform a second match or verification on the unmatched second link data, avoid the omission of abnormal link data, and further improve the integrity and effectiveness of link data collection. In addition, the unmatched link data with a small number of collections within a preset time can be screened out and stored, which can ensure that a certain amount of link data is collected for each interface or service request, and ensure the effectiveness and integrity of the stored link data.

[0087] In one embodiment, Figure 3 As shown, the server corresponding to the client, i.e., the server, can first send the sampling configuration information to the client, specifically to the Agent of the client; after that, the client can collect the link data, and then, the collected link data can be matched according to the sampling configuration information, and the link data that is successfully matched carries the sampling tag, and the link data that is not successfully matched does not carry the sampling tag. For example, in combination with the above content, the preset interface information can be used to match the key features of the link data first. If the match is not successful, it is determined whether the business request corresponding to the link data is the first request. If it is not the first request, the link data can be matched with abnormal requests, that is, it is determined whether the link data is abnormal link data. If it is not abnormal link data, single service self-adjustment sampling can be performed, that is, it can be judged according to the load information of the server and the sampling setting information of the corresponding business service. After that, the client can send the successfully matched link data carrying tags and the unsuccessfully matched link data without tags to the server. The server can make a convergence judgment on it. First, the link data carrying the label can be persistently stored. Then, it can be judged whether there is any abnormality in the link data without the label. If there is no abnormality, it can be matched with a high frequency bound, that is, it can be judged whether the number of times the link data is collected within the preset time is less than the threshold. If it is greater than the threshold, it can be determined that the link data is out-of-bounds data, that is, high-frequency data, and it can be discarded to achieve bounded frequency reduction capability for high-frequency interfaces and avoid data duplication.

[0088] Through the technical solution of the present application, the sampling strategy can be dynamically adjusted according to the real-time monitored system load and data characteristics (for example, sampling configuration information, preset interface information, etc.), and a dynamic self-adjusting sampling mechanism can be realized, wherein the abnormal links and core key business links can be fully collected and stored, and other link data can be dynamically proportionally stored, and it is also considered that at least one sample, that is, link data, is captured for each interface within the preset time, and the bounded frequency reduction capability of the high-frequency interface can be realized (for example, a small number of bounded sampling and storage of high-frequency capability interfaces is performed to avoid the increase in network and storage costs of link data that need to be stored due to higher business concurrency), reducing the complexity of the sampling configuration and achieving an effective balance between link data integrity and storage costs. Therefore, not only can the loss of key link data related to analyzing faults and business requests and the problem of insufficient representativeness of the sampled link data caused by collecting link data according to a fixed sampling rate be avoided, but also, especially in the face of large business traffic links, the storage cost and computational complexity of the link data can be saved, while ensuring the quality of monitoring link data, while maximizing the reduction of unnecessary data storage and transmission costs.

[0089] It should be noted that all the above technical solutions can be combined in any way to form optional embodiments of the present application, which will not be described one by one here.

[0090] Figure 4 A schematic diagram of a link data processing device 400 provided in an embodiment of the present application is shown as follows: Figure 4 As shown, the device 400 includes: a first acquisition module 401 , a first matching module 402 , a result sending module 403 , a first receiving module 404 , a first display module 405 , a first acquisition module 406 , and an information sending module 407 .

[0091] In one embodiment, the first collection module 401 is used to collect link data under multiple business requests; the first matching module 402 is used to match multiple link data according to at least one of the following: preset sampling configuration information, the number of collection times of multiple business requests, and the abnormal detection results of multiple business requests to obtain matching results; the result sending module 403 is used to send the matching results to the server, so that the server stores or discards the multiple link data according to the matching results.

[0092] Exemplarily, the first matching module 402 is specifically used to: match multiple link data according to at least one of the following: sampling configuration information, the number of collection times of multiple business requests, and the abnormal detection results of multiple business requests, to determine the first link data that successfully matches among the multiple link data; set a sampling label for the first link data; and determine the matching result according to the first link data with the sampling label and the second link data other than the first link data among the multiple link data.

[0093] Exemplarily, the first matching module 402 is specifically configured to: set a sampling tag for the first link data; and determine a matching result according to the first link data with the sampling tag set and the second link data.

[0094] Exemplarily, the sampling configuration information includes at least one of the following: preset interface information, sampling setting information of multiple service requests, and server load information; the first matching module 402 is specifically used to:

[0095] For any target link data among the multiple link data, the target interface information in the target link data is matched with the preset interface information, and in response to the target interface information matching the preset interface information, the target link data is determined to be the first link data; or, it is determined whether the number of collection times of the target service request corresponding to the target link data within the preset time is less than the first preset number, and in response to the number of collection times within the preset time being less than the first preset number, the target link data is determined to be the first link data; or, in response to the abnormality detection result of the target service request being a request abnormality, the target link data is determined to be the first link data; or, based on the sampling setting information of the target service request and the load information of the server, the remaining number of samples of the target service request is determined, and in response to the remaining number of samples being greater than 0, the target link data is determined to be the first link data.

[0096] Exemplarily, the first matching module 402 is specifically configured to: in response to a match between all characters, part of characters, or specific attribute parameters corresponding to the target interface information and the preset interface information, determine that the target link data is the first link data.

[0097] Exemplarily, the anomaly detection result of the target business request includes at least one of the following: the response duration of the target business request, the response status code of the target business request, and an identification result of the target business request based on the first anomaly detection algorithm and the first business request data corresponding to the target business request; correspondingly, the anomaly detection result of the target business request is a request anomaly, including at least one of the following: the response duration of the target business request is greater than the duration threshold, the response status code of the target business request is an abnormal status code, and the identification result is an abnormal identification result.

[0098] Exemplarily, the sampling setting information includes: the maximum sampling rate and sampling weight of the corresponding business request; the first matching module 402 is specifically used to: determine the target sampling rate based on the server load information and the sampling weight of the target business request, the target sampling rate is less than or equal to the maximum sampling rate of the target business request; determine the existing number of samples of the determined first link data corresponding to the target business request in multiple link data; determine the remaining number of samples based on the existing number of samples and the target sampling rate.

[0099] Exemplarily, the first matching module 402 is specifically used to: determine the maximum number of samples corresponding to the target service request according to the target sampling rate; and determine the difference between the maximum number of samples and the existing number of samples as the remaining number of samples.

[0100] Exemplarily, the first matching module 402 is specifically used to: match multiple link data in accordance with a preset matching order, based on the sampling setting information and the server load information, and at least one of the following: preset interface information, the number of collection times of multiple business requests, and the abnormal detection results of multiple business requests, to determine the first link data; wherein the matching order corresponding to the sampling setting information and the server load information is at the end of the preset matching order.

[0101] Exemplarily, the first receiving module 404 is configured to receive sampling configuration information sent by the server.

[0102] Exemplarily, the first display module 405 is used to display the sampling configuration page; the first acquisition module 406 is used to obtain the preset interface information selected or input by the user based on the sampling configuration page; the information sending module 407 is used to send the preset interface information to the server so that the server determines the sampling configuration information according to the preset interface information.

[0103] Figure 5 A schematic diagram of a link data processing device 500 provided in an embodiment of the present application is shown as follows: Figure 5 As shown, the device 500 includes: a result receiving module 501, a result processing module 502, a result processing module 503, a first determining module 504, and a first sending module 505.

[0104] In one embodiment, the result receiving module 501 is used to receive the matching results sent by the client; the result processing module 502 is used to store or discard the link data under multiple business requests according to the matching results; wherein the matching results are obtained by matching the link data under multiple business requests according to at least one of the following: preset sampling configuration information, the number of collection times of multiple business requests, and the anomaly detection results of multiple business requests.

[0105] Exemplarily, the matching result includes: first link data with a sampling label set in multiple link data and second link data other than the first link data in multiple link data; the result processing module 502 is specifically used to: store the first link data; store or discard the second link data according to the abnormality detection result of the second link data and / or the number of collection times of the business request corresponding to the second link data.

[0106] Exemplarily, the result processing module 502 is specifically used to: in response to the abnormal detection result of the second link data being a request abnormality, store the second link data; in response to the abnormal detection result of the second link data being a request normal, determine whether the number of collection times within the preset time for the business request corresponding to the second link data is less than the second preset number; if the number of collection times within the preset time is less than the second preset number, store the second link data; if the number of collection times within the preset time is greater than or equal to the second preset number, discard the second link data; wherein the abnormal detection result of the second link data is: based on the second abnormal detection algorithm and the second business request data corresponding to the business request corresponding to the second link data, an identification result of identifying the business request corresponding to the second link data, and the data volume of the second business request data is greater than the data volume of the first business request.

[0107] Exemplarily, the result processing module 503 is used to: receive the preset interface information sent by the client; the first determination module 504 is used to: determine the sampling configuration information according to the preset interface information; the first sending module 505 is used to: send the sampling configuration information to the client, so that the client determines the matching result according to the sampling configuration information.

[0108] Exemplarily, the sampling configuration information includes at least one of the following: preset interface information, sampling setting information of multiple service requests, and server load information; the sampling setting information includes: a maximum sampling rate and a sampling weight corresponding to the service request.

[0109] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here.

[0110] The above describes the device of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0111] Figure 6 A schematic diagram of an electronic device 600 provided in an embodiment of the present application.

[0112] like Figure 6 As shown, the electronic device 600 may include:

[0113] The memory 610 and the processor 620, the memory 610 is used to store the computer program and transmit the program code to the processor 620. In other words, the processor 620 can call and run the computer program from the memory 610 to implement the method in the embodiment of the present application.

[0114] For example, the processor 620 may be configured to execute the above method embodiments according to instructions in the computer program.

[0115] In some embodiments of the present application, the processor 620 may include but is not limited to:

[0116] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0117] In some embodiments of the present application, the memory 610 includes but is not limited to:

[0118] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (Direct Rambus RAM, DR RAM).

[0119] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 610 and executed by the processor 620 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0120] like Figure 6 As shown, the electronic device may also include:

[0121] The transceiver 630 may be connected to the processor 620 or the memory 610 .

[0122] The processor 620 may control the transceiver 630 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 630 may include a transmitter and a receiver. The transceiver 630 may further include an antenna, and the number of antennas may be one or more.

[0123] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0124] The present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0125] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instruction is loaded and executed on a computer, the computer can be made to perform the corresponding flow in each method in the embodiment of the present application in whole or in part, and generate the functions that can be realized by each method in the embodiment of the present application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instruction can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instruction can be transmitted from a website site, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrations. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0126] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0127] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the system, device or module can be electrical, mechanical or other forms.

[0128] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.< / prefix>

Claims

1. A link data processing method, characterized in that: include: Collect link data under multiple business requests; According to at least one of the following: preset sampling configuration information, the number of collection times of the multiple service requests, and the abnormality detection results of the multiple service requests, the multiple link data are matched to obtain a matching result; The matching result is sent to a server, so that the server stores or discards the plurality of link data according to the matching result.

2. The method according to claim 1, characterized in that The matching result is obtained by matching the plurality of link data according to at least one of the following: preset sampling configuration information, the number of collection times of the plurality of service requests, and the abnormality detection results of the plurality of service requests, including: According to at least one of the following: the sampling configuration information, the number of collection times of the multiple service requests, and the abnormal detection results of the multiple service requests, the multiple link data are matched to determine the first link data that matches successfully among the multiple link data; The matching result is determined according to the first link data and second link data other than the first link data among the plurality of link data.

3. The method according to claim 2, characterized in that The determining the matching result according to the first link data and second link data other than the first link data among the plurality of link data comprises: Setting a sampling tag for the first link data; The matching result is determined according to the first link data and the second link data for which the sampling tag is set.

4. The method according to claim 2, characterized in that: The sampling configuration information includes at least one of the following: preset interface information, sampling setting information of the multiple service requests, and server load information; The matching of the plurality of link data according to at least one of the following: the sampling configuration information, the number of collection times of the plurality of service requests, and the abnormality detection results of the plurality of service requests, and determining the first link data that matches successfully among the plurality of link data, comprises: For any target link data among the plurality of link data, matching the target interface information in the target link data with the preset interface information, and in response to the target interface information matching the preset interface information, determining that the target link data is the first link data; or, determining whether the number of target service requests corresponding to the target link data collected within a preset time is less than a first preset number, and in response to the number of collections within the preset time being less than the first preset number, determining that the target link data is the first link data; or In response to the abnormality detection result of the target service request being a request abnormality, determining that the target link data is the first link data; or, The remaining number of samples of the target service request is determined according to the sampling setting information of the target service request and the load information of the server, and in response to the remaining number of samples being greater than 0, the target link data is determined to be the first link data.

5. A link data processing method, characterized in that: include: Receive the matching results sent by the client; According to the matching result, storing or discarding the link data under the multiple service requests; The matching result is obtained by matching the link data under the multiple service requests according to at least one of the following: preset sampling configuration information, the number of collection times of the multiple service requests, and the abnormality detection results of the multiple service requests.

6. A link data processing device, characterized in that: include: A first collection module, used to collect link data under multiple service requests; A first matching module, configured to match the plurality of link data to obtain a matching result according to at least one of the following: preset sampling configuration information, the number of collection times of the plurality of service requests, and the abnormality detection results of the plurality of service requests; The result sending module is used to send the matching result to the server, so that the server stores or discards the plurality of link data according to the matching result.

7. A link data processing device, characterized in that: include: A result receiving module, used to receive the matching results sent by the client; A result processing module, used to store or discard link data under multiple service requests according to the matching results; The matching result is obtained by matching the link data under the multiple service requests according to at least one of the following: preset sampling configuration information, the number of collection times of the multiple service requests, and the abnormality detection results of the multiple service requests.

8. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 5 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

10. A computer program product comprising instructions, characterized in that When the computer program product runs on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 5.

Images