An optimization method for link data storage and a terminal

Abstract

This invention discloses an optimized method and terminal for link data storage. Based on a service request, the method obtains the link nodes traversed by the service request, generates a request link, and determines abnormal links based on whether any abnormal nodes exist in each link node. Abnormal nodes within the abnormal links are then obtained, and abnormal branch links are derived by recursively obtaining their parent nodes. The node information of the abnormal branch links is stored in a list. This invention, by storing the information of abnormal branch link nodes in abnormal links in a list format, filters out normal branch links within abnormal links, effectively reducing the storage of useless information. Furthermore, when anomaly analysis is required, it can obtain related information more quickly and accurately, improving system efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of link data storage, in particular to a link data storage optimization method and terminal. BACKGROUND

[0002] In the current Internet system, in order to support a large number of high-concurrency functions, the server system usually adopts a distributed architecture, that is, a micro-service architecture, which supports high concurrency through dynamically expanding service instances. However, due to the micro-service architecture, some additional overheads are usually brought. For example, problem queries and stability between services. In order to quickly locate these problems, some supporting ecosystems are usually used in the micro-service architecture, such as a distributed log system that can pull data on each instance node to the distributed log system for unified query, and a distributed link tracking system that can record data on each link node. However, with the rapid increase in user request volume, each request may generate a large number of logs and link records. If all requests record logs and link information, the disk and various resources of the entire distributed log or link system will be occupied. Therefore, how to effectively reduce the storage of useless information and improve system efficiency is a technical problem to be solved at present. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a link data storage optimization method and terminal, which reduces the storage of useless information and improves system efficiency.

[0004] To solve the above technical problems, the technical scheme adopted by the present application is:

[0005] A link data storage optimization method, comprising the steps of:

[0006] S1, according to a business request, obtaining link nodes passed by the business request, generating a request link, and determining an abnormal link according to whether each link node has an abnormal node;

[0007] S2, obtaining an abnormal node in the abnormal link, obtaining an abnormal branch link through a recursive parent node obtaining mode, and storing node information of the abnormal branch link in a list mode.

[0008] To solve the above technical problems, another technical scheme adopted by the present application is:

[0009] A link data storage optimization terminal, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0010] S1, according to the service request, the link node of the service request is obtained, the request link is generated, and the abnormal link is determined according to whether each link node has an abnormal node or not;

[0011] S2, the abnormal node in the abnormal link is obtained, the abnormal branch link is obtained by recursively obtaining the parent node, and the node information of the abnormal branch link is stored in the form of a list.

[0012] The beneficial effects of the application are that the link data storage optimization method and terminal of the application store the information of the abnormal branch link nodes in the abnormal link in the form of a list, filter out the normal branch links in the abnormal link, effectively reduce the storage of useless information, and when abnormal analysis is needed, the associated information can be obtained faster and more accurately, improving the system efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 The flowchart of the link data storage optimization method of the embodiment of the application is shown in the figure.

[0014] Figure 2 The structure diagram of the link data storage optimization terminal of the embodiment of the application is shown in the figure.

[0015] Label explanation:

[0016] 1, a link data storage optimization terminal; 2, a processor; 3, a memory. DETAILED DESCRIPTION

[0017] To explain the technical content, the achieved purposes and effects of the application in detail, the following will be explained in combination with the embodiments and the drawings.

[0018] Please refer to Figure 1 A link data storage optimization method, comprising the following steps:

[0019] S1, according to the service request, the link node of the service request is obtained, the request link is generated, and the abnormal link is determined according to whether each link node has an abnormal node or not;

[0020] S2, the abnormal node in the abnormal link is obtained, the abnormal branch link is obtained by recursively obtaining the parent node, and the node information of the abnormal branch link is stored in the form of a list.

[0021] From the above description, the beneficial effects of the present application are that the link data storage optimization method and terminal of the present application stores the information of the abnormal branch link nodes in the abnormal link in the form of a list, filters out the normal branch links in the abnormal link, effectively reduces the storage of useless information, and when abnormal analysis is needed, the associated information can be obtained more quickly and accurately, improving system efficiency.

[0022] Further, step S2 comprises steps of:

[0023] S21, establishing a list, recording each link node in the abnormal link, including node ID, parent node ID, branch link information and identification information whether abnormal or not;

[0024] S22, obtaining all the abnormal nodes therein, and according to the branch link information, obtaining the set of each abnormal node and each link node on the abnormal branch link corresponding thereto, to obtain an abnormal set;

[0025] S23, for each list of the abnormal link, filtering other branch data outside the abnormal branch link according to the union set of the abnormal set.

[0026] From the above description, each link node in the abnormal link is recorded by the list, and each link node on the abnormal branch link is determined through the branch link information in the abnormal node information, so that the nodes in the list which do not belong to the abnormal branch link are filtered out.

[0027] Further, step S1 comprises steps of:

[0028] S11, according to the business request, obtaining the link nodes passed by the business request, and generating a request link;

[0029] S12, judging whether there is an abnormal node in each link node in the request link, if there is an abnormal node, the request link is an abnormal link, otherwise it is a normal link.

[0030] From the above description, whether there is an abnormal node in the request link is used to judge whether the request link is abnormal.

[0031] Further, it further comprises steps of:

[0032] S3, storing the data of the normal link in a sampling storage manner according to a preset sampling ratio.

[0033] From the above description, the normal request link is stored by sampling, which reduces the data storage amount while meeting the needs of part of the business.

[0034] Please refer toFigure 2 An optimization terminal of link data storage, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:

[0035] S1, according to a service request, obtaining link nodes passed by the service request, generating a request link, and determining an abnormal link according to whether each link node has an abnormal node;

[0036] S2, obtaining abnormal nodes in the abnormal link, obtaining abnormal branch links by recursively obtaining parent nodes, and storing node information of the abnormal branch links in a list.

[0037] From the above description, the beneficial effects of the present application are that the optimization method and terminal of link data storage of the present application store the information of abnormal branch link nodes in the abnormal link in a list, filter out normal branch links in the abnormal link, effectively reduce the storage of useless information, and when abnormal analysis is needed, can obtain associated information more quickly and accurately, and improve system efficiency.

[0038] Further, step S2 comprises the following steps:

[0039] S21, establishing a list, recording each link node in the abnormal link, including node ID, parent node ID, branch link information, and identification information of whether abnormal;

[0040] S22, obtaining all abnormal nodes therein, and obtaining a set of each link node on the abnormal branch link corresponding to each abnormal node according to the branch link information, to obtain an abnormal set;

[0041] S23, for each list of the abnormal link, filtering other branch data outside the abnormal branch link according to the union set of the abnormal set.

[0042] From the above description, each link node in the abnormal link is recorded in a list, and each link node on the abnormal branch link is determined according to the branch link information in the abnormal node information, so that other nodes in the list that do not belong to the abnormal branch link are filtered out.

[0043] Further, step S1 comprises the following steps:

[0044] S11, according to a service request, obtaining link nodes passed by the service request, generating a request link;

[0045] S12, judge whether there is an abnormal node in each link node in the request link, if there is an abnormal node, the request link is an abnormal link, otherwise it is a normal link.

[0046] From the above description, it can be seen that whether there is an abnormal node in the request link is judged to determine whether the request link is abnormal.

[0047] Further, it further comprises the steps of:

[0048] S3, according to the preset sampling ratio, the data of the normal link is stored in the form of sampling storage.

[0049] From the above description, it can be seen that the normal request link is stored by sampling, which reduces the data storage amount while meeting the needs of part of the business.

[0050] The optimization method and terminal of link data storage are suitable for the scene of recording and storing distributed link tracking data under distributed architecture.

[0051] Please refer to Figure 1 The embodiment one of the application is:

[0052] An optimization method of link data storage, comprising the steps of:

[0053] S1, according to the business request, the link node passed by the business request is acquired, the request link is generated, and the abnormal link is determined according to whether each link node has an abnormal node;

[0054] Step S1 comprises the steps of:

[0055] S11, according to the business request, the link node passed by the business request is acquired, the request link is generated;

[0056] S12, judge whether there is an abnormal node in each link node in the request link, if there is an abnormal node, the request link is an abnormal link, otherwise it is a normal link.

[0057] In this embodiment, each request will access the distributed link system, when the request has a traceId field, it will directly skip the processing, when there is no traceId, the distributed link system will automatically generate a traceId and set it to the request, then all subsequent requests can be associated as the same group of requests according to the traceId. The traceId represents the ID of the request link.

[0058] In this embodiment, the link type will be distinguished. If all nodes in the link are normal, it indicates a normal link. If there is only one abnormal node in the link, it indicates an abnormal link, and the fault node in the abnormal link is indicated as an abnormal node.

[0059] S13, according to the preset sampling ratio, the data of the normal link is stored in a sampling storage manner.

[0060] In this embodiment, for the normal link, the sampling storage manner can be used, and the sampling storage ratio can be set by itself, and the abnormal link needs to be stored in its entirety. When sampling storage is performed, the message needs to be sent synchronously to inform the distributed log system, and only the same log information needs to be stored.

[0061] S2, the abnormal node in the abnormal link is obtained, the abnormal branch link is obtained by recursively obtaining the parent node, and the node information of the abnormal branch link is stored in a list manner.

[0062] Step S2 includes the following steps:

[0063] S21, a list is established, and each link node in the abnormal link is recorded, including node ID, parent node ID, branch link information and identification information whether abnormal.

[0064] In the old way, the whole link information is stored in a hierarchical recursive manner. That is, a link record is recorded, which includes a subList sublist object record, and each object in the subList is still a link, and the specific data format is as follows:

[0065] TraceInfo: other attribute information of this record, subList:

[0066] Traceinfo1: other attribute information of this record: subList:

[0067] Traceinfo11: other attribute information of this record: subList: [],

[0068] Traceinfo12: other attribute information of this record: subList: [],

[0069] ],

[0070] Traceinfo2: other attribute information of this record: subList: [],

[0071] Traceinfo3: other attribute information of this record: subList: [], ]

[0073] Based on the above format, when querying a certain piece of data, it will be necessary to recursively traverse the data one by one, and the query will take time.

[0074] Each instance in the microservice architecture (link node) is marked with a unique id. When each request arrives in the distributed link system, the system sets a field for each request with its own instance id and parent instance id, and finally sets the link information storage format to a list format, as follows: [

[0076] traceInfo1:id=a,parentId=0,path= / 0,other attribute information of this record, this node is normal;

[0077] traceInfo2:id=b,parentId=a,path= / 0 / a,other attribute information of this record, this node is normal;

[0078] traceInfo3:id=c,parentId=a,path= / 0 / a,other attribute information of this record, this node is abnormal;

[0079] traceInfo4:id=d,parentId=c,path= / 0 / a / c,other attribute information of this record, this node is abnormal;

[0080] traceInfo5:id=e,parentId=d,path= / 0 / a / c / d,other attribute information of this record, this node is normal;

[0081] traceInfo6:id=f,parentId=d,path= / 0 / a / c / d,other attribute information of this record, this node is normal; ]

[0083] As can be seen from the above format, c and d nodes are abnormal nodes, so the link is an abnormal link, and the order of the records in the list is the order of the requests arriving at each node.

[0084] S22, obtain all abnormal nodes therein, and obtain the set of each abnormal node and each link node on the abnormal branch link corresponding thereto according to the branch link information, to obtain an abnormal set.

[0085] In this embodiment, the first abnormal node is sequentially acquired, and branch link information, i.e., a path field path= / 0 / a, in the abnormal node is parsed, at this time, only all upstream node information of the self node will be recorded, and other branch data will be filtered, so that a set of all upstream nodes of each abnormal node is acquired.

[0086] S23, for each list of abnormal link, filtering other branch data outside the abnormal branch link according to the union set of the abnormal set.

[0087] In this embodiment, the union set determined in the above step is calculated, and finally all link information of the abnormal node can be simply acquired. In this way, the storage of useless branch information can be effectively reduced.

[0088] In addition, a synchronization message needs to be sent to the distributed log, and useless log record information is synchronously removed.

[0089] As can be known from the above description, when data query is performed, in the old way, the data structure is a recursive data structure, which needs to be parsed layer by layer, and is relatively time-consuming. The data structure in this embodiment is a flat list type data structure, which does not need to be parsed and can be directly traversed. At the same time, in the data structure shown in this embodiment, the path field and the like can be used to obviously know all parent node information of the self node.

[0090] Please refer to Figure 2 Embodiment two of the present application is:

[0091] An optimized terminal 1 for link data storage includes a processor 2, a memory 3, and a computer program stored in the memory 3 and executable on the processor 2, and the processor 2 implements the steps in the optimized method for link data storage in the above embodiment one when the computer program is executed.

[0092] In summary, the optimized method and terminal for link data storage provided by the present application stores information of abnormal branch link nodes in abnormal links through a list, filters normal branch links in abnormal links, effectively reduces useless information storage, and when abnormal analysis is needed, can quickly and accurately acquire associated information, and improves system efficiency.

[0093] The above is only an embodiment of the present application, and does not limit the patent range of the present application, and any equivalent transformation or direct or indirect application in related technical fields by using the content of the specification and drawings of the present application is also included in the patent protection range of the present application.

Claims

1. A method for optimizing link data storage, characterized in that, The method comprises the steps of: S1, acquiring link nodes passed by a service request according to the service request, generating a request link, and determining an abnormal link according to whether each link node exists; S2, acquiring abnormal nodes in the abnormal link, obtaining abnormal branch links by recursively acquiring parent nodes, and storing node information of the abnormal branch links in a list; Step S2 comprises the steps of: S21, establishing a list, recording each link node in the abnormal link, including node ID, parent node ID, branch link information, and identification information of whether abnormal; S22, acquiring all abnormal nodes, and acquiring a set of each abnormal node and each link node on the abnormal branch link corresponding to the abnormal node according to the branch link information, to obtain an abnormal set; S23, for each list of the abnormal link, filtering other branch data outside the abnormal branch link according to the union set of the abnormal set; Step S1 comprises the steps of: S11, acquiring link nodes passed by a service request according to the service request, generating a request link; S12, judging whether each link node in the request link exists, if an abnormal node exists, the request link is an abnormal link, otherwise, it is a normal link.

2. The method of claim 1, wherein, Further comprising the step of: S3, storing data of the normal link in a sampling storage manner according to a preset sampling ratio.

3. An optimized terminal for link data storage, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, characterized in that, The processor executes the computer program to implement the following steps: S1, acquiring link nodes passed by a service request according to the service request, generating a request link, and determining an abnormal link according to whether each link node exists; S2, acquiring abnormal nodes in the abnormal link, obtaining abnormal branch links by recursively acquiring parent nodes, and storing node information of the abnormal branch links in a list; Step S2 comprises the steps of: S21, establishing a list, recording each link node in the abnormal link, including node ID, parent node ID, branch link information, and identification information of whether abnormal; S22, acquiring all abnormal nodes, and acquiring a set of each abnormal node and each link node on the abnormal branch link corresponding to the abnormal node according to the branch link information, to obtain an abnormal set; S23, for each list of the abnormal link, filtering other branch data outside the abnormal branch link according to the union set of the abnormal set; Step S1 comprises the steps of: S11, acquiring link nodes passed by a service request according to the service request, generating a request link; S12, judging whether each link node in the request link exists, if an abnormal node exists, the request link is an abnormal link, otherwise, it is a normal link.

4. An optimized terminal for link data storage according to claim 3, characterized in that, Further comprising the step of: S3, storing data of the normal link in a sampling storage manner according to a preset sampling ratio.

Images