A link relationship storage method, device, equipment and medium

Abstract

This application discloses a link relationship storage method, apparatus, device, and medium, relating to the field of computer technology. The method includes: acquiring log data through a computing server and sending it to a memory data queue; retrieving request log data from the memory data queue through a computing coroutine; determining the index number of the storage shard based on the corresponding request ID; locating a single-shard dictionary based on the index number and locking it if a request ID exists in the single-shard dictionary; retrieving request data from the single-shard dictionary based on the request ID; calculating the current level number and checking if a SpanId dataset exists at the current level; if so, assigning a value to the SpanId slice dictionary at the current level using request entity data; updating the link tree shard cache using request entity data; and polling the request entity data in the link tree shard cache to obtain link relationship data and storing it in a database. This method can clarify the link relationships between application log data and perform efficient storage.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for storing link relationships. Background Technology

[0002] As the call relationships between customer application services become increasingly complex, each request passes through multiple application business systems, generating access to various business processes, caches, or databases. However, this scattered application data offers limited assistance for troubleshooting or process optimization. Currently, log data from each application is collected via code-based instrumentation, and each log entry is written to the log system in real time for querying, location, and display. However, for log data storage via code-based instrumentation in a production environment with billions of log records, if the sampling rate is too high, it will affect the throughput of the application system, thus impacting business operations; if the sampling rate is too low, it will affect the integrity of the storage and statistics of related relationships.

[0003] In summary, how to clarify the link relationships between multiple application services that generate massive amounts of log data through mutual calls, and how to efficiently store these link relationships, are problems that need to be solved. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for storing link relationships, which can clarify the link relationships between multiple application services that generate massive amounts of log data, and efficiently store these link relationships. The specific solution is as follows:

[0005] Firstly, this application discloses a link relationship storage method, including:

[0006] The server obtains log data according to preset rules and sends the log data to the memory data queue. Then, several computing coroutines obtain request log data from the memory data queue and determine the index number of the storage shard according to the request ID corresponding to the request log data.

[0007] Based on the index number, locate the single slice dictionary in the storage shard, and lock the single slice dictionary when the request ID exists in the single slice dictionary;

[0008] The request data in the single-slice dictionary is obtained based on the request ID, and the current level number of the request data is calculated. Then, the existence of the SpanId dataset in the current level is detected based on the current level number.

[0009] If the SpanId dataset exists, the SpanId slice dictionary of the current level is assigned a value using the created request entity data, and the link tree shard cache is updated using the request entity data based on the calculated fingerprint ID.

[0010] The request entity data in the link tree shard cache is calculated in a round-robin manner according to preset rules to obtain link relationship data, and the link relationship data is stored in the database.

[0011] Optionally, the step of obtaining log data through the computing server according to preset rules includes:

[0012] The default log consumer reports log data to the computing server through the first coroutine based on the GRPC protocol;

[0013] Log data is retrieved from the preset log consumer via a second coroutine based on the HTTP protocol on the computing server.

[0014] Optionally, determining the index number of the storage shard based on the request ID corresponding to the request log data includes:

[0015] Determine the request ID corresponding to the request log data, and calculate the hash value corresponding to the request ID using a preset hash algorithm;

[0016] The index number of the storage fragment is determined based on the hash value and the number of storage fragments.

[0017] Optionally, after retrieving request log data from the memory data queue through several coroutines, the method further includes:

[0018] Obtain the corresponding application ID and class ID based on the method ID of the request log data;

[0019] Accordingly, before updating the link tree shard cache using the request entity data based on the calculated fingerprint ID, the process further includes:

[0020] The fingerprint ID of the link is calculated based on the method ID, application ID, and class ID.

[0021] The step of polling and calculating the request entity data in the link tree shard cache according to preset rules includes:

[0022] The link tree shard cache is polled at preset time intervals to obtain link tree shard cache objects, and the pending data tasks of each coroutine are allocated according to the number of link shards.

[0023] Each coroutine obtains a copy of the corresponding single data piece, executes the calculation process of the requested entity data in the top-level node of the link tree, and executes the calculation process of the requested entity data in the sub-level nodes of the link tree after the top-level node has successfully calculated the data, until all link tree nodes have been traversed.

[0024] The process of calculating entity data requested in the top-level node of the execution link tree also includes:

[0025] If a corresponding top-level node number exists in the detection layer number data group, and if it exists, then check if there is a break in the link tree or if the break number exceeds a preset number.

[0026] If there are no faults or the fault number does not exceed the preset number, then obtain the SpanID corresponding to the top-level node of the link tree and create the entity node of the top-level link tree.

[0027] The process of calculating the requested entity data in the sub-nodes of the execution link tree also includes:

[0028] Check if the current sub-layer node has a corresponding SpanID. If it does, check if the current sub-layer node includes the SpanID feature information of the parent node.

[0029] If included, the corresponding entity list is obtained based on the SpanID feature information, and the entity list is traversed to obtain the node information of this layer.

[0030] Secondly, this application discloses a link relationship storage device, comprising:

[0031] The data acquisition module is used to acquire log data through the computing server according to preset rules, send the log data to the memory data queue, and then acquire request log data from the memory data queue through several coroutines, and determine the index number of the storage shard according to the request ID corresponding to the request log data;

[0032] The storage shard determination module is used to locate a single-shard dictionary based on the index number, and lock the single-shard dictionary when the request ID exists in the single-shard dictionary;

[0033] The link level detection module is used to obtain the request data in the single slice dictionary according to the request ID, calculate the current level number of the request data, and then detect whether the SpanId dataset exists in the current level based on the current level number.

[0034] The link cache update module is used to, if the SpanId dataset exists, assign values ​​to the SpanId slice dictionary of the current level using the created request entity data, and update the link tree shard cache based on the calculated fingerprint ID using the request entity data.

[0035] The link data storage module is used to poll and calculate the request entity data in the link tree shard cache according to preset rules to obtain link relationship data, and store the link relationship data in the database.

[0036] Thirdly, this application discloses an electronic device, including:

[0037] Memory, used to store computer programs;

[0038] A processor is configured to execute the computer program to implement the steps of the aforementioned disclosed link relationship storage method.

[0039] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed link relationship storage method.

[0040] As can be seen, this application obtains log data through a computing server according to preset rules and sends the log data to a memory data queue. Then, it retrieves request log data from the memory data queue through several computing coroutines and determines the index number of the storage shard based on the request ID corresponding to the request log data. Based on the index number, it locates the single-slice dictionary in the storage shard and locks the single-slice dictionary when the request ID exists in the single-slice dictionary. It obtains the request data in the single-slice dictionary according to the request ID and calculates the current level number of the request data. Then, it checks whether the current level has a SpanId dataset based on the current level number. If the SpanId dataset exists, it assigns a value to the SpanId slice dictionary of the current level using the created request entity data and updates the link tree shard cache using the request entity data based on the calculated fingerprint ID. It polls and calculates the request entity data in the link tree shard cache according to preset rules to obtain link relationship data and stores the link relationship data in the database. Therefore, this application first acquires log data and locks the single-slice dictionary. Then, it assigns values ​​to the current-level SpanId slice dictionary using the created request entity data. Next, it updates the link tree shard cache based on the link fingerprint ID using the request entity data. Finally, it calculates the data in the link tree shard cache to store the link relationship data in the database. This technical solution clarifies and tracks the link relationships between multiple application services that generate massive amounts of log data, and efficiently stores these link relationships in the database, reducing the overhead of service computing and storage resources. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0042] Figure 1 This is a flowchart of a link relationship storage method disclosed in this application;

[0043] Figure 2 This is a schematic diagram of a specific link relationship storage process disclosed in this application;

[0044] Figure 3 This is a schematic diagram of log data retrieval disclosed in this application;

[0045] Figure 4 This is a flowchart of a specific link relationship storage method disclosed in this application;

[0046] Figure 5 This is a schematic diagram of a specific link relationship storage process disclosed in this application;

[0047] Figure 6 This is a schematic diagram of a link relationship storage device structure disclosed in this application;

[0048] Figure 7 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0050] Currently, log data from various applications is collected via code-based tracking, and each log entry is written to a log system in real time for querying, locating, and displaying. However, for log data storage via code-based tracking, especially in production environments with billions of log entries, excessively high sampling rates can impact application system throughput and consequently affect business operations; conversely, excessively low sampling rates can compromise the integrity of related data relationships during storage and statistical analysis. Therefore, this application discloses a method, apparatus, device, and medium for storing data relationships, which can clarify the relationships between multiple application services that generate massive amounts of log data and efficiently store these relationships.

[0051] See Figure 1 and Figure 2 As shown in the figure, this application discloses a link relationship storage method, which includes:

[0052] Step S11: The computing server obtains log data according to preset rules and sends the log data to the memory data queue. Then, several computing coroutines obtain request log data from the memory data queue and determine the index number of the storage shard according to the request ID corresponding to the request log data.

[0053] In this embodiment, the above-mentioned acquisition of log data by the computing server according to preset rules may specifically include: a preset log consumer reporting log data to the computing server through a first coroutine based on the GPPC protocol; and the computing server pulling log data from the preset log consumer through a second coroutine based on the HTTP protocol. It can be understood that the computing server is first started, the number of cache shards is initialized, and configurations are performed; then two coroutines A and B are started. Coroutine A, i.e., the first coroutine mentioned above, uses the GPPC protocol to implement the passive real-time data reception service of the computing server, that is, the preset log consumer actively reports log data to the computing server through the first coroutine based on the GPPC protocol; coroutine B, i.e., the second coroutine mentioned above, uses the HTTP protocol to pull data from each log data consumer in real time. For details, please refer to [link to relevant documentation]. Figure 3 As shown. Coroutine A service is mainly provided to log consumer A service. It can communicate with the computing server, so it uses the high-efficiency gRPC long-connection communication protocol to collect data. Coroutine B service is mainly provided to log consumer B service. It communicates one-way with the computing server. Only the computing server can access log consumer B service. It uses the HTTP protocol to pull data in real time.

[0054] After obtaining the log data, it is sent to the memory data queue. This involves pulling data from the log consumer and real-time reported data, then uniformly sending them to the log memory data queue. Next, based on the configured number of coroutines N, N computation coroutines are started to compute the data received from the queue, obtain request log data in the queue in real time, and determine the storage shard index number based on the request ID corresponding to the request log data. In a specific implementation, determining the storage shard index number based on the request ID corresponding to the request log data can specifically include: determining the request ID corresponding to the request log data, calculating the hash value corresponding to the request ID using a preset hash algorithm, and determining the storage shard index number based on the hash value and the number of storage shards. That is, the request ID is a string sequence, a 32-bit integer hash value is calculated using CRC32, and then the remainder after dividing by the number of storage shards is taken as the storage shard index number.

[0055] Step S12: Locate the single slice dictionary in the storage shard based on the index number, and lock the single slice dictionary when the request ID exists in the single slice dictionary.

[0056] In this embodiment, the single slice dictionary in the storage shard is located according to the index number, and then the request ID is checked in the dictionary. If the request ID exists, the single slice dictionary is locked.

[0057] Step S13: Obtain the request data in the single-slice dictionary according to the request ID, calculate the current level number of the request data, and then detect whether the SpanId dataset exists in the current level based on the current level number.

[0058] In this embodiment, request data is retrieved from the single-slice dictionary using the request ID, and the SpanId level number of the data to be updated is calculated. SpanId represents the log level number, appended to the log level number of the called party. For example, the top level is a single number 1; after calling the lower level, the lower level's SpanId becomes 1.1, and so on, appending numbers with dots to identify levels. Then, the total number is calculated by dividing the SpanID using the level separator ".", yielding the current level number (Level) of the request log. Next, duplicate Level values ​​are removed from the corresponding level number array in the request tree; if a Level does not exist, it is incremented. Finally, the existence of the corresponding SpanId dataset for that level is checked in the SpanId data set of the request tree using the level number (Level).

[0059] Step S14: If the SpanId dataset exists, the SpanId slice dictionary of the current level is assigned a value using the created request entity data, and the link tree shard cache is updated using the request entity data based on the calculated fingerprint ID.

[0060] In this embodiment, if a SpanId dataset exists, the dictionary of the current level slice SpanId is assigned a value to request entity data; if a SpanId dataset does not exist, the SpanId dataset for this level is created, and the current SpanId is assigned to the dataset.

[0061] Furthermore, after obtaining the request log data, the process may also include: obtaining the corresponding application ID and class ID based on the method ID of the request log data; calculating the fingerprint ID of the link by comprehensively considering all method IDs, application IDs, and class IDs in the request tree; and then updating the request entity data in the link tree cache shard.

[0062] It should be noted that, in another specific embodiment, if the request ID does not exist in a single slice, an empty entity dictionary cache for the request tree is created for storage in subsequent steps. In addition, the level number of the request data, Level, is also calculated. Then, the entity data created by SpanId is assigned to the slice, and finally the request data is created to the slice entity in the link tree cache.

[0063] Step S15: Calculate the request entity data in the link tree shard cache according to preset rules to obtain link relationship data, and store the link relationship data in the database.

[0064] In this embodiment, the request entity data in the link tree shard cache is polled and calculated according to preset rules to obtain link relationship data, and the link relationship data is stored in the database.

[0065] As can be seen, this application obtains log data through a computing server according to preset rules and sends the log data to a memory data queue. Then, it retrieves request log data from the memory data queue through several computing coroutines and determines the index number of the storage shard based on the request ID corresponding to the request log data. Based on the index number, it locates the single-slice dictionary in the storage shard and locks the single-slice dictionary when the request ID exists in the single-slice dictionary. It obtains the request data in the single-slice dictionary according to the request ID and calculates the current level number of the request data. Then, it checks whether the current level has a SpanId dataset based on the current level number. If the SpanId dataset exists, it assigns a value to the SpanId slice dictionary of the current level using the created request entity data and updates the link tree shard cache using the request entity data based on the calculated fingerprint ID. It polls and calculates the request entity data in the link tree shard cache according to preset rules to obtain link relationship data and stores the link relationship data in the database. Therefore, this application first acquires log data and locks the single-slice dictionary. Then, it assigns values ​​to the current-level SpanId slice dictionary using the created request entity data. Next, it updates the link tree shard cache based on the link fingerprint ID using the request entity data. Finally, it calculates the data in the link tree shard cache to store the link relationship data in the database. This technical solution clarifies and tracks the link relationships between multiple application services that generate massive amounts of log data, and efficiently stores these link relationships in the database, reducing the overhead of service computing and storage resources.

[0066] In one specific embodiment, this application provides specific steps for obtaining link relationship data, see [link to specific implementation details]. Figure 4 and Figure 5 As shown, step S15 may specifically include:

[0067] Step S151: Poll the link tree shard cache at preset time intervals to obtain link tree shard cache objects, and allocate the data tasks to be processed for each coroutine according to the number of link shards.

[0068] In this embodiment, the number of services for analysis and computing services in the configuration file is first obtained, and then the corresponding computing service coroutine is started to perform polling tasks at preset time intervals to process cache shard computing; then the link tree shard cache object is obtained, and then the shard group data tasks to be processed by each coroutine are allocated according to the number of link shards.

[0069] Step S152: Obtain the corresponding single-piece data copy through each coroutine, and execute the calculation process of the requested entity data in the top-level node of the link tree. After the top-level node is successfully calculated, execute the calculation process of the requested entity data in the sub-level nodes of the link tree until all link tree nodes are traversed to obtain the link relationship data.

[0070] In this embodiment, the allocation calculation is performed based on the initiated and allocated link fragment cache data task group. Then, the respective allocated coroutine execution process is entered. First, the individual fragment data is traversed and retrieved, and a copy of the result of the individual fragment data is returned. The copy is to avoid data updates during the calculation and to avoid data inconsistency. Then, by traversing the copied individual fragment data, the calculation process of the requested entity data in the top-level node of the link tree is executed, that is, the calculation process of the TOP1 top-level link tree node is entered. In a specific implementation, this process includes: detecting whether there is a corresponding top-level node number in the level number data group; if so, detecting whether there is a gap in the link tree or whether the gap number exceeds a preset number; if there is no gap or the gap number does not exceed the preset number, obtaining the SpanID corresponding to the top-level node of the link tree, and creating the entity node of the top-level link tree. Specifically, first, it checks if the top-level node number 1 exists in the Level data group; if not, the incomplete request log data chain is discarded. If the existence of top-level node number 1 is detected, it further checks if there are any gaps in the level tree. If there are gaps, it checks if there are more than two; if there are more than two gaps, the request data chain is discarded. Then, it checks if there are multiple SpanId nodes in the Root top level; if there are multiple nodes, it is an incomplete request data chain, and the data needs to be discarded. Next, based on the SpanId of the top-level tree node calculated in the request chain tree in the previous step, the top-level tree is assigned values ​​to create entity nodes, and marked with whether there are gaps and the top-level SpanId information. After the top level is processed, database query and caching are performed. Based on the calculation results of the above steps, it is determined whether the TOP0 top level was calculated successfully; if the calculation fails, the request tree calculation chain is exited.

[0071] After the top-level link tree is successfully created, the top-level node ID synchronized in the database is assigned to TopNodeID, and the method ID of the top-level node is assigned to the scene ID to save the node information. After the top-level node is successfully calculated, the calculation process of the requested entity data in the sub-level nodes of the link tree is executed. The starting level is 2, and the requested data set is recursively traversed sequentially. During processing, the top-level node with level 1 is ignored. The SpanId is searched. There may be multiple SpanIds in the sub-level, and then they are traversed sequentially to enter the sub-node relationship calculation processing flow. This process specifically includes: checking whether the current sub-level node has a corresponding SpanID. If it does, it checks whether the current sub-level node includes the SpanID feature information of the parent node. If it does, it obtains the corresponding entity list based on the SpanID feature information and traverses the entity list to obtain the node information of this level. Specifically, first, it checks if there are multiple SpanIDs in the sub-level Level. If no SpanID exists in the level, the data calculation for that level is abandoned. If a sub-level SpanID exists, it calculates whether it contains the SpanId feature information of the parent node. If not, the processing for that level is abandoned. It then obtains an entity list based on the SpanIds contained under the parent node and traverses it. For entity sets that do not contain the SpanId feature, the data is discarded. It obtains the entity list, the current level's SpanId, the parent node's scene ID, and the parent node's ID to obtain the current level's tree node list. It then enters the calculation of the current level's nodes. By traversing the entity list of this level, performing database synchronization and caching, it returns the data structure node information of this level. Finally, it appends the current level's node information to the tree node created by the top-level node and recursively traverses the child node information under this level's nodes.

[0072] Step S153: Store the link relationship data in the database.

[0073] In this embodiment, the obtained link relationship data is stored in a database for analysis of the link relationships.

[0074] The technical solution of this application enables the temporary caching of data in memory based on link fingerprint data before data storage. Then, similar data is aggregated and calculated before being stored in the database at the minute level. This significantly reduces the overhead of service computing and storage resources. Previously, eight machines were needed to handle the multi-application link calculation relationship chaining of 10 billion log data entries; now, with the same configuration, only one computing server is required. Previously, the daily calculation and storage of linked log data required approximately 5GB of data; now, this has been reduced by more than tenfold to approximately 300-400MB. Furthermore, it improves the efficiency of link node fault diagnosis. Specifically, based on the results of front-end monitoring and through front-end calls, the number of records showing the amount of erroneous data at each node can be displayed, allowing for faster location of link node faults, reducing the occurrence of production quality incidents, and quickly identifying problems in link node response and processing, thus ensuring business stability.

[0075] See Figure 6 As shown in the figure, this application discloses a link relationship storage device, which includes:

[0076] The data acquisition module 11 is used to acquire log data through the computing server according to preset rules, send the log data to the memory data queue, and then acquire request log data from the memory data queue through several coroutines, and determine the index number of the storage shard according to the request ID corresponding to the request log data.

[0077] The storage shard determination module 12 is used to locate the single-shard dictionary based on the index number, and lock the single-shard dictionary when the request ID exists in the single-shard dictionary;

[0078] The link level detection module 13 is used to obtain the request data in the single slice dictionary according to the request ID, calculate the current level number of the request data, and then detect whether the SpanId dataset exists in the current level based on the current level number.

[0079] The link cache update module 14 is used to, if the SpanId dataset exists, assign values ​​to the SpanId slice dictionary of the current level using the created request entity data, and update the link tree shard cache using the request entity data based on the calculated fingerprint ID.

[0080] The link data storage module 15 is used to poll and calculate the request entity data in the link tree shard cache according to preset rules to obtain link relationship data, and store the link relationship data in the database.

[0081] As can be seen, this application obtains log data through a computing server according to preset rules and sends the log data to a memory data queue. Then, it retrieves request log data from the memory data queue through several computing coroutines and determines the index number of the storage shard based on the request ID corresponding to the request log data. Based on the index number, it locates the single-slice dictionary in the storage shard and locks the single-slice dictionary when the request ID exists in the single-slice dictionary. It obtains the request data in the single-slice dictionary according to the request ID and calculates the current level number of the request data. Then, it checks whether the current level has a SpanId dataset based on the current level number. If the SpanId dataset exists, it assigns a value to the SpanId slice dictionary of the current level using the created request entity data and updates the link tree shard cache using the request entity data based on the calculated fingerprint ID. It polls and calculates the request entity data in the link tree shard cache according to preset rules to obtain link relationship data and stores the link relationship data in the database. Therefore, this application first acquires log data and locks the single-slice dictionary. Then, it assigns values ​​to the current-level SpanId slice dictionary using the created request entity data. Next, it updates the link tree shard cache based on the link fingerprint ID using the request entity data. Finally, it calculates the data in the link tree shard cache to store the link relationship data in the database. This technical solution clarifies and tracks the link relationships between multiple application services that generate massive amounts of log data, and efficiently stores these link relationships in the database, reducing the overhead of service computing and storage resources.

[0082] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the link relationship storage method executed by the electronic device disclosed in any of the foregoing embodiments.

[0083] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0084] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0085] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0086] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system 221 can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the link relationship storage method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0087] Furthermore, embodiments of this application also disclose a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, it implements the method steps executed by the link relationship storage process disclosed in any of the foregoing embodiments.

[0088] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0089] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0090] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0091] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0092] The foregoing has provided a detailed description of a link relationship storage method, apparatus, device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A link relationship storage method characterized by comprising: include: The server obtains log data according to preset rules and sends the log data to the memory data queue. Then, several computing coroutines obtain request log data from the memory data queue and determine the index number of the storage shard according to the request ID corresponding to the request log data. The single slice dictionary in the storage shard is located based on the index number, and the single slice dictionary is locked when the request ID exists in the single slice dictionary; the single slice dictionary is a data structure in the storage shard used to store request IDs with the same index number and their corresponding request data; The request data in the single-slice dictionary is obtained based on the request ID, and the current level number of the request data is calculated. Then, the existence of a SpanId dataset in the current level is detected based on the current level number. The SpanId dataset is a collection of all SpanIds belonging to the same level in the same request tree. If the SpanId dataset exists, the SpanId slice dictionary at the current level is assigned a value using the created request entity data, and the link tree shard cache is updated using the request entity data based on the calculated fingerprint ID; the SpanId slice dictionary is a data structure used to store the mapping relationship between SpanId and corresponding request entity data, and the link tree shard cache is a cache space used to store request entity data in the request tree; The request entity data in the link tree shard cache is calculated in a round-robin fashion according to preset rules to obtain link relationship data, and the link relationship data is stored in the database; Before updating the link tree shard cache using the request entity data based on the calculated fingerprint ID, the process further includes: Calculate the fingerprint ID of the link based on the method ID, application ID, and class ID of the request log data.

2. The link relationship storage method according to claim 1, characterized by, The step of obtaining log data through a computing server according to preset rules includes: The default log consumer reports log data to the computing server through the first coroutine based on the GRPC protocol; Log data is retrieved from the preset log consumer via a second coroutine based on the HTTP protocol on the computing server.

3. The link relationship storing method according to claim 1, wherein The step of determining the index number of the storage shard based on the request ID corresponding to the request log data includes: Determine the request ID corresponding to the request log data, and calculate the hash value corresponding to the request ID using a preset hash algorithm; The index number of the storage fragment is determined based on the hash value and the number of storage fragments.

4. The link relationship storage method according to claim 1, characterized by, After retrieving request log data from the memory data queue via several coroutines, the process further includes: The corresponding application ID and class ID are obtained based on the method ID of the request log data.

5. The link relationship storage method according to any one of claims 1 to 4, characterized by, The step of polling and calculating the request entity data in the link tree shard cache according to preset rules includes: The link tree shard cache is polled at preset time intervals to obtain link tree shard cache objects, and the pending data tasks of each coroutine are allocated according to the number of link shards. Each coroutine obtains a copy of the corresponding single data piece, executes the calculation process of the requested entity data in the top-level node of the link tree, and executes the calculation process of the requested entity data in the sub-level nodes of the link tree after the top-level node has successfully calculated the data, until all link tree nodes have been traversed.

6. The link relationship storage method according to claim 5, wherein The process of calculating entity data requested in the top-level node of the execution link tree also includes: If a corresponding top-level node number exists in the detection layer number data group, and if it does, then check if the link tree has a break number and whether the break number exceeds a preset number. If there is no fault number or the fault number does not exceed the preset number, then obtain the SpanID corresponding to the top-level node of the link tree and create the entity node of the top-level link tree.

7. The link relationship storage method according to claim 5, wherein The process of calculating the requested entity data in the sub-nodes of the execution link tree also includes: Check if the current sub-layer node has a corresponding SpanID. If it does, check if the current sub-layer node includes the SpanID feature information of the parent node. If included, the corresponding entity list is obtained based on the SpanID feature information, and the entity list is traversed to obtain the node information of this layer.

8. A link relationship storage device characterized by comprising: include: The data acquisition module is used to acquire log data through the computing server according to preset rules, send the log data to the memory data queue, and then acquire request log data from the memory data queue through several coroutines, and determine the index number of the storage shard according to the request ID corresponding to the request log data; The storage shard determination module is used to locate the single-shard dictionary based on the index number, and lock the single-shard dictionary when the request ID exists in the single-shard dictionary; the single-shard dictionary is a data structure in the storage shard used to store request IDs with the same index number and their corresponding request data; The link level detection module is used to obtain the request data in the single slice dictionary according to the request ID, calculate the current level number of the request data, and then detect whether the current level has a SpanId dataset based on the current level number; the SpanId dataset is a set of all SpanIds belonging to the same level in the same request tree; The link cache update module is used to, if the SpanId dataset exists, assign values ​​to the SpanId slice dictionary of the current level using the created request entity data, and update the link tree shard cache based on the calculated fingerprint ID using the request entity data; the SpanId slice dictionary is a data structure used to store the mapping relationship between SpanId and corresponding request entity data, and the link tree shard cache is a cache space used to store request entity data in the request tree; The link data storage module is used to poll and calculate the request entity data in the link tree shard cache according to preset rules to obtain link relationship data, and store the link relationship data in the database; Before updating the link tree shard cache using the request entity data based on the calculated fingerprint ID, the device is further used to calculate the link fingerprint ID based on the method ID, application ID, and class ID of the request log data.

9. An electronic device, comprising: include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the link relationship storage method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used for storing computer programs; wherein, when the computer program is executed by a processor, it implements the steps of the link relationship storage method as described in any one of claims 1 to 7.

Images