A data processing method and device, equipment, and storage medium

Abstract

The application discloses a data processing method and device, equipment and a storage medium, comprising: determining a first code block currently running in a target service; obtaining a first micro-service identifier and a first location identifier of a first micro-service to which the first code block belongs; creating a first code block identifier of the first code block; the first code block identifier is used for uniquely pointing to the first code block; generating a service identifier of the first code block based on at least the first micro-service identifier, the first location identifier, the first code block identifier and a target service identifier; and sending the service identifier of the first code block to a service link monitoring end, so that the service link monitoring end establishes a service logic relationship of the target service based on at least the service identifier of the first code block.

Description

Technical Field

[0001] This application relates to the field of microservices technology, including but not limited to data processing methods, apparatus, devices, and storage media. Background Technology

[0002] As microservice architecture continues to evolve, inter-service calls within a system are becoming increasingly frequent. The advantages of microservice architecture include encapsulating the entire business process within different service processes. Microservices are loosely coupled and independent, enabling rapid response, quick deployment, and easy local modification, resulting in extremely high development and deployment efficiency. However, microservice architecture also makes the overall system structure more complex. Service calls span multiple microservice nodes, increasing the probability of inconsistencies in distributed transaction data. Overall, data tracing, business flow analysis, and troubleshooting become more difficult in systems employing microservice architecture.

[0003] Against this backdrop, distributed end-to-end service monitoring technology has emerged to address the challenge of tracing cross-service call chains. Most end-to-end monitoring methods utilize probe technology, dynamically acquiring relevant data and logs of microservices during runtime and sending them to the monitoring system. Simultaneously, the monitoring system uses link identifiers to connect data from different microservices belonging to the same business chain. This allows users to quickly retrieve microservice function calls within the same business chain, reducing the difficulty of problem localization and improving problem-solving efficiency.

[0004] However, it can be seen that the relevant technologies connect different microservices in the same business chain at the microservice level. However, they still cannot locate functions or methods within microservices, or code blocks or lines of code within methods. Therefore, when the logic of functions or methods within microservices is relatively complex, it is still not possible to quickly find the line of code where the problem is located. Summary of the Invention

[0005] This application provides a data processing method, apparatus, device, and storage medium. In this solution, a first microservice can obtain the business code of the currently running code block and send the business identifier of the code block to the business link monitoring end. This enables the business monitoring end to establish the business logic relationship of the target business based on the code block dimension, thereby quickly locating the code block within the microservice of the target business and improving the efficiency of location.

[0006] The technical solution of this application is implemented as follows:

[0007] In a first aspect, this application provides a data processing method, which is applied to a first microservice terminal, wherein the first microservice terminal is any microservice terminal running a target business, and the method includes:

[0008] Determine the first code block currently running in the target business logic;

[0009] Obtain the first microservice identifier and the first location identifier of the first microservice to which the first code block belongs; the first microservice identifier is used to uniquely point to the first microservice, and the first location identifier is used to characterize the position of the first microservice in the business logic relationship of the target business;

[0010] Create a first code block identifier for the first code block; the first code block identifier is used to uniquely point to the first code block;

[0011] The business identifier of the first code block is generated based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier;

[0012] The service identifier of the first code block is sent to the service link monitoring terminal so that the service link monitoring terminal can establish the service logic relationship of the target service based at least on the service identifier of the first code block.

[0013] Secondly, this application provides a data processing apparatus, which is deployed on a first microservice terminal, wherein the first microservice terminal is any microservice terminal running the target business, and the apparatus includes:

[0014] The determination unit is used to determine the first code block currently running in the target business;

[0015] The acquisition unit is used to acquire the first microservice identifier and the first location identifier of the first microservice to which the first code block belongs; the first microservice identifier is used to uniquely point to the first microservice, and the first location identifier is used to characterize the position of the first microservice in the business logic relationship of the target business.

[0016] A creation unit is used to create a first code block identifier for the first code block; the first code block identifier is used to uniquely point to the first code block.

[0017] The generation unit is configured to generate a business identifier for the first code block based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier;

[0018] The sending unit is used to send the service identifier of the first code block to the service link monitoring terminal, so that the service link monitoring terminal can establish the service logic relationship of the target service based at least on the service identifier of the first code block.

[0019] Thirdly, this application also provides a data processing device, including: a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement the above-described data processing method.

[0020] Fourthly, this application also provides a storage medium on which a computer program is stored, which, when executed by a processor, implements the above-described data processing method.

[0021] The data processing method, apparatus, device, and storage medium provided in this application include: determining a first code block currently running in a target service; obtaining a first microservice identifier and a first location identifier of a first microservice to which the first code block belongs; the first microservice identifier is used to uniquely point to the first microservice, and the first location identifier is used to characterize the position of the first microservice in the business logic relationship of the target service; creating a first code block identifier for the first code block; the first code block identifier is used to uniquely point to the first code block; generating a business identifier for the first code block based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target service identifier; and sending the business identifier of the first code block to a business link monitoring terminal, so that the business link monitoring terminal establishes the business logic relationship of the target service based at least on the business identifier of the first code block.

[0022] In the scheme of this application, the first microservice creates a first code block identifier for the currently running code block, and generates a business identifier for the first code block based on the first code block identifier, the first microservice identifier, and the first location identifier; and sends the first code block to the business link monitoring end; so that the business monitoring end can establish the business logic relationship of the target business based on the code block dimension. In this way, firstly, the first code block can be quickly located in the logical relationship of the target business based on its business identifier (the first microservice can be quickly located based on the first microservice identifier and the first location identifier, and the first code block can be quickly located in the first microservice based on the first code block identifier), thus improving the location efficiency; secondly, the process of generating the business identifier of the first code block does not rely on remote procedure calls of network interfaces, so it can be applied to target businesses triggered by scheduled tasks, thus expanding the application scope; thirdly, the first location identifier is used to characterize the position of the first microservice in the business logic relationship of the target business. The business identifier of the first code block generated based on the first location identifier can clearly determine the position of the first microservice to which the first code block belongs in the business logic relationship of the target business, avoiding the problem of possible location errors when determining the position of the microservice in the business logic relationship of the target business based on the time order of the received microservice data at the business link monitoring end. Therefore, this application also improves the accuracy of microservice location. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of an optional structure of the data processing system provided in an embodiment of this application;

[0024] Figure 2 A schematic diagram of a first optional data processing method provided in an embodiment of this application;

[0025] Figure 3 A schematic diagram of a second optional data processing method provided in an embodiment of this application;

[0026] Figure 4 A schematic diagram of a third optional data processing method provided in the embodiments of this application;

[0027] Figure 5 A schematic diagram of a fourth optional data processing method provided in the embodiments of this application;

[0028] Figure 6 A schematic diagram of a fifth optional data processing method provided in the embodiments of this application;

[0029] Figure 7 A schematic diagram of a sixth optional data processing method provided in the embodiments of this application;

[0030] Figure 8 This is a schematic diagram of an optional structure of the data processing apparatus provided in an embodiment of this application;

[0031] Figure 9 This is a schematic diagram of an optional structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of the application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0033] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0034] In the following description, the terms "first," "second," and "third" are used only to distinguish different objects and do not represent a specific order of objects, nor are they constituting a chronological order. It is understood that "first," "second," and "third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0036] This application provides data processing methods, apparatus, devices, and storage media. In practical applications, the data processing method can be implemented by a data processing apparatus, and the functional entities in the data processing apparatus can be collaboratively implemented by the hardware resources of electronic devices (such as terminal devices), such as computing resources like processors and communication resources (such as those used to support various communication methods like optical fiber and cellular).

[0037] The data processing method provided in this application embodiment is applied to a data processing system, which includes a first microservice terminal, a second server terminal, and a business link monitoring terminal.

[0038] The first microservice endpoint is used to execute: determine the first code block currently running in the target business; obtain the first microservice identifier and the first location identifier of the first microservice to which the first code block belongs; the first microservice identifier is used to uniquely point to the first microservice, and the first location identifier is used to characterize the position of the first microservice in the business logic relationship of the target business; create a first code block identifier for the first code block; the first code block identifier is used to uniquely point to the first code block; generate a business identifier for the first code block based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier; and send the business identifier of the first code block to the business link monitoring endpoint so that the business link monitoring endpoint establishes the business logic relationship of the target business based at least on the business identifier of the first code block.

[0039] The execution process of the second microservice is similar to that of the first microservice, and will not be described in detail here.

[0040] The first and second microservice terminals can be electronic devices with relevant data processing capabilities. For example, a microservice terminal can be an independent physical electronic device such as a server or host; or, a microservice terminal can be a virtual electronic device such as a cloud host or virtual machine.

[0041] The business link monitoring terminal is used to perform the following: receive the business identifier of the first code block sent by the first microservice terminal, as well as the business identifiers of other code blocks of the first microservice; receive the business identifier of the code block of the second microservice sent by the second microservice terminal; and establish the business logic relationship of the target business based on the business identifiers of these code blocks.

[0042] The business link monitoring terminal can be an electronic device with relevant data processing capabilities. For example, the business link monitoring terminal can be an independent physical electronic device such as a server, or a virtual electronic device such as a virtual machine.

[0043] This application does not impose specific limitations on the deployment method of the microservice terminal and the business link monitoring terminal, which can be determined according to the actual situation. For example, the microservice terminal and the business link monitoring terminal can be deployed independently on different electronic devices, or they can be centrally deployed on the same electronic device.

[0044] This application does not limit the specific number of microservices included in the data processing system; the number can be configured according to actual needs. For example, the data processing system may also include a third microservice, etc.

[0045] As an example, the structure of a data processing system can be as follows: Figure 1 As shown, it includes: a first microservice terminal 10, a second service terminal 20, a third service terminal 30, and a business link monitoring terminal 40.

[0046] The first microservice terminal 10 is used to execute: determining the first code block currently running in the target business; obtaining the first microservice identifier and the first location identifier of the first microservice to which the first code block belongs; the first microservice identifier is used to uniquely point to the first microservice, and the first location identifier is used to characterize the position of the first microservice in the business logic relationship of the target business; creating a first code block identifier for the first code block; the first code block identifier is used to uniquely point to the first code block; generating a business identifier for the first code block based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier; and sending the business identifier of the first code block to the business link monitoring terminal, so that the business link monitoring terminal establishes the business logic relationship of the target business based at least on the business identifier of the first code block.

[0047] The execution process of the second and third microservices (20 and 30) is similar to that of the first microservice (10).

[0048] The business link monitoring terminal 40 is used to perform the following: receiving the business identifier of the first code block sent by the first microservice terminal 10, as well as the business identifiers of other code blocks of the first microservice; receiving the business identifier of the code block of the second microservice sent by the second microservice terminal 20; receiving the business identifier of the code block of the third microservice sent by the second microservice terminal 30; and establishing the business logic relationship of the target business based on the business identifiers of these code blocks.

[0049] For ease of understanding, some technical terms involved in the embodiments of this application are explained.

[0050] Microservice architecture can be understood as a cloud-native architectural approach where a single application consists of many loosely coupled and independently deployable smaller components or services. A small component or service is called a microservice.

[0051] A code block is a collection of code in a program that follows a specific format. For example, a code block can be a function body or a sub-function body.

[0052] For example, the target business is a post-medical application that consists of multiple microservices.

[0053] Below, in conjunction with Figure 1 The schematic diagram of the data processing system shown illustrates various embodiments of the data processing methods, apparatus, devices, and storage media provided in this application.

[0054] In a first aspect, embodiments of this application provide a data processing method. This method is applied to a data processing device, which can be deployed on an electronic device serving as a first microservice. The functions implemented by this method can be achieved by a processor in the electronic device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the electronic device includes at least a processor and a storage medium.

[0055] The following describes the data processing method provided in this application embodiment, taking the first microservice as an example.

[0056] The first microservice is any microservice running the target business.

[0057] This application does not specifically limit the type and process of the target service, which can be determined according to the actual situation. For example, the target service can be smart healthcare or smart education, etc.

[0058] This application does not limit the specific number of microservices involved in the target service, and can determine it according to the actual situation. The number of microservices involved in the target service can be an integer greater than or equal to 2.

[0059] Figure 2 This is a flowchart illustrating the data processing method according to an embodiment of this application, as shown below. Figure 2 As shown, this data processing method generates a business identifier for a code block and sends the business identifier to the business link monitoring terminal, enabling the monitoring terminal to establish the business logic relationship of the target business based on the received business identifier. The processing procedure for the first code block is described below using the first code block as an example. For the processing procedures of other code blocks, please refer to the description of the first code block.

[0060] Specifically, the process may include, but is not limited to, S201 to S204 below.

[0061] S201. The first microservice determines the first code block currently running for the target business.

[0062] The first code block is the code block currently running in the target business logic.

[0063] For example, S201 can be implemented as follows: The first microservice uses bytecode enhancement technology to dynamically parse the compiled bytecode file and obtain the currently running code block (taking Java language as an example, the code statements within curly braces {} are code blocks, and the statements in function bodies, branch structures and loop structures within {} are all the same code block).

[0064] It should be noted that the first code block will change as the target business program runs; that is, the first code block will traverse every code block in the target business program during the execution of the target business.

[0065] S202, the first microservice terminal obtains the first microservice identifier and the first location identifier of the first microservice to which the first code block belongs.

[0066] The first microservice is the microservice to which the first code block belongs.

[0067] The first microservice identifier is the microservice identifier of the first microservice. The first microservice identifier is used to uniquely point to the first microservice. That is, the microservice identifier of a microservice is used to uniquely point to that microservice. This application embodiment does not limit the specific format of the first microservice identifier, which can be determined according to the actual situation. For example, the first microservice identifier can be a fixed-length string.

[0068] The first location identifier is the location identifier of the first microservice. The first location identifier is used to characterize the position of the first microservice within the business logic relationship of the target service. This application embodiment does not limit the specific format of the first location identifier pair, and it can be determined according to the actual situation. For example, the first location identifier can be a numerical value. For instance, if the first location identifier is 1, it indicates that the first microservice is the first microservice called by the target service during runtime. If the first location identifier is 2, it indicates that the first microservice is the second microservice called by the target service during runtime.

[0069] It should be noted that the specific methods for obtaining the first microservice identifier and the first location identifier may differ or be the same for different first code blocks. This application embodiment does not impose a unique limitation on the specific process by which the first microservice obtains the first microservice identifier and the first location identifier of the first microservice to which the first code block belongs, and can be determined according to the actual situation.

[0070] The first micro-service identifier here can be obtained through parsing or it can be newly created. The first location identifier here can be obtained based on the parsed location identifier or it can be newly configured.

[0071] S203. The first microservice creates the first code block identifier of the first code block.

[0072] The first code block identifier is used to uniquely point to the first code block. This application embodiment limits the specific process of creating the first code block identifier, which can be determined according to actual circumstances.

[0073] This application does not limit the specific format of the first code block identifier, which can be determined according to the actual situation. For example, the first code block identifier can be a string of a fixed length.

[0074] For example, in S203, the first microservice generates a fixed-length string based on the location and function name of the first code block, and uses this string as the identifier of the first code block.

[0075] S204. The first microservice terminal generates the business identifier of the first code block based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier.

[0076] The target service identifier is used to uniquely identify the target service. This application embodiment does not limit the specific format of the target service identifier, which can be determined according to actual circumstances. For example, the target service identifier can be a fixed-length character string.

[0077] The service identifier of the first code block is used to locate the first code block in the target service. This application embodiment does not limit the method of generating the service identifier of the first code block; it can be determined according to the actual situation.

[0078] The embodiments of this application do not limit the content on which the business identifier for generating the first code block by the first microservice is based, and can be determined according to the actual situation.

[0079] In one possible implementation, the first microservice generates a business identifier for the first code block based on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier.

[0080] In another possible implementation, the first microservice generates the business identifier of the first code block based on the first microservice identifier, the first location identifier, the first code block identifier, the loop count of the first code block, and the target business identifier.

[0081] In another possible implementation, the first microservice generates the business identifier of the first code block based on the first microservice identifier, the first location identifier, the first code block identifier, the exception identifier, and the target business identifier.

[0082] In another possible implementation, the first microservice generates the business identifier of the first code block based on the first microservice identifier, the first location identifier, the first code block identifier, the exception identifier, the loop count of the first code block, and the target business identifier.

[0083] S205. The first microservice sends the business identifier of the first code block to the business link monitoring terminal, so that the business link monitoring terminal can establish the business logic relationship of the target business based at least on the business identifier of the first code block.

[0084] In one possible implementation, the first microservice directly sends the business identifier for generating the first code block to the business link monitoring terminal through a relevant interface. This method is characterized by its simplicity.

[0085] In another possible implementation, the first microservice caches the business identifier for generating the first code block in a cache area. After storing a certain number of data pairs in the cache area, they are sent together to the business link monitoring end through the relevant interface. This method can alleviate the pressure on network transmission.

[0086] This application embodiment does not limit the thread that executes the action of the first microservice in S201 to S205, and can determine it according to the actual situation.

[0087] In one possible implementation, the first microservice executes S201 to S205 via the main thread.

[0088] Understandably, if other services are running concurrently, S201 to S205 can also be executed through other threads.

[0089] Understandably, as the target business program runs, the first microservice will send the business identifier of each of its code blocks to the business chain monitoring end. Other microservices will also send the business identifier of each of their code blocks to the business chain monitoring end.

[0090] For the handling of other microservices and other code blocks, please refer to the detailed descriptions in S201 to S205, which will not be repeated here.

[0091] Correspondingly, the business link monitoring terminal receives the business identifier of each code block, parses the business identifier of each code block, obtains the microservice identifier and location identifier of all microservices, as well as the code block identifier of the code block in the microservice, and establishes the business logic relationship of the target business based on the microservice identifier, location identifier and code block identifier of the code block in the microservice.

[0092] This application does not limit the specific format of the established logical relationship of the target business in the embodiments, and it can be determined according to the actual situation. For example, the logical relationship of the target business can be a logical relationship diagram, in which one node corresponds to one microservice, and multiple microservices are connected through the position of the microservices. The attribute information of the microservices contains the code block identifier of the code blocks they include; or the logical relationship of the target business can be a logical relationship table.

[0093] The data processing scheme provided in this application includes: determining a first code block currently running in the target business; obtaining a first microservice identifier and a first location identifier of a first microservice to which the first code block belongs; the first microservice identifier is used to uniquely point to the first microservice, and the first location identifier is used to characterize the position of the first microservice in the business logic relationship of the target business; creating a first code block identifier for the first code block; the first code block identifier is used to uniquely point to the first code block; generating a business identifier for the first code block based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier; and sending the business identifier of the first code block to a business link monitoring terminal, so that the business link monitoring terminal establishes the business logic relationship of the target business based at least on the business identifier of the first code block.

[0094] In the scheme of this application, the first microservice creates a first code block identifier for the currently running code block, and generates a business identifier for the first code block based on the first code block identifier, the first microservice identifier, and the first location identifier; and sends the first code block to the business link monitoring end; so that the business monitoring end can establish the business logic relationship of the target business based on the code block dimension. In this way, firstly, the first code block can be quickly located in the logical relationship of the target business based on its business identifier (the first microservice can be quickly located based on the first microservice identifier and the first location identifier, and the first code block can be quickly located in the first microservice based on the first code block identifier), thus improving the location efficiency; secondly, the process of generating the business identifier of the first code block does not rely on remote procedure calls of network interfaces, so it can be applied to target businesses triggered by scheduled tasks, thus expanding the application scope; thirdly, the first location identifier is used to characterize the position of the first microservice in the business logic relationship of the target business. The business identifier of the first code block generated based on the first location identifier can clearly determine the position of the first microservice to which the first code block belongs in the business logic relationship of the target business, avoiding the problem of possible location errors when determining the position of the microservice in the business logic relationship of the target business based on the time order of the received microservice data at the business link monitoring end. Therefore, this application also improves the accuracy of microservice location.

[0095] Understandably, after the first code block obtains the business identifier, it needs to pass the business identifier of the first code block to the next executed code block, so that the next executed code block can know the microservice identifier and location identifier of the microservice to which the previous executed code block belongs after parsing the business identifier.

[0096] The following describes the process by which the first microservice terminal in S202 obtains the first microservice identifier and the first location identifier of the first microservice to which the first code block belongs. Specifically, this may include, but is not limited to, the following case 1 or case 2.

[0097] Case 1: If the first code block is the first code block to run in the first microservice, obtain the first microservice identifier and the first location identifier;

[0098] Case 2: If the first code block is not the first code block to run in the first microservice, obtain the first microservice identifier and the first location identifier.

[0099] The following describes the process of obtaining the first microservice identifier and the first location identifier when the first code block is the first running code block in the first microservice (Scenario 1). Figure 3 As shown, the specific details may include, but are not limited to, S301 to S303 below.

[0100] This application does not limit the method for detecting the first running code block in its embodiments; it can be determined according to the actual situation. For example, a microservice program includes a main function, and the first sub-function in the main function is determined as the first code block.

[0101] S301, The first microservice generates the identifier of the first microservice.

[0102] Since the first code block is the first code block in the first microservice, all first microservices create a new first microservice identifier during the processing of the first code block.

[0103] The embodiments of this application do not limit the specific method for generating or creating the first microservice identifier, and can be determined according to the actual situation.

[0104] For example, the first microservice converts its name into a string, padding it with zeros if necessary, to obtain a fixed-length identifier for the first microservice.

[0105] For example, the first microservice performs an MD5 signature based on the microservice name in the configuration file, and then obtains the process ID of the microservice in the operating system and concatenates them into a string (for example, the MD5 signature of the microservice name test-microservice generates a 32-bit string 82517674fc9304e6c1a7891dc73a35dd). Since a 64-bit operating system can have a maximum of 4,194,304 processes, process IDs with fewer than 7 digits are padded with leading zeros, resulting in the first microservice identifier being 82517674fc9304e6c1a7891dc73a35dd0005365. Here, 0005365 is the process ID, representing the MD5 signature of the microservice name 82517674fc9304e6c1a7891dc73a35dd.

[0106] S302. If the first microservice receives the business identifier of the second code block, it parses the business identifier of the second code block, obtains the second location identifier of the second microservice, and increments the second location identifier by one to obtain the first location identifier.

[0107] The second position identifier is used to characterize the position of the second microservice in the business logic relationship of the target business.

[0108] The second code block is the last code block to run in the second microservice, and the first code block runs immediately after the second code block. That is, the first code block runs after the second code block finishes running.

[0109] The second microservice is different from the first microservice.

[0110] Therefore, if the first microservice can receive the business identifier of the second code block of the second microservice, it means that the first microservice is not the first microservice. Since the generation of the business identifier of the second code block depends on the second position identifier of the second microservice, parsing the business identifier of the second code block can obtain the second position identifier of the second microservice. It can be known that the first microservice runs after the second microservice runs, so adding one to the second position identifier can obtain the first position identifier.

[0111] S303. If the first microservice does not receive the business identifier of the second code block, it determines that the first location identifier is one.

[0112] If the first microservice does not receive the business identifier of the second code block, the corresponding first microservice is the first microservice executed in the target business. Therefore, the first position identifier is directly configured to one.

[0113] The following describes the process of obtaining the first microservice identifier and the first location identifier when the first code block in scenario 2 is not the first code block to run in the first microservice. Figure 4 As shown, the specific implementation may include, but is not limited to, S401 and S402 described below.

[0114] S401, The first microservice receives the business identifier of the third code block.

[0115] The third code block belongs to the first microservice, and the first code block continues to run after the third code block.

[0116] This application embodiment does not limit the specific method by which the first server receives the service identifier of the third code block, and can be configured according to the actual situation.

[0117] For example, after generating its business identifier, the third code block caches the business identifier in the business identifier stack. The first microservice then calls the newly generated business identifier in the business identifier stack to obtain the business identifier of the third code block.

[0118] S402. The first microservice parses the business identifier of the third code block to obtain the first microservice identifier and the first location identifier of the first microservice.

[0119] Since the third code block also belongs to the first microservice, the business identifier of the third code block is obtained based on the first microservice identifier and the first position identifier of the first microservice. Therefore, by parsing the business identifier of the third code block, the first microservice identifier and the first position identifier can be obtained directly.

[0120] The process of parsing the first microservice identifier and the first position identifier corresponds to the process of generating the business identifier. For example, if the first microservice identifier is a 24-character string and the first position identifier is a 4-character string, and when concatenating to generate the business identifier, the first microservice identifier is placed in the first 24 characters, and the first position identifier is placed in characters 25 to 28. Therefore, when parsing the business identifier of the third code block, the first 24 characters after parsing are used as the first microservice identifier, and the 25th to 28th characters after parsing are used as the first position identifier.

[0121] The process of creating the first code block identifier of the first code block in the first microservice in S203 will be described below.

[0122] In one possible implementation, the process may include, but is not limited to, Figure 5 S2031 to S2033 are shown.

[0123] S2031, The first microservice determines the identifier of the first microservice, the function class name and function name of the first code block, and the position of the first code block in the first microservice.

[0124] For example, S2031 can be implemented as follows: the first microservice determines the identifier of the first microservice in the configuration file, determines the function class name and function name of the first code block in the program file of the first code block, and determines the position of the first code block in the first microservice in the program of the first code block.

[0125] S2032, The first microservice concatenates the identifier of the first microservice, the function class name and function name of the first code block, and the position of the first code block to obtain the first string.

[0126] The embodiments of this application do not limit the sequential relationship between the various identifiers during splicing, and can be determined according to the actual situation.

[0127] For example, the first microservice concatenates the identifier of the first microservice, the function class name and function name of the first code block, and the position of the first code block in sequence to obtain the first string.

[0128] For example, the first microservice can also concatenate the function class name and function name of the first code block, the position of the first code block, and the identifier of the first microservice in sequence to obtain the first string.

[0129] S2033. The first microservice terminal converts the first string into the first code block identifier based on a hash algorithm.

[0130] For example, the first microservice in S2033 converts the first string into a fixed number of characters based on a hash algorithm to obtain the identifier of the first code block.

[0131] This application does not limit the specific type of hash algorithm, which can be determined according to the actual situation. For example, hash algorithms may include, but are not limited to: SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512.

[0132] The following describes the process by which the first microservice in S204 generates the business identifier of the first code block based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier.

[0133] The process may include, but is not limited to, any one of the following methods 1 to 4.

[0134] Method 1: The first microservice generates the business identifier of the first code block based on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier;

[0135] Method 2: The first microservice generates the business identifier of the first code block based on the first microservice identifier, the first location identifier, the first code block identifier, the loop count of the first code block, and the target business identifier;

[0136] Method 3: The first microservice generates the business identifier of the first code block based on the first microservice identifier, the first location identifier, the first code block identifier, the exception identifier, and the target business identifier.

[0137] Method 4: The first microservice generates the business identifier of the first code block based on the first microservice identifier, the first location identifier, the first code block identifier, the exception identifier, the loop count of the first code block, and the target business identifier.

[0138] In Method 1, the first microservice concatenates the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier to obtain the business identifier of the first code block.

[0139] The embodiments of this application do not limit the splicing order, and can be configured according to the actual situation.

[0140] The following describes the process by which the first microservice in Method 2 generates the business identifier of the first code block based on the first microservice identifier, the first location identifier, the first code block identifier, the number of loops of the first code block, and the target business identifier.

[0141] In Method 2, the data processing method provided in this application embodiment also includes, but is not limited to, the following S20421 to S20423.

[0142] S20421. If the first code block is a code block in a loop, the first microservice determines the number of times the first code block is executed in a loop.

[0143] For example, S20421 can be implemented as follows: when parsing the first code block, if the first microservice identifies the keywords of the loop statement (such as For and While), it determines that the first code block is a code block in the loop body, and the first microservice further determines the number of times the first code block is executed.

[0144] When determining the number of times the first code block is executed, the loop statement keyword is identified and the number of times the first code block is executed is set to 1. Then, the number of times the first code block is executed is incremented by 1 each time the loop is executed.

[0145] S20422. The first microservice updates the loop count of the first code block based on the number of loop executions.

[0146] S20423. The first microservice concatenates the first microservice identifier, the first location identifier, the first code block identifier, the loop count of the first code block, and the target business identifier to obtain the business identifier of the first code block.

[0147] The embodiments of this application do not impose specific limitations on the splicing order, which can be determined according to the actual situation.

[0148] Since the number of bits for the business identifier in the first code block is fixed, if the number of bits is insufficient after concatenation, it can be padded with zeros to make up the number of bits.

[0149] The following describes the process by which the first microservice in Method 3 generates the business identifier of the first code block based on the first microservice identifier, the first location identifier, the first code block identifier, the exception identifier, and the target business identifier.

[0150] In Method 3, the data processing method provided in this application embodiment also includes, but is not limited to, the following S20431 to S20433.

[0151] S20431. When the first microservice detects an exception in the first code block, it retrieves the exception content from the exception stack.

[0152] The first microservice runs through a probe monitoring program. Once an error occurs during the execution of a certain first code block, the probe will detect the error, identify the code that caused the error, and obtain the error information, i.e., the exception content, from the Java Virtual Machine stack.

[0153] S20432. The first microservice maps the abnormal content to the corresponding abnormal identifier.

[0154] This application does not limit the number of digits in the anomaly identifier or the content represented by each digit; it can be configured according to actual needs.

[0155] For example, an exception identifier may include an exception type code and an exception code.

[0156] The mapping relationship between exception content and corresponding exception identifier is pre-configured. For example, the mapping relationship between exception content and corresponding exception identifier is pre-configured and stored in a configuration file. In S20432, the first microservice reads the configuration file and maps the exception content to the corresponding exception identifier according to the mapping relationship between exception content and corresponding exception identifier in the configuration file.

[0157] For example, no exceptions were caught, and the exception identifier was 000000.

[0158] For example, if the probe's main thread captures an exception as java.lang.OutOfMemoryError, the exception identifier will be 100005. Here, 1 indicates a system exception; 00005 indicates insufficient memory.

[0159] For example, if a custom exception is encountered (custom exceptions are defined by developers and are usually strongly related to the microservice's business logic), such as a User Type Exception, the exception identifier would be 200112. Here, 2 indicates a custom exception, and 00112 indicates a user type error.

[0160] For example, when a higher log level such as warning, error, or critical is captured from a logging statement, an exception identifier is generated based on the log level mapping in the configuration file, such as exception identifier 300002. Here, 3 indicates a logging exception, and 00002 indicates an error occurred in the log.

[0161] S20433, The first microservice concatenates the first microservice identifier, the first location identifier, the first code block identifier, the exception identifier, and the target business identifier to obtain the business identifier of the first code block.

[0162] The embodiments of this application do not impose specific limitations on the splicing order, which can be determined according to the actual situation.

[0163] Since the number of bits for the business identifier in the first code block is fixed, if the number of bits is insufficient after concatenation, it can be padded with zeros to make up the number of bits.

[0164] The following describes the process by which the first microservice in Method 4 generates the business identifier of the first code block based on the first microservice identifier, the first location identifier, the first code block identifier, the exception identifier, the loop count of the first code block, and the target business identifier.

[0165] In Method 4, the data processing method provided in this application embodiment also includes, but is not limited to, the following S20441 to S20445.

[0166] S20441. If the first code block is a code block in a loop, the first microservice determines the number of times the first code block is executed.

[0167] The implementation of S20441 can be found in the detailed description in S20421, and will not be repeated here.

[0168] S20442. The first microservice updates the loop count of the first code block based on the number of loop executions.

[0169] S20443. When the first microservice detects an exception in the first code block, it retrieves the exception content from the exception stack.

[0170] The implementation of S20443 can be found in the detailed description in S20431, and will not be repeated here.

[0171] S20444, The first microservice maps the abnormal content to the corresponding abnormal identifier.

[0172] The implementation of S20444 can be found in the detailed description in S20432, and will not be repeated here.

[0173] S20445. The first microservice concatenates the first microservice identifier, the first location identifier, the first code block identifier, the loop count of the first code block, the exception identifier, and the target business identifier to obtain the business identifier of the first code block.

[0174] The embodiments of this application do not impose specific limitations on the splicing order, which can be determined according to the actual situation.

[0175] Since the number of bits for the business identifier in the first code block is fixed, if the number of bits is insufficient after concatenation, it can be padded with zeros to make up the number of bits.

[0176] The data processing method provided in the embodiments of this application may include, but is not limited to, other than, the following: Figure 6 S206 to S208 are shown.

[0177] S206. When the first microservice detects that there is an anomaly in the first code block, it retrieves at least one abnormal code block and the variables of each of the at least one abnormal code block from the code block cache.

[0178] The abnormal code block is the predicted code block that causes the first code block to be abnormal.

[0179] Code block caching is used to cache a certain number of recently executed code blocks. As the program runs, the code blocks in the code block cache are continuously updated.

[0180] When an anomaly is detected in the first code block, the logical relationship between the code blocks is complex, so the anomaly may not necessarily be caused by the first code block, but by the code blocks preceding the first code block. Therefore, the code blocks in the code block cache are used as predicted abnormal code blocks that may cause the first code block to anomaly.

[0181] S206 can be implemented as follows: when the first microservice detects that there is an exception in the first code block, it retrieves at least one exception code block from the code block cache; for each exception code block in the at least one exception code block, it reads the variables of each code block.

[0182] S207. The first microservice associates the at least one exception code block and the variables of each exception code block with the business identifier of the first code block and caches them in the first cache area.

[0183] The embodiments of this application do not limit the specific method of association, which can be determined according to the actual situation. For example, association can be carried out through a table or through a certain identifier, etc.

[0184] The first cache area is used to cache the associated exception code block and the variables of the exception code block.

[0185] It should be noted that, when caching here, exception code blocks and their variables can be cached according to the execution order of the program.

[0186] S208. When the first sending condition is met, the first microservice sends the data in the first cache area to the business link monitoring terminal.

[0187] The embodiments of this application do not limit the specific content of the first sending condition, which can be determined according to the actual situation.

[0188] In one possible implementation, the first sending condition can be a certain number of exception code blocks.

[0189] In another possible implementation, the first sending condition can be a time period.

[0190] In this way, if the first sending condition is met, the data in the first buffer area is sent to the business link monitoring end, which can avoid resource consumption problems and sending efficiency problems caused by frequent network interface calls.

[0191] The data processing method provided in this application embodiment may also include steps S209 to S210.

[0192] S209. The first microservice collects log information during the execution of the first code block and obtains N log messages.

[0193] N is an integer greater than 1.

[0194] S209 can be implemented as follows: The first microservice uses a probe to identify the log statements in the first code block, collects log information during the execution of the first code block, and obtains N log messages.

[0195] S210. The first microservice filters the N log messages based on the log filtering level to obtain M log messages.

[0196] M is less than or equal to N.

[0197] The log filtering level can be configured according to the actual situation, and this application embodiment does not make specific limitations on it.

[0198] This allows for the filtering out of unnecessary log information, thereby improving processing efficiency.

[0199] S211. The first microservice associates the M log entries with the business identifier of the first code block and caches them in the second cache area.

[0200] The embodiments of this application do not limit the specific method of association, which can be determined according to the actual situation. For example, association can be carried out through a table or through a certain identifier, etc.

[0201] The second cache area is used to cache the associated log information to be sent.

[0202] S212. When the second sending condition is met, the first microservice sends the data in the second cache area to the business link monitoring terminal.

[0203] The second sending condition can be the same as or different from the first sending condition. This application embodiment does not specifically limit the content of the second sending condition, and it can be configured according to the actual situation.

[0204] In this way, if the second sending condition is met, the data in the second buffer area is sent to the business link monitoring end, which can avoid resource consumption problems and sending efficiency problems caused by frequent network interface calls.

[0205] In one possible implementation, the processing of the business identifier in the first code block can be executed through the main thread. The handling of exception blocks and log information can be performed through a separate thread. This way, on the one hand, the execution of the main thread is not affected, and on the other hand, processing efficiency can be improved.

[0206] Furthermore, in order to improve the transmission efficiency of log information, the log information can be merged before being transmitted to the business link monitoring end.

[0207] Specifically, this may include merging the M log entries.

[0208] The merging process includes: determining the similarity between the first log information and the reference log for the first log information; the first log information is any one of the M log information.

[0209] If the similarity is greater than or equal to the similarity threshold, a difference log is recorded and the merging count is incremented by one; the similar log is used as a new reference log, and the similarity is used as a new similarity threshold to continue merging the second log information. The merging of the second log information is similar to the merging of the first log information, and will not be described in detail here.

[0210] The similarity log is used to characterize the identical content between the first log information and the reference log, and the difference log is used to characterize the different content between the first log information and the reference log; the second log is any log information other than the first log information among the M log information.

[0211] If the similarity is less than the similarity threshold, then all the contents of the first log information are recorded.

[0212] The data processing method provided in the embodiments of this application will be described below through a complete process.

[0213] For example, related technology 1 discloses a microservice link monitoring method, including: creating a target probe and a target monitoring log format; collecting log data corresponding to the calls of the target microservice link based on the target probe and according to the target monitoring log format; obtaining log data from the target probe through a data collector and sending the log data to a search server so that the search server can monitor and analyze the target microservice link.

[0214] In related technology 1, which uses trace identifiers (TraceId) and span identifiers (SpanId) to connect microservices, this approach has the following problems:

[0215] Issue 1: Using TraceId and SpanId as link identifiers can only link remote procedure calls (RPCs) based on network interfaces. When a microservice's scheduled task is initiated as a business function, TraceId and SpanId cannot be generated, causing the entire business link to fail to link.

[0216] Question 2: While TraceId and SpanId can be used to link microservices and functions or methods within microservices, they cannot pinpoint the code blocks or lines within functions or methods. Therefore, when the logic of a function or method is complex, it is still not possible to quickly identify the line of code where the problem occurs.

[0217] Question 3: The monitoring solution relies on system logs, requiring programmers to insert logging statements in sensitive areas of the code. If a system problem occurs and the code section where the problem occurred is not logged, the problem cannot be located.

[0218] Question 4: System logs consist of multiple strings (including numerous Chinese characters, English letters, numbers, symbols, etc.). When encoded into byte streams for network transmission as network packets (or message queue messages), they often consume a significant amount of bandwidth and network traffic. Furthermore, the network components receiving the logs require substantial memory, placing a burden on the hardware of the monitoring system.

[0219] In this embodiment of the application, a business ID (equivalent to the business identifier of the code block) is used instead of TraceId and SpanId in the related technologies. This business ID is composed of a microservice ID (msId) (equivalent to the microservice identifier), a microservice count (msNum) (equivalent to the microservice location identifier), a block ID (blockId) (equivalent to the block identifier), a block loop count (loopNum) (equivalent to the block loop count), a universally unique identifier (UUID) (equivalent to the target business identifier), an error type code (errorType), and an error code (errorNum) (equivalent to the exception identifier). With the corresponding algorithm and processing scheme, the above four problems can be solved or optimized.

[0220] Below, for reference Figure 7The process may include, but is not limited to, S701 to S711 described below.

[0221] S701, the probe of microservice A starts the main thread, reads the configuration file, creates msId and msNum, and then starts other threads and components.

[0222] Regarding msId, it's important to understand that a microservice system is a mesh-like system composed of numerous microservices. These microservices call each other, and these call relationships are the lines on this mesh structure. Each microservice is a point on this mesh structure, called a microservice node. To accurately locate a specific microservice within this mesh-like microservice system, an identifier is needed, namely the msId (microservice ID or microservice identifier). Within the same microservice system, the msId is unique and cannot be duplicated.

[0223] Regarding msNum, it's important to understand that in a mesh-structured microservice system, a business flow is a line that traverses several microservice nodes, and these traverses follow a strict chronological order. To indicate the position of a microservice within this business flow, an identifier, msNum, is needed. When the business flow passes through the first microservice, msNum is created, initially set to 1, representing the microservice's position in this business flow as 1. If the business flow proceeds to the next microservice node, msNum increments by 1, becoming 2, representing the microservice's position in this business flow as 2.

[0224] In S701, the probe first starts the main thread and reads the configuration file.

[0225] The probe's main thread generates a microservice ID (msId) based on the microservice name. The msId is used to identify the nodes corresponding to different microservices in the same business chain.

[0226] For example, the configuration file may include, but is not limited to, the following:

[0227] The microservice name, used to generate msId in S701;

[0228] Code block cache size refers to the size of the cache maintained by the code block cache thread;

[0229] The number of exception backtracking code blocks refers to the number of code blocks available for backtracking in the code block call queue;

[0230] Error coding mapping, used to generate errorNum in S704;

[0231] Log level mapping, used to generate errorNum in S704;

[0232] Log filtering level, used in S706 to filter low-level logs by log level;

[0233] Log buffer size refers to the size of the buffer maintained by the log buffer thread;

[0234] Log merge similarity is used to provide a basis for log merging in S706.

[0235] After reading the configuration file in the main thread, the probe starts other threads and components.

[0236] The other threads started may include, but are not limited to, the following:

[0237] The code block cache thread is used to maintain the code block cache area (equivalent to the first cache area mentioned above); and when certain conditions are met, it sends the contents of the code block cache area to an external message queue to be sent to the business link monitoring end.

[0238] The log caching thread is used to maintain the log cache area (equivalent to the second cache area mentioned above); and when certain conditions are met, it sends the contents of the log cache area to an external message queue to be sent to the business link monitoring end.

[0239] The code block call queue (equivalent to the code block cache area mentioned above) is a (First Input First Output, FIFO) queue used to cache a specified number of code block information.

[0240] Step S702: The main thread uses dynamic identification of the running code block to create a blockId, puts the code block information into the code block call queue, and assembles the businessId.

[0241] Specifically, during program execution, the main thread dynamically parses the compiled bytecode file using bytecode enhancement technology, obtains the currently running code block (in Java, for example, the code statements within curly braces {} constitute a code block; the function bodies, branch structures, and loop structures within {} all belong to the same code block), generates a code block number (blockId), and simultaneously obtains the variable values ​​for that code block during runtime, converting these values ​​to strings. The blockId and the corresponding variable values ​​are then placed into the code block call queue as code block information.

[0242] Because the code block call queue has a length limit and follows the first-in-first-out standard, once more than a specified number of new code block messages enter the queue, the queue will automatically discard the code block message that entered the queue first.

[0243] A FIFO queue has a fixed (adjustable) length. Once the number of stored data reaches the queue length, storing N more data will automatically discard the first N stored data. The purpose of using a FIFO queue to store code block call information is that there is a large amount of code block call information, so its number must be controlled. Secondly, when code block information is used to assist in troubleshooting, only a certain number of calls are generally needed; premature context (calls many times before the code block where the problem occurred) is not very helpful for troubleshooting. Therefore, the FIFO data structure is naturally suitable for this use case. It saves memory, and the FIFO queue itself handles the discarding of premature context, making subsequent code block backtracking easier.

[0244] Specifically, the blockId and its corresponding variable value are placed into the code block call queue as code block information for subsequent backtracking in S705.

[0245] Regarding `blockId`, it's important to note that it's an identifier for a code block. Its key characteristic is that each `blockId` should be unique across all code within the entire microservice system. The specific generation method is as follows: a string is constructed based on the microservice identifier (unique), the fully qualified class name and function name of the function containing the code block, and the position of the code block within the function. This string is then used to generate the `blockId` using a hash algorithm. This method ensures that each `blockId` is independent and unique throughout the entire microservice system.

[0246] Then, the main thread creates the code block loop count loopNum (initially 0), the universal identifier UUID (used to uniquely identify the current business), the error type code errorType (initially 0), and the error code errorNum (initially 00000). It then assembles UUID + msId + msNum + blockId + loopNum + errorType + errorNum into businessId. The values ​​of several fields can be modified by the following steps.

[0247] Step S703: When the main thread encounters loop code, it merges the code blocks in the loop body and the logs, and modifies loopNum.

[0248] When the main thread parses a code block, if it recognizes the keywords for loop statements (For and While), it creates a loop count identifier `loopNum` for the code block within the loop body and increments `loopNum` sequentially based on the number of times the code block is called in the loop. Therefore, if the same code block is executed many times in a loop, a new `blockId` is created instead of a duplicate, and then `loopNum` is incremented. This merging method reduces the number of looping code blocks.

[0249] Step S704: When the main thread detects a system error and records the exception stack information, it modifies errorType and errorNum.

[0250] It's important to note that the probe works by monitoring program execution within the JVM (the virtual machine environment in which Java programs run). Once an error occurs in a code block, the probe immediately detects the error in the JVM, identifies the problematic code, and retrieves the error information, i.e., the exception stack information, from the Java Virtual Machine stack.

[0251] When the main thread catches a system exception, it retrieves the exception stack contents from the system's exception stack and creates the error type code errorType and the error code errorNum.

[0252] The errorType has three values: 0 represents no error, 1 represents a system exception, 2 represents a custom exception, and 3 represents a log error.

[0253] errorNum is determined by the error coding mapping in the configuration file:

[0254] 1. For example, if the probe's main thread does not capture any exceptions, errorType+errorNum is 000000.

[0255] 2. For example, if the probe's main thread captures a java.lang.OutOfMemoryError (out of memory error) thrown by the system, the errorType + errorNum is 100005.

[0256] 3. For example, the probe's main thread might capture a custom exception (this exception is defined by the developer and is usually strongly related to the microservice's business logic), such as UserTypeException (user type error), where errorType + errorNum is 200112.

[0257] 4. For example, when the main thread captures higher-level logs such as warnings, errors, and critical warnings thrown by logging statements, it will form an errorNum according to the log level mapping in the configuration file. For example, the errorType + errorNum corresponding to the error type log is 300002.

[0258] At this point, after a code block has run, all field values ​​in `businessId` are confirmed, and the `businessId` for that code block will not change. Since the generation of `businessId` does not rely on network-based RPC, it can be applied to the function (method) code block triggered by a scheduled task, thus solving the first problem mentioned above.

[0259] Step S705: The main thread backtracks the code block information based on whether an exception has been caught, caches the businessId and exception information, and passes the businessId to the next code block.

[0260] If the main thread catches any exception in step S704, it will trigger a backtracking of the blockId and variable values ​​in the code block call queue. The code blocks and variable values ​​that may have caused the error will be concatenated from newest to oldest, and then combined with the exception stack information from step S704 to form the exception information. This exception information is then paired with the businessId of the current code block and sent to the code block cache. If no error occurs during the execution of the current code block, the exception information will be empty.

[0261] By tracing back code block calls, a complete code block call chain can be obtained, along with the variable values ​​within each code block in the chain, even when log points are missing. This method is highly accurate and convenient for locating problems, effectively solving problems two and three mentioned above.

[0262] Step S706: When the main thread encounters a log statement, it sends the log that needs to be sent to the log buffer according to the log filtering level in the configuration file.

[0263] It's important to note that the probe works by monitoring program execution within the JVM (the virtual machine environment in which Java programs run). Therefore, it can identify logging statements within code blocks; for example, `logger.error("xxxx")` is a statement that logs an error.

[0264] When the main thread parses a code block, it identifies logging statements and filters out logs that do not need to be collected, such as logs with low levels but large quantities (e.g., DEBUG level), based on the log filtering level in the configuration file. Logs that meet the collection criteria are then sent to the log cache for caching, along with the businessId of the current code block.

[0265] When a code block is a loop, it will also generate a large number of similar or even identical logs. The main thread merges the logs in the loop body according to the algorithm in step S703. When merging loop logs, since there may be a large amount of identical content and a small amount of inconsistency, the configuration file provides a log merging similarity score (greater than 0, less than 1) as the basis for merging.

[0266] The log merging method is as follows. Assuming the configured log merging similarity is X%, when the first log entry is retrieved, it is not merged. This log entry is set as the baseline log A (equivalent to the reference log mentioned above), with a baseline similarity of X% (equivalent to the similarity threshold mentioned above). When the second log entry B is retrieved, A is compared with B using a pattern matching algorithm, and the similarity Y% is calculated. The similar log content a and the difference log b are then compared. Y is compared to X. If Y is greater than X, log B is considered mergeable with A, and only the difference log b and the current loop count n are recorded. Then, the similar log a is set as the new baseline log A, and the similarity Y% is set as the new baseline similarity X%. If Y is less than X, log B and A are considered unmergeable. All of B's ​​content is recorded, but the current loop count is not recorded. The similar log content a is set as the new baseline log A, and X%-Y% is set as the new baseline similarity. The merging process for the next log entry is then initiated.

[0267] The above log merging algorithm is highly efficient. Secondly, by dynamically adjusting the baseline log and similarity, it can merge similar log content as much as possible, reduce the content of circular logs, make more efficient use of the log cache, and reduce the amount of data transmitted over the network, thus optimizing bandwidth utilization and thus improving problem four to some extent.

[0268] S707. After receiving the businessId, the next code block parses it and starts working to generate a businessId belonging to this code block.

[0269] According to step S705, after the previous code block finishes execution, the main thread generates a businessId based on the previous code block and passes it to the next code block to be called. Upon receiving the businessId, the next code block parses it and obtains the blockId and loopNum. Since it is a new code block, the main thread generates a new blockId, sets loopNum to zero, and ensures that msId, msNum, and UUID remain unchanged. The generation of blockId, loopNum, errorType, and errorNum repeats the process from S702 to S706.

[0270] Step S708: The code block cache thread and the log cache thread send the content that meets the conditions in their respective caches to the message queue.

[0271] Because frequent data transmission constantly occupies the host's network ports and places a significant burden on the central processing unit (CPU), this application uses caching technology to temporarily store code block information and log information. The code block caching thread and the log caching thread are two independent threads that run concurrently with the main thread and do not interfere with each other.

[0272] The caching threads continuously monitor the size of their respective caches. When the data in a cache reaches its storage limit, the two threads assemble the contents of the cache in a strict sequential order and then send them to the message queue. This method reduces the need to frequently open and close network ports, saving CPU resources.

[0273] Step S709: The last code block executed by microservice A passes the businessId to the next microservice B to be invoked via network pass-through.

[0274] When the probe main thread of microservice A detects that microservice A needs to call microservice B via RPC, the probe main thread of A will pass the businessId generated by the last executed code block of A to microservice B through network pass-through. This network pass-through method is consistent with the traditional TraceId approach; the main difference is that the businessId is included in the sent network data packet, allowing microservice B's probe to retrieve the businessId from the received network data packet.

[0275] Step S710: The probe of microservice B receives the businessId passed from upstream microservice A, parses it, obtains the UUID, and starts running.

[0276] When the probe main thread of microservice B obtains the network data packet sent by A in step S709, it retrieves the businessId generated by A, parses it, and obtains the UUID. This is because the process of A calling B belongs to the same business link, so the same UUID is used for identification. At the same time, the probe main thread of B detects the execution of the first code block of B, creates msId, msNum, etc., jumps to step S702 and repeats the execution until all the code of microservice B has been executed, and then sends the businessId of the last code block of B to microservice C.

[0277] The reason for using msNum instead of the solutions in related technologies (based on traceId and spanId) is as follows: Related technologies use spanId to identify the previous microservice, thereby determining the call order of the previous and current microservices in the business chain. However, there is a special case: There are three microservices A, B, and C, and a business chain call pattern is A->B->C->B->A. When microservice B executes its business logic, it receives parentIds from microservices A and C sequentially. According to the business logic, A->B should precede C->B. However, if, due to some special circumstances (such as network congestion), C->B arrives at the monitoring platform before A->B when B sends the two call messages to the monitoring system, the platform will place C->B before A->B when connecting the business chains, resulting in a final business chain of C->B->A->B, which does not match the correct business chain.

[0278] In this embodiment of the application, msNum is used, which increments by 1 each time a new microservice is introduced. This ensures that the microservice nodes in the business chain modify msNum according to a strict size sequence, guaranteeing the correct order of calls between microservices. This solution solves the problem in a very simple way without introducing new components or entities, with very low cost and high efficiency, without introducing new high time complexity.

[0279] Step S711: The monitoring platform receives code block information and logs from the message queue and associates them using the businessId.

[0280] The monitoring platform retrieves code block information issued by all microservice probe code block cache threads and log information issued by log cache threads from the message queue.

[0281] Since the code block information contains businessId and exception information, and the log information contains businessId and logs, the monitoring platform iterates through each businessId and parses it.

[0282] Retrieve the UUID from businessId, categorize the code block information (including msId+msNum+blockId+loopNum+errorType+errorNum) and logs belonging to the same UUID, and retrieve msNum from businessId, then concatenate each code block information and log in ascending order of msNum.

[0283] Because the code block information includes exception stack traces, the last executed code blocks, and internal variable values, problems can be accurately and easily located even if logs are missing. Furthermore, techniques for handling looping code blocks and merging logs reduce the pressure on network data transmission and reception, making data retrieval easier. Finally, since the businessId contains errorType and errorNum, the monitoring platform can categorize and filter all abnormal code block information based on errorType and errorNum. This method is simple and intuitive, eliminating the need for additional big data cleaning and analysis, thus reducing implementation complexity.

[0284] This embodiment of the application has the following characteristics:

[0285] First, this embodiment uses a businessId constructed from UUID+msId+msNum+blockId+loopNum+errorType+errorNum, which carries more information than the scheme using TraceId and SpanId. Furthermore, the generation of businessId does not depend on network-based RPC, and can target the function (method) code block when a scheduled task is triggered, thereby solving the problem of missing business links in business initiated by scheduled tasks.

[0286] Secondly, using msId and msNum together can more easily solve the problem of out-of-order links in the microservice callback (A->B->C->B->A) method.

[0287] Third, the system delves into the execution of code blocks within the business logic. When an error occurs, it records information about several code blocks in the call chain and the variable values ​​within those blocks through backtracking. This allows for quick and effective problem location and troubleshooting even in the event of missing logs.

[0288] Fourth, a similarity-based log merging algorithm is proposed, which has certain advantages in time complexity. It merges similar log content as much as possible, reduces the content of circular logs, makes more efficient use of the log cache, and can also reduce the amount of data transmitted during network transmission to a certain extent, thus optimizing network bandwidth utilization.

[0289] Fifth, the solution using errorType and errorNum together can more easily and quickly classify system and log errors by error type without the need for additional data cleaning and analysis work based on massive amounts of error information and logs.

[0290] Secondly, embodiments of this application provide a data processing device, which is deployed on a first microservice terminal. The first microservice terminal can be any microservice terminal running the target business, such as... Figure 8 As shown, the data processing device 80 may include a determining unit 801, an acquiring unit 802, a creating unit 803, a generating unit 804, and a sending unit 805. Wherein:

[0291] Determining unit 801 is used to determine the first code block currently running in the target service;

[0292] The acquisition unit 802 is used to acquire the first microservice identifier and the first location identifier of the first microservice to which the first code block belongs; the first microservice identifier is used to uniquely point to the first microservice, and the first location identifier is used to characterize the position of the first microservice in the business logic relationship of the target business.

[0293] Creation unit 803 is used to create a first code block identifier for the first code block; the first code block identifier is used to uniquely point to the first code block.

[0294] The generation unit 804 is configured to generate a business identifier for the first code block based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier;

[0295] The sending unit 805 is used to send the service identifier of the first code block to the service link monitoring terminal, so that the service link monitoring terminal can establish the service logic relationship of the target service based at least on the service identifier of the first code block.

[0296] In some embodiments, if the first code block is the first running code block in the first microservice, the acquisition unit 802 is configured to: generate the first microservice identifier; if the business identifier of the second code block is received, parse the business identifier of the second code block, obtain the second position identifier of the second microservice, increment the second position identifier by one, and obtain the first position identifier; the second code block is the last running code block in the second microservice, and the first code block runs after the second code block; if the business identifier of the second code block is not received, determine that the first position identifier is one.

[0297] In some embodiments, if the first code block is not the first code block to run in the first microservice, the acquisition unit 802 is configured to: receive the service identifier of the third code block; the third code block belongs to the first microservice, and the first code block runs after the third code block; parse the service identifier of the third code block to obtain the first microservice identifier and the first location identifier of the first microservice.

[0298] In some embodiments, the creation unit 803 is configured to: determine the identifier of the first microservice, the function class name and function name of the first code block, and the position of the first code block in the first microservice; concatenate the identifier of the first microservice, the function class name and function name of the first code block, and the position of the first code block to obtain a first string; and convert the first string into the identifier of the first code block based on a hash algorithm.

[0299] In some embodiments, the generation unit 804 is further configured to: if the first code block is a code block in a loop body, determine the number of times the first code block is executed in a loop; update the number of times the first code block is executed in a loop based on the number of times the first code block is executed in a loop; and concatenate the first microservice identifier, the first location identifier, the first code block identifier, the number of times the first code block is executed in a loop and the target business identifier to obtain the business identifier of the first code block.

[0300] In some embodiments, the generation unit 804 is further configured to: obtain exception content from the exception stack when an exception is detected in the first code block; map the exception content to a corresponding exception identifier; and concatenate the first microservice identifier, the first location identifier, the first code block identifier, the exception identifier, and the target business identifier to obtain the business identifier of the first code block.

[0301] In some embodiments, the generation unit 804 is further configured to: if the first code block is a code block in a loop body, determine the number of times the first code block is executed; update the number of times the first code block is executed based on the number of times the loop is executed; if an exception is detected in the first code block, obtain the exception content in the exception stack; map the exception content to the corresponding exception identifier; and concatenate the first microservice identifier, the first location identifier, the first code block identifier, the number of times the first code block is executed, the exception identifier, and the target business identifier to obtain the business identifier of the first code block.

[0302] In some embodiments, the data processing apparatus 80 further includes an abnormal code block processing unit, which is configured to: when an abnormality is detected in the first code block, retrieve at least one abnormal code block and variables of each of the at least one abnormal code block from a code block cache; the abnormal code block is a predicted code block that causes the first code block to be abnormal; associate the at least one abnormal code block and variables of each of the at least one abnormal code block with the service identifier of the first code block and cache them in a first cache area; and when a first sending condition is met, send the data in the first cache area to the service link monitoring terminal.

[0303] In some embodiments, the data processing apparatus 80 further includes a log processing unit, which is used for:

[0304] Log information is collected during the execution of the first code block to obtain N log messages; N is an integer greater than 1; the N log messages are filtered based on the log filtering level to obtain M log messages; M is less than or equal to N; the M log messages are associated with the service identifier of the first code block and then cached in a second cache area; under the condition of satisfying the second sending condition, the data in the second cache area is sent to the service link monitoring terminal.

[0305] In some embodiments, the log processing unit is further configured to perform the following before sending the data in the second buffer area to the service link monitoring terminal, provided that the second sending condition is met: merging the M log messages; wherein the merging includes: determining the similarity between the first log message and a reference log for the first log message; the first log message being any one of the M log messages; if the similarity is greater than or equal to a similarity threshold, recording a difference log and incrementing the merging count by one; using the similar log as a new reference log and the similarity as a new similarity threshold, continuing the merging of the second log message; the similar log is used to characterize the same content between the first log message and the reference log, and the difference log is used to characterize the different content between the first log message and the reference log; the second log is any one of the M log messages except the first log message; if the similarity is less than the similarity threshold, recording all the content of the first log message.

[0306] It should be noted that the data processing device provided in this application embodiment includes all the units included, which can be implemented by a processor in an electronic device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA), etc.

[0307] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0308] It should be noted that, in the embodiments of this application, if the above-described data processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0309] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor executes the program, it implements the steps in the data processing method provided in the above embodiments.

[0310] The following is combined Figure 9 The electronic device 90 shown is illustrated with a structural diagram of the electronic device.

[0311] In one example, electronic device 90 can be the aforementioned electronic device. For example... Figure 9As shown, the electronic device 90 includes: a processor 901, at least one communication bus 902, a user interface 903, at least one external communication interface 904, and a memory 905. The communication bus 902 is configured to enable communication between these components. The user interface 903 may include a display screen, and the external communication interface 904 may include standard wired and wireless interfaces.

[0312] The memory 905 is configured to store instructions and applications executable by the processor 901, and can also cache data to be processed or already processed by the processor 901 and various modules in the electronic device (e.g., performance data, alarm rule data), which can be implemented by flash memory or random access memory (RAM).

[0313] Fourthly, embodiments of this application provide a storage medium, namely a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps in the data processing method provided in the above embodiments.

[0314] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0315] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0316] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0317] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0318] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0319] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0320] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0321] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0322] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method is applied to a first microservice, which is any microservice running the target business, and the method includes: Determine the first code block currently running in the target business logic; Obtain the first microservice identifier and the first location identifier of the first microservice to which the first code block belongs; the first microservice identifier is used to uniquely point to the first microservice, and the first location identifier is used to characterize the position of the first microservice in the business logic relationship of the target business; Create a first code block identifier for the first code block; the first code block identifier is used to uniquely point to the first code block; The business identifier of the first code block is generated based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier; The service identifier of the first code block is sent to the service link monitoring terminal so that the service link monitoring terminal can establish the service logic relationship of the target service based at least on the service identifier of the first code block.

2. The method according to claim 1, characterized in that, If the first code block is the first code block to run in the first microservice, obtaining the first microservice identifier and first location identifier of the first microservice to which the first code block belongs includes: Generate the first microservice identifier; If the business identifier of the second code block is received, the business identifier of the second code block is parsed to obtain the second position identifier of the second microservice, and the second position identifier is incremented by one to obtain the first position identifier; the second code block is the last code block to run in the second microservice, and the first code block runs after the second code block; If the service identifier of the second code block is not received, then the first location identifier is determined to be one.

3. The method according to claim 1, characterized in that, If the first code block is not the first running code block in the first microservice, obtaining the first microservice identifier and first location identifier of the first microservice to which the first code block belongs includes: Receive the service identifier of the third code block; the third code block belongs to the first microservice, and the first code block runs after the third code block; The business identifier of the third code block is parsed to obtain the first microservice identifier and the first location identifier of the first microservice.

4. The method according to claim 1, characterized in that, The first code block identifier that creates the first code block includes: Determine the identifier of the first microservice, the function class name and function name of the first code block, and the position of the first code block in the first microservice; The identifier of the first microservice, the function class name and function name of the first code block, and the position of the first code block are concatenated to obtain the first string; The first string is converted into the identifier of the first code block based on a hash algorithm.

5. The method according to claim 1, characterized in that, The step of generating the business identifier of the first code block based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier includes: If the first code block is a code block in a loop body, determine the number of times the first code block is executed; Update the loop count of the first code block based on the number of loop executions; The first microservice identifier, the first location identifier, the first code block identifier, the loop count of the first code block, and the target business identifier are concatenated to obtain the business identifier of the first code block.

6. The method according to claim 1, characterized in that, The step of generating the business identifier of the first code block based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier includes: If an exception is detected in the first code block, the exception content is retrieved from the exception stack. Map the abnormal content to the corresponding abnormal identifier; The first microservice identifier, the first location identifier, the first code block identifier, the exception identifier, and the target business identifier are concatenated to obtain the business identifier of the first code block.

7. The method according to claim 1, characterized in that, The step of generating the business identifier of the first code block based at least on the first microservice identifier, the first location identifier, the first code block identifier, and the target business identifier includes: If the first code block is a code block in a loop body, determine the number of times the first code block is executed; Update the loop count of the first code block based on the number of loop executions; If an exception is detected in the first code block, the exception content is retrieved from the exception stack. Map the abnormal content to the corresponding abnormal identifier; The first microservice identifier, the first location identifier, the first code block identifier, the loop count of the first code block, the exception identifier, and the target business identifier are concatenated to obtain the business identifier of the first code block.

8. The method according to claim 1, characterized in that, The method further includes: If an anomaly is detected in the first code block, at least one abnormal code block and variables of each of the at least one abnormal code block are retrieved from the code block cache; the abnormal code block is the predicted code block that caused the anomaly in the first code block. The at least one abnormal code block and the variables of each of the at least one abnormal code block are associated with the business identifier of the first code block and then cached in the first cache area; If the first sending condition is met, the data in the first buffer area will be sent to the service link monitoring terminal.

9. The method according to claim 1, characterized in that, The method further includes: Log information is collected during the execution of the first code block to obtain N log messages; where N is an integer greater than 1. The N log messages are filtered based on the log filtering level to obtain M log messages; M is less than or equal to N. The M log entries are associated with the business identifier of the first code block and then cached in the second cache area; If the second sending condition is met, the data in the second buffer area will be sent to the service link monitoring terminal.

10. The method according to claim 9, characterized in that, Before sending the data in the second buffer area to the service link monitoring terminal when the second sending condition is met, the method further includes: Merge the M log entries; The merging process includes: determining the similarity between the first log information and the reference log for the first log information; the first log information is any one of the M log information; If the similarity is greater than or equal to the similarity threshold, a difference log is recorded and the merging count is incremented by one; the similar log is used as a new reference log, and the similarity is used as a new similarity threshold to continue merging the second log information; the similar log is used to characterize the same content between the first log information and the reference log, and the difference log is used to characterize the different content between the first log information and the reference log; the second log is any log information other than the first log information among the M log information; If the similarity is less than the similarity threshold, then all the contents of the first log information are recorded.

11. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the program to implement the data processing method of any one of claims 1 to 10.

Images