Data processing method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to a data processing method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: adding a corresponding request identifier to a received data processing request for a to-be-analyzed system to obtain an updated data processing request; recording associated data of a request branch under the condition that the updated data processing request flows through the request branch of the to-be-analyzed system; the associated data is used for representing an association relationship between the request identifier and the branch identifier of the request branch; and obtaining a branch identifier associated with the same request identifier based on the associated data, and determining a superposition link of the to-be-analyzed system based on the branch identifier associated with the same request identifier. By adopting the method, the link carding efficiency can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to a data processing method, apparatus, electronic device, storage medium, and computer program product. Background Technology

[0002] With the rapid development of business, the number of business scenarios connected to the system is increasing, and different business scenarios can be combined and superimposed; therefore, it is very important to sort out the system's links.

[0003] In related technologies, current link sorting methods generally rely on manual analysis of the system's business scenarios to identify the overlapping links of the system, such as business scenario H → business scenario I → business scenario J. However, the system has numerous business scenarios, and theoretically there may be trillions of overlapping links. Manually sorting the links requires a lot of time, resulting in low link sorting efficiency. Summary of the Invention

[0004] This disclosure provides a data processing method, apparatus, electronic device, storage medium, and computer program product to at least solve the problem of low link clearing efficiency in related technologies. The technical solution of this disclosure is as follows:

[0005] According to a first aspect of the present disclosure, a data processing method is provided, comprising:

[0006] Add a corresponding request identifier to the received data processing request for the system to be analyzed to obtain the updated data processing request;

[0007] When the updated data processing request flows through the request branch of the system to be analyzed, the associated data of the request branch is recorded; the associated data is used to characterize the relationship between the request identifier and the branch identifier of the request branch;

[0008] Based on the associated data, branch identifiers associated with the same request identifier are obtained, and based on the branch identifiers associated with the same request identifier, the overlay links of the system to be analyzed are determined.

[0009] In one exemplary embodiment, before recording the associated data of the requested branch, the method further includes:

[0010] When the updated data processing request flows through the request branch of the system to be analyzed, the preset data recording instruction of the request branch is invoked to obtain the request identifier of the updated data processing request and the branch identifier of the request branch.

[0011] Generate the association between the request identifier and the branch identifier;

[0012] Based on the aforementioned relationship, the associated data of the requested branch is obtained.

[0013] In one exemplary embodiment, before adding a corresponding request identifier to the received data processing request for the system to be analyzed to obtain the updated data processing request, the method further includes:

[0014] Generate a preset data record instruction for the requested branch;

[0015] Add the preset data recording instruction to the branch instruction set of the requested branch;

[0016] The step of invoking the preset data record instruction of the request branch to obtain the request identifier of the updated data processing request and the branch identifier of the request branch includes:

[0017] Invoke the preset data record instruction in the branch instruction set of the request branch to obtain the request identifier of the updated data processing request and the branch identifier of the request branch.

[0018] In one exemplary embodiment, before obtaining the branch identifier associated with the same request identifier based on the associated data, the method further includes:

[0019] The associated data is stored in a message queue;

[0020] The step of obtaining branch identifiers associated with the same request identifier based on the associated data includes:

[0021] Extract the associated data from the message queue;

[0022] Based on the extracted associated data, the branch identifiers associated with the same request identifier are obtained.

[0023] In an exemplary embodiment, obtaining the branch identifier associated with the same request identifier based on the extracted association data includes:

[0024] Based on the request identifier in the extracted associated data, the associated data is classified to obtain associated data with the same request identifier;

[0025] Based on the branch identifier in the associated data of the same request identifier, the branch identifier associated with the same request identifier is obtained.

[0026] In one exemplary embodiment, the associated data further includes a timestamp corresponding to the branch identifier, the timestamp being used to characterize the time of execution of the requested branch;

[0027] The step of determining the overlay link of the system to be analyzed based on the branch identifier associated with the same request identifier includes:

[0028] Determine the order of the timestamps corresponding to the branch identifiers associated with the same request identifier;

[0029] Based on the aforementioned sequence, the stacking order of the branch identifiers associated with the same request identifier is determined;

[0030] According to the superposition order, the branch identifiers associated with the same request identifier are superimposed to obtain the superposition link of the system to be analyzed.

[0031] In an exemplary embodiment, after determining the overlay link of the system to be analyzed based on the branch identifier associated with the same request identifier, the method further includes:

[0032] The superimposed links of the system to be analyzed were verified, and the verification results were obtained.

[0033] If the verification result indicates that the superimposed link of the system to be analyzed has passed the verification, the superimposed link of the system to be analyzed is sent to the target terminal; the target terminal includes at least one of a monitoring terminal and a test terminal.

[0034] According to a second aspect of the present disclosure, a data processing apparatus is provided, comprising:

[0035] The identifier adding unit is configured to add a corresponding request identifier to the received data processing request for the system to be analyzed, thereby obtaining the updated data processing request;

[0036] The data recording unit is configured to record the associated data of the request branch when the updated data processing request flows through the request branch of the system to be analyzed; the associated data is used to characterize the association relationship between the request identifier and the branch identifier of the request branch;

[0037] The link determination unit is configured to perform operations based on the associated data to obtain branch identifiers associated with the same request identifier, and to determine the overlay links of the system to be analyzed based on the branch identifiers associated with the same request identifier.

[0038] In one exemplary embodiment, the apparatus further includes a data generation unit configured to, when the updated data processing request flows through a request branch of the system to be analyzed, invoke a preset data recording instruction of the request branch to obtain the request identifier of the updated data processing request and the branch identifier of the request branch; generate an association relationship between the request identifier and the branch identifier; and obtain associated data of the request branch based on the association relationship.

[0039] In one exemplary embodiment, the apparatus further includes an instruction adding unit configured to execute a preset data recording instruction for generating the request branch; and to add the preset data recording instruction to the branch instruction set of the request branch;

[0040] The data generation unit is further configured to execute the preset data record instruction in the branch instruction set of the request branch, and obtain the request identifier of the updated data processing request and the branch identifier of the request branch.

[0041] In one exemplary embodiment, the apparatus further includes a data storage unit configured to store the associated data in a message queue;

[0042] The link determination unit is further configured to extract the associated data from the message queue; and based on the extracted associated data, obtain the branch identifier associated with the same request identifier.

[0043] In one exemplary embodiment, the link determination unit is further configured to perform classification processing on the associated data according to the request identifier in the extracted associated data to obtain associated data with the same request identifier; and to obtain the branch identifier associated with the same request identifier based on the branch identifier in the associated data with the same request identifier.

[0044] In one exemplary embodiment, the associated data further includes a timestamp corresponding to the branch identifier, the timestamp being used to characterize the time of execution of the requested branch;

[0045] The link determination unit is further configured to determine the order of timestamps corresponding to branch identifiers associated with the same request identifier; determine the superposition order of branch identifiers associated with the same request identifier based on the order of ...

[0046] In one exemplary embodiment, the apparatus further includes a link transmission unit configured to perform verification of the overlay link of the system to be analyzed and obtain a verification result; if the verification result indicates that the overlay link of the system to be analyzed has passed verification, the overlay link of the system to be analyzed is transmitted to a target terminal; the target terminal includes at least one of a monitoring terminal and a test terminal.

[0047] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0048] processor;

[0049] Memory used to store the processor's executable instructions;

[0050] The processor is configured to execute the instructions to implement the data processing method as described in any embodiment of the first aspect.

[0051] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform a data processing method as described in any embodiment of the first aspect.

[0052] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including instructions that, when executed by a processor of an electronic device, enable the electronic device to perform a data processing method as described in any embodiment of the first aspect.

[0053] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0054] The process involves adding a corresponding request identifier to each received data processing request for the system under analysis, resulting in an updated data processing request. Next, as the updated data processing request flows through request branches of the system under analysis, the associated data of these request branches is recorded. This associated data characterizes the relationship between the request identifier and the branch identifiers of the request branches. Finally, based on the associated data, the branch identifiers associated with the same request identifier are obtained, and the overlay links of the system under analysis are determined based on these branch identifiers. In this way, by using the request identifier corresponding to the updated data processing request for the system under analysis, the branch identifiers of the request branches through which the updated data processing request flows are chained together, allowing for the rapid determination of the branch identifiers associated with the same request identifier. This enables the rapid determination of the overlay links of the system under analysis, achieving the goal of automatically determining the overlay links of the system under analysis. The entire process eliminates the need for manual link analysis, simplifying the link analysis process, reducing link analysis time, and thus improving link analysis efficiency.

[0055] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0056] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0057] Figure 1 This is a flowchart illustrating a data processing method according to an exemplary embodiment.

[0058] Figure 2 This is a flowchart illustrating a method for superimposed link sorting of a complex system according to an exemplary embodiment.

[0059] Figure 3 This is a flowchart illustrating the steps of obtaining associated data for a requested branch according to an exemplary embodiment.

[0060] Figure 4 This is a flowchart illustrating the steps of obtaining a branch identifier associated with the same request identifier, according to an exemplary embodiment.

[0061] Figure 5 This is a flowchart illustrating the steps of determining the overlay links of a system to be analyzed, according to an exemplary embodiment.

[0062] Figure 6 This is a flowchart illustrating another data processing method according to an exemplary embodiment.

[0063] Figure 7 This is a block diagram illustrating a data processing apparatus according to an exemplary embodiment.

[0064] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0065] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0066] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0067] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0068] In recent years, with the rapid development of e-commerce, the number of business scenarios integrated into the system has been increasing. These different business scenarios can be combined and superimposed, forming a Cartesian product-like superimposed link space, theoretically potentially containing trillions of possible superimposed links. However, due to the mutual exclusion relationships between different businesses, it is difficult to comprehensively identify all possible combinations manually, making system quality assurance extremely challenging and testing unable to accurately cover all online combinations and superimposed links. Traditional link identification relies on manual experience. For example, if a system has four business scenarios A, B, C, and D, experience suggests that A and C cannot be superimposed, while the others can. Theoretically, this results in 11 possible superimposed links: A, B, C, D, AB, AD, BC, BD, CD, ABD, and BCD; see Table 1 for details.

[0069] Table 1 Combined Overlay Links

[0070] Scene A B C D A / √ × √ B √ / √ √ C × √ / √ D √ √ √ /

[0071] Therefore, traditional link analysis methods that rely on empiricism are unreliable, failing to guarantee that the analyzer fully understands all business scenarios and accurately grasps their mutually exclusive relationships. In practice, errors are easily made, and the analyzed results do not perfectly match real-world online traffic. To address this, this publication proposes a data processing method, specifically a superimposed link analysis method (traffic coloring) for complex systems, which effectively improves link analysis efficiency.

[0072] Figure 1 This is a flowchart illustrating a data processing method according to an exemplary embodiment, such as... Figure 1 As shown, this data processing method is used in a server; it is understood that the method can also be applied to a system including terminals and servers, and implemented through interaction between the terminals and servers. The terminals can be, but are not limited to, various personal computers, laptops, smartphones, and tablets; the servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services. In this exemplary embodiment, the method includes the following steps:

[0073] In step S110, a corresponding request identifier is added to the received data processing request for the system to be analyzed, and the updated data processing request is obtained.

[0074] The system to be analyzed refers to a complex system encompassing multiple business scenarios, specifically a complex multi-scenario superposition system, such as a multi-scenario business system, like an e-commerce system. Different business scenarios can be combined and superimposed, for example, business scenario H → business scenario I → business scenario J, business scenario X → business scenario Y → business scenario Z, business scenario O → business scenario P → business scenario Q, etc.

[0075] Among these, data processing requests for the system to be analyzed refer to data processing requests that need to pass through the request branches of the system to be analyzed, such as order submission requests, delivery fee calculation requests, loan approval requests, etc. Figure 2 The flow rates shown are Flow Rate 1, Flow Rate 2, etc.

[0076] The request identifier is a unique identifier for the data processing request to the system being analyzed. It is used throughout the entire data processing request process and specifically refers to the request ID (Identifier), such as 123, 456, 578, or 982. In practical scenarios, data processing requests refer to traffic, which has a corresponding traffic ID, such as... Figure 2 The data points shown are: Flow 1, Flow 2, Flow 3, Flow 4, Flow 5, and Flow 6.

[0077] The updated data processing request carries or includes the corresponding request identifier.

[0078] It should be noted that adding a corresponding request identifier to the received data processing requests for the system to be analyzed is actually a traffic coloring process, used to track the request branches through which the data processing request flows.

[0079] For example, the server receives a data processing request for the system to be analyzed from the terminal. Then, based on the data processing request, the server generates a corresponding request identifier. This identifier can be generated by combining the timestamp, business prefix, and random string of the data processing request. Alternatively, the server can combine the timestamp, business prefix, and random string to obtain the updated request identifier. The server then adds the request identifier to the corresponding position in the data processing request, resulting in an updated data processing request.

[0080] For example, see reference. Figure 2 The server tags the online traffic to the system, such as traffic 1, traffic 2, traffic 3, etc.

[0081] In step S120, when the updated data processing request flows through the request branch of the system to be analyzed, the associated data of the request branch is recorded; the associated data is used to characterize the relationship between the request identifier and the branch identifier of the request branch.

[0082] Here, the request branch of the system to be analyzed refers to the code branch corresponding to the business scenario of the system to be analyzed, specifically the if-else branch, for example... Figure 2 The options shown are "Cross-border Goods", "Gift Promotions", "Use Now, Pay Later", and "Half-Screen Ordering".

[0083] In this context, the updated data processing request flowing through the request branch of the system to be analyzed refers to the updated data processing request triggering the request branch of the system to be analyzed. For example, for an order submission request on an e-commerce platform, if the user is a VIP member (if (user.isVIP())), the "VIP exclusive discount calculation logic" is triggered; if the order amount exceeds 1000 yuan (if(order.amount > 1000)), the "automatic deduction logic for full-reduction coupons" is triggered; if the delivery address is in a remote area (if(address.isRemote())), the "remote area shipping fee surcharge logic" is triggered.

[0084] The branch identifier of a request branch refers to the unique identification information of the request branch, such as request branch A, request branch B, request branch 100, request branch 17, etc.

[0085] The associated data of the request branch includes the relationship between the request identifier of the data processing request that triggered the update of the request branch and the branch identifier of the request branch. Specifically, it includes request identifier-branch identifier, such as traffic 1-branch identifier A, traffic 3-branch identifier D, traffic 5-branch identifier G, etc.

[0086] For example, when the updated data processing request flows through the request branch of the system to be analyzed, the server obtains the request identifier of the updated data processing request and the branch identifier of the request branch. Then, based on the request identifier of the updated data processing request and the branch identifier of the request branch, it generates an association relationship between the request identifier and the branch identifier (such as request identifier-branch identifier) ​​and uses the association relationship between the request identifier and the branch identifier as the association data of the request branch.

[0087] For example, see reference. Figure 2 For traffic 1, when traffic 1 passes through code branch A, record the association: traffic 1 - code branch A; when traffic 1 passes through code branch E, record the association: traffic 1 - code branch E; when traffic 1 passes through code branch H, record the association: traffic 1 - code branch H; when traffic 1 passes through code branch I, record the association: traffic 1 - code branch I.

[0088] In step S130, based on the associated data, the branch identifiers associated with the same request identifier are obtained, and based on the branch identifiers associated with the same request identifier, the overlay links of the system to be analyzed are determined.

[0089] Here, the branch identifier associated with the same request identifier refers to all branch identifiers associated with the same request identifier. Specifically, it is determined based on the branch identifiers in the associated data of the same request identifier, for example... Figure 2 The traffic flow shown is associated with code branches A, E, H, and I; traffic flow 2 is associated with code branches A, B, and F; traffic flow 3 is associated with code branches D, H, and I; traffic flow 4 is associated with code branches D, B, and C; traffic flow 5 is associated with code branches G, H, and F; and traffic flow 6 is associated with code branches G, D, E, and C.

[0090] The overlay link of the system to be analyzed is obtained by overlaying all branch identifiers associated with the same request identifier, for example... Figure 2 The diagram shows GDEC (corresponding to traffic 6), DBC (corresponding to traffic 4), GHF (corresponding to traffic 5), ABF (corresponding to traffic 2), DHI (corresponding to traffic 3), and AEHI (corresponding to traffic 1). It should be noted that the system to be analyzed has a large number of overlay links, and each overlay link is determined through steps S110 to S130.

[0091] For example, the server filters out associated data with the same request identifier from the associated data, obtaining associated data for the same request identifier; then, based on the branch identifiers in the associated data for the same request identifier, it determines the branch identifiers associated with the same request identifier; finally, according to the order of branch flow, the branch identifiers associated with the same request identifier are superimposed to obtain the superimposed link, which serves as the superimposed link for the system to be analyzed. Furthermore, the server can utilize the superimposed link of the system to be analyzed to perform test case supplementation, traffic replay identification, monitoring fluctuation alerts, data benefit branching, etc., for details. Figure 2 .

[0092] For example, see reference. Figure 2 Traffic 1 is associated with code branches A, E, H, and I. By superimposing these code branches, an overlay link AEHI can be obtained in the system. Traffic 2 is associated with code branches A, B, and F. By superimposing these code branches, an overlay link ABF can be obtained in the system.

[0093] In the above data processing method, a corresponding request identifier is added to the received data processing request for the system to be analyzed, resulting in an updated data processing request. Next, as the updated data processing request flows through the request branches of the system to be analyzed, the associated data of the request branches is recorded. This associated data characterizes the relationship between the request identifier and the branch identifiers of the request branches. Finally, based on the associated data, the branch identifiers associated with the same request identifier are obtained, and based on these branch identifiers, the overlay links of the system to be analyzed are determined. In this way, by using the request identifier corresponding to the updated data processing request for the system to be analyzed, the branch identifiers of the request branches through which the updated data processing request flows are chained together, the branch identifiers associated with the same request identifier can be quickly obtained. This allows for the rapid determination of the overlay links of the system to be analyzed, achieving the goal of automatically determining the overlay links of the system to be analyzed. The entire process eliminates the need for manual link analysis, thus simplifying the link analysis process, reducing link analysis time, and improving link analysis efficiency.

[0094] In one exemplary embodiment, such as Figure 3 As shown, step S120 above, before recording the associated data of the requested branch, also includes a step of obtaining the associated data of the requested branch, which can be implemented through the following steps:

[0095] In step S310, when the updated data processing request flows through the request branch of the system to be analyzed, the preset data recording instruction of the request branch is invoked to obtain the request identifier of the updated data processing request and the branch identifier of the request branch.

[0096] In step S320, the association between the request identifier and the branch identifier is generated.

[0097] In step S330, the associated data of the requested branch is obtained based on the association relationship.

[0098] Among them, the preset data recording instruction refers to the preset instruction used to obtain the request identifier and the branch identifier of the updated data processing request, and to record the association between the request identifier and the branch identifier. This is specifically implemented through event tracking. It should be noted that each request branch has a corresponding preset data recording instruction, such as... Figure 2 The examples shown are TraceRecorder.trace (“Cross-border Goods”), TraceRecorder.trace (“Gift Promotion”), TraceRecorder.trace (“Use Now, Pay Later”), and TraceRecorder.trace (“Half-Screen Order”).

[0099] For example, when the updated data processing request flows through the request branch of the system to be analyzed, the server calls the preset data recording instruction of the request branch to obtain the request identifier of the updated data processing request and the branch identifier of the request branch; then, based on the request identifier of the updated data processing request and the branch identifier of the request branch, the association relationship between the request identifier and the branch identifier is generated, and finally the association relationship is used as the association data of the request branch.

[0100] The technical solution provided in this disclosure, when the updated data processing request flows through the request branch of the system to be analyzed, calls the preset data recording instruction of the request branch to generate the association relationship between the request identifier and the branch identifier. This facilitates the subsequent use of the association relationship between the request identifier and the branch identifier to link the branch identifiers of the request branches through which the updated data processing request flows, which is beneficial for quickly obtaining the branch identifiers associated with the same request identifier, thereby improving the efficiency of link sorting.

[0101] In an exemplary embodiment, before adding a corresponding request identifier to the received data processing request for the system to be analyzed and obtaining the updated data processing request in step S110, the following steps are also included: generating a preset data recording instruction for the request branch; and adding the preset data recording instruction to the branch instruction set of the request branch.

[0102] In step S310 above, the preset data recording instruction of the request branch is invoked to obtain the request identifier of the updated data processing request and the branch identifier of the request branch. Specifically, this includes the following: invoking the preset data recording instruction in the branch instruction set of the request branch to obtain the request identifier of the updated data processing request and the branch identifier of the request branch.

[0103] The branch instruction set for the requested branch includes the execution instructions for the requested branch, specifically the execution code for the requested branch, such as... Figure 2 If(itemSkuInfo.isCrossBorder()), if(productInfo.hasGiftProduct()), if(isPayAfterUse(context)), if(halfDynamic(context)), etc. are shown.

[0104] Adding the preset data recording instruction to the branch instruction set of the request branch refers to adding the preset data recording instruction to the corresponding position (such as the end) in the branch instruction set of the request branch, for example... Figure 2The code snippets shown are: if(itemSkuInfo.isCrossBorder()){TraceRecorder.trace("Cross-border goods")}, if(productInfo.hasGiftProduct()){TraceRecorder.trace("Gift promotion")}, if(isPayAfterUse(context)){TraceRecorder.trace("Use first, pay later")}, and if(halfDynamic(context)){TraceRecorder.trace("Half-screen order")}.

[0105] It should be noted that the execution process of the branch instruction set requesting a branch does not affect the execution process of the preset data record instruction.

[0106] For example, the server generates a preset data recording instruction for the request branch of the system to be analyzed. Then, it determines the insertion position of the preset data recording instruction in the branch instruction set of the request branch and adds it to the corresponding position in the branch instruction set of the request branch. Next, when the updated data processing request flows through the request branch of the system to be analyzed, the server calls the preset data recording instruction in the branch instruction set of the request branch to obtain the request identifier of the updated data processing request and the branch identifier of the request branch. Then, it generates an association between the request identifier and the branch identifier, obtains the associated data of the request branch based on the association, and finally records the associated data of the request branch.

[0107] The technical solution provided in this disclosure obtains the request identifier of the updated data processing request and the branch identifier of the request branch by calling a preset data record instruction in the branch instruction set of the request branch. This facilitates the subsequent generation of the association relationship between the request identifier and the branch identifier, and the connection of the branch identifiers of the request branches through which the updated data processing request flows. This is beneficial for quickly obtaining the branch identifiers associated with the same request identifier, and eliminates the need for manual link sorting, thus improving the efficiency of link sorting. Moreover, it avoids errors or omissions that may occur when manually sorting the link, thus improving the accuracy of link sorting.

[0108] In an exemplary embodiment, before obtaining the branch identifier associated with the same request identifier based on the associated data, step S130 above further includes the following: storing the associated data in a message queue.

[0109] Step S130 above, based on the associated data, obtains the branch identifier associated with the same request identifier, specifically including the following: extracting the associated data from the message queue; and obtaining the branch identifier associated with the same request identifier based on the extracted associated data.

[0110] Among them, message queues are used to temporarily store related data, such as Kafka message queues.

[0111] For example, the server stores associated data in a message queue, then retrieves the associated data from the message queue; next, it filters out associated data with the same request identifier from the retrieved associated data, obtaining associated data with the same request identifier; finally, based on the branch identifiers in the associated data with the same request identifier, it determines the branch identifier associated with the same request identifier, for example... Figure 2 Traffic 1 is associated with code branches A, E, H, and I.

[0112] The technical solution provided in this disclosure first stores the associated data in a message queue, then extracts the associated data from the message queue, and obtains the branch identifier associated with the same request identifier based on the extracted associated data. This avoids the drawback of sending all associated data to the server for processing at once, which would result in a large processing load on the server and helps to reduce the processing load on the server.

[0113] In one exemplary embodiment, such as Figure 4 As shown, based on the extracted association data, the branch identifiers associated with the same request identifier can be obtained, which can be achieved through the following steps:

[0114] In step S410, the associated data is classified according to the request identifier in the extracted associated data to obtain associated data with the same request identifier.

[0115] In step S420, the branch identifier associated with the same request identifier is obtained based on the branch identifier in the associated data of the same request identifier.

[0116] Among them, associated data for the same request identifier refers to multiple associated data corresponding to the same request identifier.

[0117] For example, the server categorizes the associated data according to the request identifiers in the extracted associated data, obtaining associated data with the same request identifier, which is then used as associated data for the same request identifier; next, it obtains the branch identifiers from each associated data for the same request identifier, thus obtaining all branch identifiers associated with the same request identifier. For example, refer to... Figure 2For traffic 1, the corresponding associated data includes traffic 1-code branch A, traffic 1-code branch E, traffic 1-code branch H, and traffic 1-code branch I. Therefore, the branch identifiers associated with traffic 1 are code branch A, code branch E, code branch H, and code branch I.

[0118] The technical solution provided in this disclosure first classifies the associated data according to the request identifier in the extracted associated data to obtain associated data with the same request identifier. Then, based on the branch identifier in the associated data with the same request identifier, the branch identifier associated with the same request identifier is obtained. This achieves the purpose of linking the branch identifiers associated with the same request identifier together, which is conducive to quickly obtaining the actual superimposed link of the system to be analyzed and further improves the efficiency of link sorting.

[0119] In one exemplary embodiment, the associated data also includes a timestamp corresponding to the branch identifier, the timestamp being used to characterize the timestamp of the execution request branch.

[0120] like Figure 5 As shown, step S130 above, based on the branch identifiers associated with the same request identifier, determines the overlay link of the system to be analyzed. This can be achieved through the following steps:

[0121] In step S510, the order of timestamps corresponding to branch identifiers associated with the same request identifier is determined.

[0122] In step S520, the stacking order of branch identifiers associated with the same request identifier is determined based on the sequence of events.

[0123] In step S530, branch identifiers associated with the same request identifier are superimposed according to the superposition order to obtain the superimposed link of the system to be analyzed.

[0124] The timestamp corresponding to the branch identifier of the request branch refers to the timestamp when the request branch was executed, such as 17:15:11, which is used to distinguish the flow order of the request branches.

[0125] The stacking order of branch identifiers associated with the same request identifier refers to the chronological order of the timestamps corresponding to the branch identifiers associated with the same request identifier.

[0126] For example, the server obtains the timestamps corresponding to the branch identifiers associated with the same request identifier from the associated data of the request branches associated with the same request identifier. Then, it determines the order of the timestamps corresponding to the branch identifiers associated with the same request identifier as the superposition order of the branch identifiers associated with the same request identifier. Finally, it superimposes the branch identifiers associated with the same request identifier according to the superposition order of the branch identifiers associated with the same request identifier to obtain the superimposed link of the system to be analyzed.

[0127] For example, see reference. Figure 2 Traffic 1 is associated with code branches A, E, H, and I, and the stacking order is: code branch A, code branch E, code branch H, and code branch I. Therefore, the stacking link is AEHI. Traffic 2 is associated with code branches A, B, and F, and the stacking order is: code branch A, code branch B, and code branch F. Therefore, the stacking link is ABF.

[0128] The technical solution provided in this disclosure determines the chronological order of the timestamps corresponding to the branch identifiers associated with the same request identifier. Then, based on the chronological order, it determines the superposition order of the branch identifiers associated with the same request identifier. Finally, it superimposes the branch identifiers associated with the same request identifier according to the superposition order to obtain the superimposed link of the system to be analyzed. This achieves the purpose of superimposing the branch identifiers associated with the same request identifier according to the superposition order, ensuring that the obtained superimposed link is more accurate and further improving the accuracy of link sorting.

[0129] In an exemplary embodiment, after determining the overlay link of the system to be analyzed based on the branch identifier associated with the same request identifier, step S130 further includes the following: verifying the overlay link of the system to be analyzed and obtaining a verification result; if the verification result indicates that the overlay link of the system to be analyzed has passed the verification, sending the overlay link of the system to be analyzed to the target terminal; the target terminal includes at least one of a monitoring terminal and a test terminal.

[0130] In this context, "passing the verification of the overlay link of the system to be analyzed" means that there are no mutually exclusive request branches in the overlay link of the system to be analyzed.

[0131] Among them, the monitoring terminal is used to overlay links based on the system to be analyzed to realize monitoring fluctuation early warning, data benefit analysis, etc.

[0132] Among them, the test terminal is used to supplement test cases and identify traffic replay based on the superimposed links of the system to be analyzed.

[0133] For example, the server obtains a verification instruction to verify whether there are mutually exclusive request branches in the overlay links of the system to be analyzed. Then, it uses the verification instruction to verify the overlay links of the system to be analyzed and obtains the verification result of the overlay links of the system to be analyzed. Finally, the verification result is identified. If the verification result indicates that there are no mutually exclusive request branches in the overlay links of the system to be analyzed, the verification of the overlay links of the system to be analyzed is confirmed to be successful, and the overlay links of the system to be analyzed are sent to the target terminal. If the target terminal is a monitoring terminal, monitoring fluctuation warnings and data benefit analysis are performed through the monitoring terminal. If the target terminal is a testing terminal, test case supplementation and traffic replay identification are performed through the testing terminal.

[0134] The technical solution provided in this disclosure verifies the overlay link of the system to be analyzed and obtains the verification result. If the verification result indicates that the overlay link of the system to be analyzed has passed the verification, the overlay link of the system to be analyzed is then sent to the target terminal. In this way, by verifying the overlay link of the system to be analyzed, the accuracy of link sorting can be further improved, while ensuring the accuracy of the overlay link sent to the target terminal, thereby improving the monitoring accuracy and testing accuracy of the target terminal.

[0135] Figure 6 This is a flowchart illustrating another data processing method according to an exemplary embodiment, such as... Figure 6 As shown, this data processing method is used in a server and can be implemented through the following steps:

[0136] In step S610, a preset data recording instruction for the request branch of the system to be analyzed is generated; the preset data recording instruction is added to the branch instruction set of the request branch.

[0137] In step S620, a corresponding request identifier is added to the received data processing request for the system to be analyzed, and the updated data processing request is obtained.

[0138] In step S630, when the updated data processing request flows through the request branch of the system to be analyzed, the preset data recording instruction in the branch instruction set of the request branch is called to obtain the request identifier of the updated data processing request and the branch identifier of the request branch.

[0139] In step S640, the association between the request identifier and the branch identifier is generated; based on the association, the associated data of the request branch is obtained; and the associated data is stored in the message queue.

[0140] In step S650, associated data is extracted from the message queue; the associated data is classified according to the request identifier in the extracted associated data to obtain associated data with the same request identifier.

[0141] In step S660, the branch identifier associated with the same request identifier is obtained based on the branch identifier in the associated data of the same request identifier.

[0142] In step S670, the order of timestamps corresponding to branch identifiers associated with the same request identifier is determined; based on the order, the stacking order of branch identifiers associated with the same request identifier is determined.

[0143] In step S680, branch identifiers associated with the same request identifier are superimposed according to the superposition order to obtain the superimposed link of the system to be analyzed.

[0144] In step S690, the overlay link of the system to be analyzed is verified to obtain the verification result; if the verification result indicates that the overlay link of the system to be analyzed has passed the verification, the overlay link of the system to be analyzed is sent to the target terminal; the target terminal includes at least one of a monitoring terminal and a test terminal.

[0145] In the above data processing method, by using the request identifier corresponding to the updated data processing request for the system to be analyzed, the branch identifiers of the request branches through which the updated data processing request flows can be chained together. This allows for the rapid determination of the superimposed links of the system to be analyzed, achieving the goal of automatically determining the superimposed links of the system to be analyzed. The entire process does not require manual link sorting, thus simplifying the link sorting process, reducing link sorting time, and improving link sorting efficiency.

[0146] To more clearly illustrate the data processing method provided in the embodiments of this disclosure, the following describes the data processing method in detail with reference to a specific embodiment. In one embodiment, as... Figure 2 As shown, this disclosure also provides a new link analysis method, namely, a superimposed link analysis method for complex systems—traffic coloring. This method transforms traditional empiricism into factualism, solving the problem of difficult link analysis in complex multi-scenario superimposed systems, thereby improving the efficiency of system quality assurance.

[0147] The online code flow is a black box; it's impossible to track which if-else branches are executed. This publication uses a combination of instrumentation and log reporting (see reference). Figure 2 This process involves explicitly identifying and marking the customized code for each requirement, thus white-boxing the code execution flow. All if-else custom logic passed through each request can be traced and linked together, allowing for the aggregation of all overlapping links online. This process is called traffic coloring. With this coloring result, many things can be done, such as supplementing test cases, monitoring and alerting for specific overlapping scenarios, and performing data benefit analysis.

[0148] At the code level, TraceRecorder.trace is used to display and annotate code snippets in customized if-else statements for business scenarios. When a request ends, logs of all hit traces are printed, and these are converted into reports via log2kafka, thus obtaining the full online real-world combined and overlaid traces. Regarding the granularity of coloring tracing, this publicly available specification is based on the granularity of requirements. Requirements are the basic units of code, tests, and functions, and each requirement, once deployed, must have visual tracing capabilities to understand its online traffic requests and stability.

[0149] The technical solutions provided in this disclosure can achieve the following technical effects:

[0150] (1) Improved efficiency of link sorting: Experiments have shown that traffic coloring can improve efficiency by more than 80% compared with traditional sorting schemes.

[0151] (2) Improved accuracy of link sorting: Experiments have shown that traffic coloring improves accuracy by more than 60% compared with traditional sorting schemes, effectively avoiding problems such as omissions and errors in sorting based on experience.

[0152] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0153] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.

[0154] Based on the same inventive concept, this disclosure also provides a data processing apparatus for implementing the data processing method described above.

[0155] Figure 7 This is a block diagram illustrating a data processing apparatus according to an exemplary embodiment. (Refer to...) Figure 7 The device includes an identification adding unit 710, a data recording unit 720, and a link determination unit 730.

[0156] The identifier adding unit 710 is configured to add a corresponding request identifier to the received data processing request for the system to be analyzed, thereby obtaining the updated data processing request.

[0157] The data recording unit 720 is configured to record the associated data of the request branch when the updated data processing request flows through the request branch of the system to be analyzed; the associated data is used to characterize the relationship between the request identifier and the branch identifier of the request branch.

[0158] The link determination unit 730 is configured to perform operations based on associated data to obtain branch identifiers associated with the same request identifier, and to determine the superimposed links of the system to be analyzed based on the branch identifiers associated with the same request identifier.

[0159] In one exemplary embodiment, the data processing apparatus further includes a data generation unit configured to, when an updated data processing request flows through a request branch of the system to be analyzed, invoke a preset data recording instruction of the request branch to obtain the request identifier of the updated data processing request and the branch identifier of the request branch; generate an association relationship between the request identifier and the branch identifier; and obtain associated data of the request branch based on the association relationship.

[0160] In one exemplary embodiment, the data processing apparatus further includes an instruction adding unit configured to execute a preset data recording instruction for generating a request branch; and to add the preset data recording instruction to the branch instruction set of the request branch.

[0161] The data generation unit is also configured to execute preset data record instructions in the branch instruction set of the request branch, obtain the request identifier of the updated data processing request, and the branch identifier of the request branch.

[0162] In one exemplary embodiment, the data processing apparatus further includes a data storage unit configured to store associated data in a message queue.

[0163] The link determination unit 730 is also configured to extract associated data from the message queue and, based on the extracted associated data, obtain the branch identifier associated with the same request identifier.

[0164] In one exemplary embodiment, the link determination unit 730 is further configured to perform classification processing on the associated data according to the request identifier in the extracted associated data to obtain associated data with the same request identifier; and to obtain the branch identifier associated with the same request identifier based on the branch identifier in the associated data with the same request identifier.

[0165] In one exemplary embodiment, the associated data also includes a timestamp corresponding to the branch identifier, the timestamp being used to characterize the timestamp of the execution request branch.

[0166] The link determination unit 730 is also configured to determine the order of timestamps corresponding to branch identifiers associated with the same request identifier; determine the superposition order of branch identifiers associated with the same request identifier based on the order of order; and superimpose the branch identifiers associated with the same request identifier according to the superposition order to obtain the superimposed link of the system to be analyzed.

[0167] In one exemplary embodiment, the data processing apparatus further includes a link transmission unit configured to perform verification of the overlay link of the system to be analyzed and obtain a verification result; if the verification result indicates that the overlay link of the system to be analyzed has passed verification, the overlay link of the system to be analyzed is transmitted to a target terminal; the target terminal includes at least one of a monitoring terminal and a test terminal.

[0168] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0169] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0170] Figure 8 This is a block diagram illustrating an electronic device 800 for implementing a data processing method according to an exemplary embodiment. For example, the electronic device 800 may be a server. (Refer to...) Figure 8 The electronic device 800 includes a processing component 820, which further includes one or more processors, and memory resources represented by memory 822 for storing instructions, such as application programs, that can be executed by the processing component 820. The application programs stored in memory 822 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 820 is configured to execute instructions to perform the methods described above.

[0171] Electronic device 800 may further include: a power supply component 824 configured to perform power management of electronic device 800, a wired or wireless network interface 826 configured to connect electronic device 800 to a network, and an input / output (I / O) interface 828. Electronic device 800 may operate on an operating system stored in memory 822, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or similar.

[0172] In one exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 822 including instructions, which can be executed by a processor of an electronic device 800 to perform the above-described method. The storage medium can be a computer-readable storage medium, for example, a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0173] In one exemplary embodiment, a computer program product is also provided, the computer program product including instructions that can be executed by a processor of an electronic device 800 to perform the above-described method.

[0174] It should be noted that the above-mentioned apparatus, electronic equipment, computer-readable storage medium, computer program product, etc., may also include other implementation methods according to the description of the method embodiments. For specific implementation methods, please refer to the description of the relevant method embodiments, which will not be elaborated here.

[0175] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0176] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A data processing method, characterized in that, include: Add a corresponding request identifier to the received data processing request for the system to be analyzed to obtain the updated data processing request; When the updated data processing request flows through the request branch of the system to be analyzed, the associated data of the request branch is recorded; The associated data is used to characterize the association relationship between the request identifier and the branch identifier of the request branch; Based on the associated data, branch identifiers associated with the same request identifier are obtained, and based on the branch identifiers associated with the same request identifier, the overlay links of the system to be analyzed are determined.

2. The method according to claim 1, characterized in that, Before recording the associated data for the requested branch, the process also includes: When the updated data processing request flows through the request branch of the system to be analyzed, the preset data recording instruction of the request branch is invoked to obtain the request identifier of the updated data processing request and the branch identifier of the request branch. Generate the association between the request identifier and the branch identifier; Based on the aforementioned relationship, the associated data of the requested branch is obtained.

3. The method according to claim 2, characterized in that, Before adding the corresponding request identifier to the received data processing request for the system to be analyzed, and obtaining the updated data processing request, the process also includes: Generate a preset data record instruction for the requested branch; Add the preset data recording instruction to the branch instruction set of the requested branch; The step of invoking the preset data record instruction of the request branch to obtain the request identifier of the updated data processing request and the branch identifier of the request branch includes: Invoke the preset data record instruction in the branch instruction set of the request branch to obtain the request identifier of the updated data processing request and the branch identifier of the request branch.

4. The method according to claim 1, characterized in that, Before obtaining the branch identifier associated with the same request identifier based on the associated data, the process also includes: The associated data is stored in a message queue; The step of obtaining branch identifiers associated with the same request identifier based on the associated data includes: Extract the associated data from the message queue; Based on the extracted associated data, the branch identifiers associated with the same request identifier are obtained.

5. The method according to claim 4, characterized in that, The step of obtaining branch identifiers associated with the same request identifier based on the extracted association data includes: Based on the request identifier in the extracted associated data, the associated data is classified to obtain associated data with the same request identifier; Based on the branch identifier in the associated data of the same request identifier, the branch identifier associated with the same request identifier is obtained.

6. The method according to claim 1, characterized in that, The associated data also includes the timestamp corresponding to the branch identifier, which is used to represent the timestamp when the requested branch was executed. The step of determining the overlay link of the system to be analyzed based on the branch identifier associated with the same request identifier includes: Determine the order of the timestamps corresponding to the branch identifiers associated with the same request identifier; Based on the aforementioned sequence, the stacking order of the branch identifiers associated with the same request identifier is determined; According to the superposition order, the branch identifiers associated with the same request identifier are superimposed to obtain the superposition link of the system to be analyzed.

7. The method according to any one of claims 1 to 6, characterized in that, After determining the overlay link of the system to be analyzed based on the branch identifier associated with the same request identifier, the method further includes: The superimposed links of the system to be analyzed were verified, and the verification results were obtained. If the verification result indicates that the superimposed link of the system to be analyzed has passed the verification, the superimposed link of the system to be analyzed is sent to the target terminal; the target terminal includes at least one of a monitoring terminal and a test terminal.

8. A data processing apparatus, characterized in that, include: The identifier adding unit is configured to add a corresponding request identifier to the received data processing request for the system to be analyzed, thereby obtaining the updated data processing request; The data recording unit is configured to record the associated data of the request branch when the updated data processing request flows through the request branch of the system to be analyzed; The associated data is used to characterize the association relationship between the request identifier and the branch identifier of the request branch; The link determination unit is configured to perform operations based on the associated data to obtain branch identifiers associated with the same request identifier, and to determine the overlay links of the system to be analyzed based on the branch identifiers associated with the same request identifier.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the data processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the data processing method as described in any one of claims 1 to 7.

11. A computer program product, the computer program product comprising instructions, characterized in that, When the instruction is executed by the processor of the electronic device, the electronic device is able to perform the data processing method as described in any one of claims 1 to 7.

Images