Traffic recording and playback method and device

Abstract

The invention provides a traffic recording and playback method and device, and relates to the field of data processing, in particular to the technical field of computers. According to the specific implementation scheme, traffic recording is carried out on a target process to obtain recorded data, a vector database is established according to the recorded data, and the vector database at least comprises a first vector corresponding to a first sub-call parameter in the recorded data and a first sub-call parameter return value; determining to-be-played-back data in the recorded data, and initiating playback of the to-be-played-back data in the test environment; in response to a playback operation, monitored in the playback process, of a second sub-call parameter included in the to-be-played-back data, performing matching from the vector database to obtain a second sub-call parameter return value corresponding to the second sub-call parameter, and performing a subsequent playback step based on the second sub-call parameter return value; and in response to monitoring that the playback of the to-be-played-back data is completed, obtaining a playback result of the to-be-played-back data.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, in particular to the field of computer technology, and specifically to a method and device for traffic recording and playback. Background Art

[0002] In the process of modern software development and operation and maintenance, the recording and playback technology of online traffic has become a key means to improve system stability, optimize performance, and accelerate problem solving. As the complexity of software systems continues to increase, traditional testing methods often fail to fully cover real user scenarios, thus affecting system reliability and user experience. Therefore, how to effectively use real online traffic data for testing and playback has become an important direction for current technological development. Summary of the invention

[0003] The present invention provides a method, device, equipment and storage medium for traffic recording and playback.

[0004] According to one aspect of the present disclosure, a method for traffic recording and playback is provided, wherein recorded data is obtained by performing traffic recording on a target process, and a vector database is established based on the recorded data, wherein the vector database at least includes a first vector and a first sub-call parameter return value corresponding to a first sub-call parameter in the recorded data; data to be played back in the recorded data is determined, and playback is initiated under a test environment for the data to be played back; in response to monitoring a playback operation of a second sub-call parameter included in the data to be played back during the playback process, a second sub-call parameter return value corresponding to the second sub-call parameter is matched from the vector database, and subsequent playback steps are performed based on the second sub-call parameter return value; in response to monitoring that playback of the data to be played back is completed, a playback result of the data to be played back is obtained.

[0005] The present application can capture user requests and responses occurring in an actual production environment by recording the traffic of the target process; the vector database established by the present application stores the first vector and the first sub-call parameter return value corresponding to the first sub-call parameter in the recorded data, providing data support for subsequent playback; on the other hand, converting the first sub-call parameter into the first vector can significantly improve the matching speed during playback, because the vector usually has a fixed dimension and format, especially when frequent queries are required.

[0006] According to one embodiment of the present application, a vector database is established based on recorded data, including: obtaining sub-call related information included in each piece of recorded data, the sub-call related information including a first sub-call parameter and a first sub-call parameter return value corresponding to the first sub-call parameter; performing feature extraction on the first sub-call parameter to obtain a first vector corresponding to the first sub-call parameter; binding the first vector and the first sub-call parameter return value to generate vector related information, and storing the vector related information in the vector database.

[0007] According to one embodiment of the present application, a second sub-call parameter return value corresponding to a second sub-call parameter is matched from a vector database, including: performing feature extraction on the second sub-call parameter to obtain a second vector corresponding to the second sub-call parameter; querying the vector database based on the second vector to obtain a target first vector corresponding to the second vector; obtaining a target first sub-call parameter return value bound to the target first vector; and using the target first sub-call parameter return value as the second sub-call parameter return value corresponding to the second sub-call parameter.

[0008] According to one embodiment of the present application, a vector database is queried based on a second vector to obtain a target first vector corresponding to the second vector, including: selecting multiple candidate first vectors related to the second vector from the vector database; calculating the vector similarity and / or vector distance between the second vector and each candidate first vector respectively; and determining the target first vector from the multiple candidate first vectors according to the vector similarity and / or vector distance.

[0009] According to one embodiment of the present application, multiple candidate first vectors related to the second vector are selected from a vector database, including: obtaining a filtering parameter corresponding to the second sub-call parameter, the filtering parameter including at least one of a user identifier and a sub-call interface name; and selecting a first vector that meets the filtering parameter from the vector database as a candidate first vector.

[0010] According to one embodiment of the present application, traffic recording is performed on a target process to obtain recording data, including: receiving recording configuration parameters issued by a configuration center, and waking up a recording service corresponding to the target process; based on the recording service, calling a recording plug-in according to the recording configuration parameters to perform traffic recording on the target process to obtain recording data; wherein each recording data corresponds to a unique recording identifier, and each recording data includes at least a first entry parameter, a first entry parameter return value, a first sub-call parameter, and a first sub-call parameter return value.

[0011] According to one embodiment of the present application, based on a recording service, a recording plug-in is called according to recording configuration parameters to record the traffic of a target process to obtain recording data, including: obtaining the plug-in configuration parameters included in the recording configuration parameters, and performing logical enhancement on the recording plug-in included in the recording service according to the plug-in configuration parameters; obtaining the recording constraint parameters included in the recording configuration parameters, and determining the traffic to be recorded corresponding to the target process according to the recording constraint parameters; based on the recording service, calling the recording plug-in to record the traffic to be recorded to obtain recording data.

[0012] According to one embodiment of the present application, the traffic to be recorded corresponding to the target process is determined according to the recording constraint parameters, including: determining the traffic filtering dimension and the recording sampling ratio corresponding to the target process according to the recording constraint parameters, wherein the traffic filtering dimension is at least one of the user identification dimension, the domain name dimension or the process interface dimension; determining the traffic to be recorded of the target process according to the traffic filtering dimension and the recording sampling ratio.

[0013] According to an embodiment of the present application, data to be replayed in recorded data is determined, and playback is initiated for the data to be replayed under a test environment, including: receiving playback configuration parameters issued by a configuration center, waking up a playback service corresponding to a target process under a test environment; obtaining a target recording identifier carried by the playback configuration parameters, and determining the data to be replayed from the recorded data according to the target recording identifier; and initiating playback of the data to be replayed by calling a playback plug-in based on the playback service.

[0014] According to an embodiment of the present application, the traffic recording and playback method further includes: generating a difference analysis report based on the recording data and the playback results.

[0015] According to another aspect of the present disclosure, a device for traffic recording and playback is provided, including: a traffic recording module, used to perform traffic recording on a target process to obtain recording data, and establish a vector database based on the recording data, wherein the vector database at least includes a first vector and a first sub-call parameter return value corresponding to a first sub-call parameter in the recording data; a first playback module, used to determine the data to be played back in the recording data, and initiate playback of the data to be played back under a test environment; a second playback module, used to respond to monitoring during the playback process of a playback operation of a second sub-call parameter included in the data to be played back, match the second sub-call parameter return value corresponding to the second sub-call parameter from the vector database, and perform subsequent playback steps based on the second sub-call parameter return value; a third playback module, used to respond to monitoring that the playback of the data to be played back is complete, and obtain the playback result of the data to be played back.

[0016] According to one embodiment of the present application, the traffic recording module is also used to: obtain sub-call related information included in each recording data, the sub-call related information including a first sub-call parameter and a first sub-call parameter return value corresponding to the first sub-call parameter; perform feature extraction on the first sub-call parameter to obtain a first vector corresponding to the first sub-call parameter; bind the first vector and the first sub-call parameter return value to generate vector related information, and store the vector related information in a vector database.

[0017] According to one embodiment of the present application, the second playback module is also used to: perform feature extraction on the second sub-call parameter to obtain a second vector corresponding to the second sub-call parameter; query the vector database based on the second vector to obtain a target first vector corresponding to the second vector; obtain a target first sub-call parameter return value bound to the target first vector; and use the target first sub-call parameter return value as the second sub-call parameter return value corresponding to the second sub-call parameter.

[0018] According to one embodiment of the present application, the second playback module is also used to: select multiple candidate first vectors related to the second vector from a vector database; calculate the vector similarity and / or vector distance between the second vector and each candidate first vector respectively; and determine the target first vector from the multiple candidate first vectors based on the vector similarity and / or vector distance.

[0019] According to one embodiment of the present application, the second playback module is further used to: obtain a filtering parameter corresponding to the second sub-call parameter, the filtering parameter including at least one of a user identifier and a sub-call interface name; and select a first vector that meets the filtering parameter from a vector database as a candidate first vector.

[0020] According to one embodiment of the present application, the traffic recording module is also used to: receive the recording configuration parameters issued by the configuration center, and wake up the recording service corresponding to the target process; based on the recording service, call the recording plug-in according to the recording configuration parameters to perform traffic recording on the target process to obtain recording data; wherein each recording data corresponds to a unique recording identifier, and each recording data includes at least a first entry parameter, a first entry parameter return value, a first sub-call parameter, and a first sub-call parameter return value.

[0021] According to one embodiment of the present application, the traffic recording module is also used to: obtain the plug-in configuration parameters included in the recording configuration parameters, and perform logical enhancement on the recording plug-in included in the recording service according to the plug-in configuration parameters; obtain the recording constraint parameters included in the recording configuration parameters, and determine the traffic to be recorded corresponding to the target process according to the recording constraint parameters; and call the recording plug-in based on the recording service to record the traffic to be recorded to obtain recording data.

[0022] According to one embodiment of the present application, the traffic recording module is also used to: determine the traffic filtering dimension and recording sampling ratio corresponding to the target process according to the recording constraint parameters, wherein the traffic filtering dimension is at least one of the user identification dimension, the domain name dimension or the process interface dimension; determine the traffic to be recorded of the target process according to the traffic filtering dimension and the recording sampling ratio.

[0023] According to one embodiment of the present application, the first playback module is also used to: receive the playback configuration parameters issued by the configuration center, wake up the corresponding playback service of the target process in the test environment; obtain the target recording identifier carried by the playback configuration parameters, and determine the data to be played back from the recorded data according to the target recording identifier; and initiate playback of the data to be played back based on the playback service by calling the playback plug-in.

[0024] According to an embodiment of the present application, the traffic recording and playback device further includes: an analysis module for generating a difference analysis report based on the recording data and the playback results.

[0025] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned method of traffic recording and playback.

[0026] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned traffic recording and playback method.

[0027] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the above-mentioned traffic recording and playback method when executed by a processor.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0030] Figure 1 It is a schematic diagram of an exemplary implementation of a method for traffic recording and playback according to an exemplary embodiment of the present disclosure.

[0031] Figure 2It is a schematic diagram of an exemplary implementation of a method for traffic recording and playback according to an exemplary embodiment of the present disclosure.

[0032] Figure 3 It is a schematic diagram of an exemplary implementation of a method for traffic recording and playback according to an exemplary embodiment of the present disclosure.

[0033] Figure 4 It is a framework diagram of a method for traffic recording and playback according to an exemplary embodiment of the present disclosure.

[0034] Figure 5 It is a schematic diagram of a device for traffic recording and playback according to an exemplary embodiment of the present disclosure.

[0035] Figure 6 is a schematic diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0037] Traffic recording refers to capturing data of actual user requests in a production environment, including but not limited to HTTP / HTTPS requests, database queries, message queue messages, etc. This data usually contains various input data during actual runtime and can be used to reproduce user behavior in a production environment.

[0038] Traffic replay is to replay previously recorded traffic data in a test environment to simulate user behavior in a production environment. By replaying real online traffic, problems in the production environment can be reproduced in a controlled test environment, helping developers and operation and maintenance personnel to quickly locate and solve problems.

[0039] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0040] Figure 1 is a schematic diagram of an exemplary implementation of a method for traffic recording and playback shown in the present application, such as Figure 1 As shown, the method for traffic recording and playback includes the following steps:

[0041] S101, recording the traffic of the target process to obtain recording data, and establishing a vector database according to the recording data, wherein the vector database at least includes a first vector corresponding to a first sub-call parameter in the recording data and a return value of the first sub-call parameter.

[0042] Among them, traffic recording is performed on the target process, mainly recording traffic data in the production environment, especially traffic data or abnormal data during peak hours, so as to facilitate subsequent playback in the test environment. Problems in the production environment can be reproduced in a controlled test environment, helping developers and operation and maintenance personnel to quickly locate and solve problems.

[0043] Each recording data may include an input parameter, an input parameter return value, at least one sub-call parameter, and a sub-call parameter return value corresponding to each sub-call parameter. In addition, the recording data may also include data such as a user ID and an interface name corresponding to the recording data.

[0044] In order to distinguish them from the parameters and return values ​​in the subsequent playback data, in this application, the input parameters, input parameter return values, at least one sub-call parameter and the sub-call parameter return value corresponding to each sub-call parameter included in the recorded data are respectively referred to as: the first input parameter, the first input parameter return value, at least one first sub-call parameter and the first sub-call parameter return value corresponding to each first sub-call parameter.

[0045] In the present application, after obtaining the first sub-call parameter, it is necessary to perform feature extraction on the first sub-call parameter to obtain the first vector corresponding to the first sub-call parameter, and bind the first vector corresponding to the first sub-call parameter with the first sub-call parameter return value corresponding to the first sub-call parameter to generate 1 vector-related information, thereby establishing a vector database.

[0046] It is not difficult to understand that the vector database at least includes the first vector and the first sub-call parameter return value corresponding to the first sub-call parameter in the recorded data. In addition, each vector-related information in the vector database may also include the first sub-call parameter corresponding to the vector-related information, the user identifier corresponding to the recorded data where the first sub-call parameter corresponding to the vector-related information is located, and the first sub-call parameter interface name of the first sub-call parameter corresponding to the vector-related information.

[0047] S102, determining the data to be replayed in the recorded data, and initiating playback of the data to be replayed in a test environment.

[0048] In the present application, all the obtained recorded data may be used as the data to be played back, or part of the recorded data may be determined from the obtained recorded data as the data to be played back.

[0049] After the data to be replayed is determined, in the test environment corresponding to the target process, the client is simulated to initiate playback based on the entry parameters corresponding to the data to be replayed.

[0050] S103, in response to monitoring the playback operation of the second sub-call parameter included in the playback data during the playback process, matching the second sub-call parameter return value corresponding to the second sub-call parameter from the vector database, and performing subsequent playback steps based on the second sub-call parameter return value.

[0051] During the playback process of the data to be played back, the playback operation of the sub-call parameters in the data to be played back is involved. In this application, the sub-call parameters in the data to be played back are called second sub-call parameters, and the return value corresponding to the second sub-call parameters is called the second sub-call parameter return value.

[0052] In the present application, if a playback operation of a second sub-call parameter included in the data to be played back is monitored during the playback process, the third-party call service of the second sub-call parameter is intercepted, and the second sub-call parameter return value corresponding to the second sub-call parameter is directly matched from the vector database. If the data to be played back includes multiple second sub-call parameters, the playback operation of each second sub-call parameter is performed in sequence based on the above scheme, that is, the second sub-call parameter return value corresponding to each second sub-call parameter is obtained in sequence.

[0053] S104, in response to monitoring that playback of the data to be played back is completed, obtaining playback results of the data to be played back.

[0054] If it is detected that the playback of the data to be played back is completed, the playback result of the data to be played back is obtained.

[0055] The embodiment of the present application provides a method for traffic recording and playback, which obtains recording data by recording the traffic of a target process, and establishes a vector database according to the recording data, wherein the vector database at least includes a first vector and a first sub-call parameter return value corresponding to a first sub-call parameter in the recording data; determines the data to be played back in the recording data, and initiates playback of the data to be played back in a test environment; in response to monitoring the playback operation of the second sub-call parameter included in the data to be played back during the playback process, matches the second sub-call parameter return value corresponding to the second sub-call parameter from the vector database, and performs subsequent playback steps based on the second sub-call parameter return value; in response to monitoring that the playback of the data to be played back is completed, obtains the playback result of the data to be played back. The present application can capture user requests and responses occurring in an actual production environment by recording the traffic of the target process; the vector database established by the present application stores the first vector and the first sub-call parameter return value corresponding to the first sub-call parameter in the recording data, providing data support for subsequent playback; on the other hand, converting the first sub-call parameter into a first vector can significantly improve the matching speed during playback, especially when frequent queries are required, because the vector usually has a fixed dimension and format.

[0056] Figure 2 is a schematic diagram of an exemplary implementation of a method for traffic recording and playback shown in the present application, such as Figure 2 As shown, the method for traffic recording and playback includes the following steps:

[0057] S201, recording the traffic of the target process to obtain recording data.

[0058] S202, obtaining sub-call related information included in each piece of recording data, where the sub-call related information includes a first sub-call parameter and a first sub-call parameter return value corresponding to the first sub-call parameter.

[0059] S203: Perform feature extraction on the first sub-call parameter to obtain a first vector corresponding to the first sub-call parameter.

[0060] The first subcall parameter usually refers to the parameter passed when a function or method is called in a program.

[0061] In this application, the goal of feature extraction is to convert the first subcall parameter into a vector form so that it can represent its important features in the vector space. After the feature extraction is completed, the extracted features are combined into a vector of fixed length.

[0062] S204: Bind the first vector and the first sub-call parameter return value to generate vector related information, and store the vector related information in a vector database.

[0063] The first vector corresponding to the first sub-call parameter is bound to the first sub-call parameter return value corresponding to the first sub-call parameter to generate one piece of vector related information, thereby establishing a vector database.

[0064] S205, determining the data to be replayed in the recorded data, and initiating playback of the data to be replayed in the test environment.

[0065] S206 , in response to monitoring a playback operation on a second sub-call parameter included in the data to be played back during the playback process, performing feature extraction on the second sub-call parameter to obtain a second vector corresponding to the second sub-call parameter.

[0066] In the present application, if a playback operation on a second sub-call parameter included in the playback data is monitored during the playback process, the third-party call service of the second sub-call parameter is intercepted, and feature extraction is performed on the second sub-call parameter to obtain a second vector corresponding to the second sub-call parameter.

[0067] S207: query a vector database based on the second vector to obtain a target first vector corresponding to the second vector.

[0068] First, multiple candidate first vectors related to the second vector are selected from a vector database; vector similarities and / or vector distances between the second vector and each candidate first vector are calculated; and a target first vector is determined from the multiple candidate first vectors according to the vector similarities and / or vector distances.

[0069] Optionally, the candidate first vector with the greatest vector similarity is determined as the target first vector.

[0070] Optionally, the candidate first vector with the smallest vector distance is determined as the target first vector.

[0071] Optionally, in order to reduce calculation operations, in the present application, when determining multiple candidate first vectors related to the second vector, the screening parameters corresponding to the second sub-call parameter can be first obtained, and the screening parameters include at least one of the user identifier and the sub-call interface name; then the first vector that meets the screening parameters is selected from the vector database as the candidate first vector. By selecting the candidate first vector related to the second vector, the number of vectors that need to calculate similarity or distance can be significantly reduced, thereby reducing the calculation complexity and time cost, and ensuring that the target first vector finally selected is more relevant, thereby improving the accuracy of the match.

[0072] S208, obtaining a target first sub-call parameter return value bound to the target first vector.

[0073] The first sub-call parameter return value bound to the target first vector in the vector database is used as the target first sub-call parameter return value.

[0074] S209: Use the target first sub-call parameter return value as the second sub-call parameter return value corresponding to the second sub-call parameter.

[0075] S210, performing subsequent playback steps based on the second sub-call parameter return value.

[0076] S211, in response to monitoring that playback of the data to be played back is completed, obtaining a playback result of the data to be played back.

[0077] It is not difficult to understand that directly using the original sub-call parameters for matching may be limited to simple string matching or simple numerical comparison, and cannot support more complex logic. In the embodiment of the present application, the vector data corresponding to the sub-call parameters are used to construct and retrieve the vector database. Compared with directly using the original sub-call parameters for matching, it supports more complex matching logic, improves the flexibility and intelligence of the playback process, and can improve data security and privacy protection, reducing the risk of data leakage.

[0078] Figure 3 is a schematic diagram of an exemplary implementation of a method for traffic recording and playback shown in the present application, such as Figure 3 As shown, the method for traffic recording and playback includes the following steps:

[0079] S301, receiving the recording configuration parameters sent by the configuration center, and waking up the recording service corresponding to the target process.

[0080] Combine the following Figure 4 Give an example introduction. Figure 4 This is a framework diagram of a method for traffic recording and playback shown in this application, such as Figure 4 As shown, the recording service can be pre-attached to the target process based on the probe technology. The playback service can be pre-attached to the corresponding playback process in the test environment based on the probe technology. The configuration center is used to send down configuration recording parameters and playback recording parameters.

[0081] The configuration of recording parameters includes plug-in configuration parameters and recording constraint parameters.

[0082] The plug-in configuration parameters support configuration based on protocols, which are used to define the cut-off points of the recording protocols.

[0083] Among them, the recording constraint parameters may include the recording sampling ratio, the interfaces to be ignored during recording, the maximum number of items recorded at a time, whether to reset the last number of items recorded, the disabled sub-plugin list, the Java sub-methods to be recorded / mocked, the recording record delivery address and other parameters.

[0084] Among them, the playback recording parameters are mainly used to select qualified data from the recorded data as the data to be played back.

[0085] In this application, the configuration recording parameters can be sent to an instance according to business needs, and the traffic on the current instance will be recorded; the configuration recording parameters can also be sent to the entire cluster, and the traffic of the entire cluster will be recorded; the same is true for the playback configuration.

[0086] S302: Based on the recording service, the recording plug-in is called according to the recording configuration parameters to perform traffic recording on the target process to obtain recording data.

[0087] First, the plug-in configuration parameters included in the recording configuration parameters are obtained, and the recording plug-ins (including the inlet traffic plug-in and the sub-call plug-in) included in the recording service are logically enhanced according to the plug-in configuration parameters. Among them, the plug-in configuration parameters support configuration based on the protocol, which is used to define the cut-off point of the recording protocol. After determining the plug-in configuration parameters, the recording enhancement logic is woven according to the cut-off point definition.

[0088] By configuring the plug-in parameters, you can flexibly adjust and enhance the functions of the recording service according to different recording protocol requirements, adapt to various business scenarios and requirements, and improve the adaptability and flexibility of the solution.

[0089] Among them, the inlet traffic plug-in is responsible for processing the parameters and return values ​​of the inlet traffic, and the sub-call plug-in is responsible for processing the parameters and return values ​​of the sub-call.

[0090] Then, the recording constraint parameters (such as recording sampling ratio, user ID, etc.) included in the recording configuration parameters are obtained, and the traffic to be recorded corresponding to the target process is determined according to the recording constraint parameters. The recording plug-in is called based on the recording service to record the traffic to be recorded, and the parameter extraction logic and return value extraction logic defined in the recording plug-in are executed in sequence to obtain the recording data.

[0091] Among them, each recording data corresponds to a unique recording identifier, and each recording data at least includes the first entry parameter and the first entry parameter return value obtained by recording the entry traffic plug-in, and the first sub-call parameter and the first sub-call parameter return value obtained by recording the sub-call plug-in.

[0092] Among them, determining the traffic to be recorded corresponding to the target process according to the recording constraint parameters includes: determining the traffic filtering dimension and the recording sampling ratio corresponding to the target process according to the recording constraint parameters, wherein the traffic filtering dimension is at least one of the user identification dimension, the domain name dimension or the process interface dimension; determining the traffic to be recorded of the target process according to the traffic filtering dimension and the recording sampling ratio.

[0093] Doing so, collecting data according to specific recording constraint parameters can ensure that only necessary data is recorded, reduce unnecessary data processing, optimize resource usage, and reduce system burden and storage requirements.

[0094] Among them, the recording plug-in can be embedded in the recording service.

[0095] After obtaining the recorded data, the recorded data can be sent to the data management platform for management. The data management platform supports users to query and display data in a structured manner.

[0096] S303, establishing a vector database according to the recorded data, wherein the vector database at least includes a first vector corresponding to a first sub-call parameter in the recorded data and a first sub-call parameter return value.

[0097] S304, receiving the playback configuration parameters sent by the configuration center, and waking up the corresponding playback service of the target process in the test environment.

[0098] The playback service is used to simulate the client calling related interfaces using the recorded parameters to obtain the playback results.

[0099] Among them, the playback plug-in can be embedded in the playback service.

[0100] S305: Obtain a target recording identifier carried in the playback configuration parameter, and determine the data to be played back from the recorded data according to the target recording identifier.

[0101] Each recorded data corresponds to a unique recording identifier. In the present application, the data to be played back is determined from the recorded data according to the target recording identifier carried by the playback configuration parameter.

[0102] S306: Invoke the playback plug-in based on the playback service to initiate playback of the playback data.

[0103] After the data to be replayed is determined, in the test environment corresponding to the target process, the client is simulated to initiate playback based on the entry parameters corresponding to the data to be replayed.

[0104] S307, in response to monitoring the playback operation of the second sub-call parameter included in the playback data during the playback process, matching the second sub-call parameter return value corresponding to the second sub-call parameter from the vector database, and performing subsequent playback steps based on the second sub-call parameter return value.

[0105] After the playback request is initiated, the sub-call in the request will not actually occur, but the playback plug-in is responsible for MOCK. If the playback operation of the second sub-call parameter included in the playback data is detected during the playback process, the third-party call service of the second sub-call parameter is intercepted, and the second sub-call parameter is feature extracted to obtain the second vector corresponding to the second sub-call parameter, and the vector database is queried based on the second vector to obtain the target first vector corresponding to the second vector.

[0106] S308: In response to monitoring that playback of the data to be played back is completed, obtaining playback results of the data to be played back.

[0107] The embodiment of the present application can capture user requests and responses occurring in an actual production environment by recording the traffic of the target process; the vector database established by the present application stores the first vector and the first sub-call parameter return value corresponding to the first sub-call parameter in the recorded data, providing data support for subsequent playback; on the other hand, the first sub-call parameter is converted into a first vector, and since the vector usually has a fixed dimension and format, the matching speed during the playback process can be significantly improved, especially when frequent queries are required.

[0108] Furthermore, after obtaining the playback results, a difference analysis tool (diff tool) is used to generate a difference analysis report based on the recorded data and the playback results. The difference analysis report may include the comparison results of specific data values ​​in the recorded data and the playback results, the comparison results of data types, the comparison results of the steps and operation sequences executed during the recording and playback processes, etc. The difference analysis report provides detailed difference information, which helps the development and testing teams debug and fix problems more effectively.

[0109] Figure 5 is a schematic diagram of a flow recording and playback device shown in the present application, such as Figure 5 As shown, the traffic recording and playback device 500 includes a traffic recording module 501, a first playback module 502, a second playback module 503 and a third playback module 504, wherein:

[0110] The traffic recording module 501 is used to perform traffic recording on the target process to obtain recording data, and to establish a vector database based on the recording data. The vector database includes at least the first vector corresponding to the first sub-call parameter in the recording data and the first sub-call parameter return value.

[0111] The first playback module 502 is used to determine the data to be played back in the recorded data, and initiate playback of the data to be played back in the test environment.

[0112] The second playback module 503 is used to respond to the playback operation of the second sub-call parameter included in the playback data monitored during the playback process, match the second sub-call parameter return value corresponding to the second sub-call parameter from the vector database, and perform subsequent playback steps based on the second sub-call parameter return value.

[0113] The third playback module 504 is configured to obtain a playback result of the data to be played back in response to monitoring that the playback of the data to be played back is completed.

[0114] By recording the traffic of the target process, the device can capture user requests and responses occurring in an actual production environment; the vector database established by the device stores the first vector and the first sub-call parameter return value corresponding to the first sub-call parameter in the recorded data, providing data support for subsequent playback; on the other hand, converting the first sub-call parameter into the first vector can significantly improve the matching speed during playback, especially when frequent queries are required, because vectors usually have fixed dimensions and formats.

[0115] According to one embodiment of the present application, the traffic recording module 501 is also used to: obtain sub-call related information included in each recording data, the sub-call related information including a first sub-call parameter and a first sub-call parameter return value corresponding to the first sub-call parameter; perform feature extraction on the first sub-call parameter to obtain a first vector corresponding to the first sub-call parameter; bind the first vector and the first sub-call parameter return value to generate vector related information, and store the vector related information in a vector database.

[0116] According to one embodiment of the present application, the second playback module 503 is also used to: perform feature extraction on the second sub-call parameter to obtain a second vector corresponding to the second sub-call parameter; query the vector database based on the second vector to obtain a target first vector corresponding to the second vector; obtain a target first sub-call parameter return value bound to the target first vector; and use the target first sub-call parameter return value as the second sub-call parameter return value corresponding to the second sub-call parameter.

[0117] According to one embodiment of the present application, the second playback module 503 is also used to: select multiple candidate first vectors related to the second vector from a vector database; calculate the vector similarity and / or vector distance between the second vector and each candidate first vector respectively; and determine the target first vector from the multiple candidate first vectors based on the vector similarity and / or vector distance.

[0118] According to one embodiment of the present application, the second playback module 503 is further used to: obtain a filtering parameter corresponding to the second sub-call parameter, the filtering parameter including at least one of a user identifier and a sub-call interface name; and select a first vector that meets the filtering parameter from the vector database as a candidate first vector.

[0119] According to one embodiment of the present application, the traffic recording module 501 is also used to: receive the recording configuration parameters issued by the configuration center, and wake up the recording service corresponding to the target process; based on the recording service, call the recording plug-in according to the recording configuration parameters to perform traffic recording on the target process to obtain recording data; wherein each recording data corresponds to a unique recording identifier, and each recording data includes at least a first entry parameter, a first entry parameter return value, a first sub-call parameter, and a first sub-call parameter return value.

[0120] According to one embodiment of the present application, the traffic recording module 501 is also used to: obtain the plug-in configuration parameters included in the recording configuration parameters, and perform logical enhancement on the recording plug-in included in the recording service according to the plug-in configuration parameters; obtain the recording constraint parameters included in the recording configuration parameters, and determine the traffic to be recorded corresponding to the target process according to the recording constraint parameters; and record the traffic to be recorded based on the recording service by calling the recording plug-in to obtain recording data.

[0121] According to one embodiment of the present application, the traffic recording module 501 is also used to: determine the traffic filtering dimension and recording sampling ratio corresponding to the target process according to the recording constraint parameters, wherein the traffic filtering dimension is at least one of the user identification dimension, the domain name dimension or the process interface dimension; determine the traffic to be recorded of the target process according to the traffic filtering dimension and the recording sampling ratio.

[0122] According to one embodiment of the present application, the first playback module 502 is also used to: receive the playback configuration parameters issued by the configuration center, wake up the corresponding playback service of the target process in the test environment; obtain the target recording identifier carried by the playback configuration parameters, and determine the data to be played back from the recorded data according to the target recording identifier; and initiate playback of the data to be played back based on the playback service by calling the playback plug-in.

[0123] According to an embodiment of the present application, the traffic recording and playback device 500 further includes: an analysis module, configured to generate a difference analysis report based on the recording data and the playback results.

[0124] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0125] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0126] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0127] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0128] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the method of traffic recording and playback. For example, in some embodiments, the method of traffic recording and playback may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the method of traffic recording and playback described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the method of traffic recording and playback in any other appropriate manner (e.g., by means of firmware).

[0129] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0131] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0133] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0134] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0135] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0136] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for traffic recording and playback, wherein: include: Perform traffic recording on the target process to obtain recording data, and establish a vector database based on the recording data, wherein the vector database at least includes a first vector corresponding to a first sub-call parameter in the recording data and a return value of the first sub-call parameter; Determine the data to be replayed in the recorded data, and initiate playback of the data to be replayed in a test environment; In response to monitoring a playback operation on a second sub-call parameter included in the data to be played back during the playback process, matching and obtaining a second sub-call parameter return value corresponding to the second sub-call parameter from the vector database, and performing subsequent playback steps based on the second sub-call parameter return value; In response to monitoring that playback of the data to be played back is completed, a playback result of the data to be played back is obtained.

2. The method according to claim 1, wherein: The step of establishing a vector database according to the recorded data comprises: Obtaining sub-call related information included in each piece of the recorded data, wherein the sub-call related information includes a first sub-call parameter and a first sub-call parameter return value corresponding to the first sub-call parameter; Performing feature extraction on the first sub-call parameter to obtain a first vector corresponding to the first sub-call parameter; The first vector and the first sub-call parameter return value are bound to generate vector related information, and the vector related information is stored in a vector database.

3. The method according to claim 2, wherein: The step of matching and obtaining the second sub-call parameter return value corresponding to the second sub-call parameter from the vector database includes: Performing feature extraction on the second sub-call parameter to obtain a second vector corresponding to the second sub-call parameter; Query the vector database based on the second vector to obtain a target first vector corresponding to the second vector; Obtain a target first sub-call parameter return value bound to the target first vector; The target first sub-call parameter return value is used as the second sub-call parameter return value corresponding to the second sub-call parameter.

4. The method according to claim 3, wherein: The querying the vector database based on the second vector to obtain the target first vector corresponding to the second vector includes: Selecting a plurality of candidate first vectors related to the second vector from the vector database; Calculate and obtain the vector similarity and / or vector distance between the second vector and each of the candidate first vectors; A target first vector is determined from the plurality of candidate first vectors according to the vector similarity and / or the vector distance.

5. The method according to claim 4, wherein: The selecting a plurality of candidate first vectors related to the second vector from the vector database comprises: Obtaining a screening parameter corresponding to the second sub-call parameter, wherein the screening parameter includes at least one of a user identifier and a sub-call interface name; A first vector that meets the screening parameter is selected from the vector database as a candidate first vector.

6. The method according to any one of claims 1 to 5, wherein: The step of recording the traffic of the target process and obtaining recording data includes: Receive the recording configuration parameters sent by the configuration center, and wake up the recording service corresponding to the target process; Based on the recording service, calling the recording plug-in according to the recording configuration parameters to perform traffic recording on the target process to obtain recording data; Each of the recording data corresponds to a unique recording identifier, and each of the recording data at least includes a first entry parameter, a first entry parameter return value, a first sub-call parameter and a first sub-call parameter return value.

7. The method according to claim 6, wherein: Based on the recording service, calling the recording plug-in according to the recording configuration parameters to perform traffic recording on the target process to obtain recording data includes: Acquire the plug-in configuration parameters included in the recording configuration parameters, and perform logic enhancement on the recording plug-in included in the recording service according to the plug-in configuration parameters; Acquire the recording constraint parameters included in the recording configuration parameters, and determine the to-be-recorded traffic corresponding to the target process according to the recording constraint parameters; The recording plug-in is called based on the recording service to record the traffic to be recorded to obtain recording data.

8. The method according to claim 7, wherein: The determining, according to the recording constraint parameter, the to-be-recorded traffic corresponding to the target process includes: Determining a traffic screening dimension and a recording sampling ratio corresponding to the target process according to the recording constraint parameter, wherein the traffic screening dimension is at least one of a user identification dimension, a domain name dimension, or a process interface dimension; The traffic to be recorded of the target process is determined according to the traffic screening dimension and the recording sampling ratio.

9. The method according to claim 8, wherein: The determining the data to be replayed in the recorded data and initiating replay of the data to be replayed in the test environment includes: Receive the playback configuration parameters sent by the configuration center, and wake up the corresponding playback service of the target process in the test environment; Acquire the target recording identifier carried by the playback configuration parameter, and determine the data to be played back from the recorded data according to the target recording identifier; The playback plug-in is called based on the playback service to initiate playback of the data to be played back.

10. The method according to claim 9, wherein: The method further comprises: A difference analysis report is generated based on the recording data and the playback results.

11. A device for recording and replaying traffic, wherein: include: A traffic recording module, used to perform traffic recording on a target process to obtain recording data, and to establish a vector database based on the recording data, wherein the vector database includes at least a first vector corresponding to a first sub-call parameter in the recording data and a first sub-call parameter return value; A first playback module, used to determine the data to be played back in the recorded data, and initiate playback of the data to be played back in a test environment; A second playback module is configured to, in response to monitoring a playback operation on a second sub-call parameter included in the data to be played back during the playback process, match and obtain a second sub-call parameter return value corresponding to the second sub-call parameter from the vector database, and perform subsequent playback steps based on the second sub-call parameter return value; The third playback module is used to obtain the playback result of the data to be played back in response to monitoring that the playback of the data to be played back is completed.

12. The device according to claim 11, wherein The traffic recording module is also used for: Obtaining sub-call related information included in each piece of the recorded data, wherein the sub-call related information includes a first sub-call parameter and a first sub-call parameter return value corresponding to the first sub-call parameter; Performing feature extraction on the first sub-call parameter to obtain a first vector corresponding to the first sub-call parameter; The first vector and the first sub-call parameter return value are bound to generate vector related information, and the vector related information is stored in a vector database.

13. The device according to claim 12, wherein: The second playback module is further used for: Performing feature extraction on the second sub-call parameter to obtain a second vector corresponding to the second sub-call parameter; Query the vector database based on the second vector to obtain a target first vector corresponding to the second vector; Obtain a target first sub-call parameter return value bound to the target first vector; The target first sub-call parameter return value is used as the second sub-call parameter return value corresponding to the second sub-call parameter.

14. The device according to claim 13, wherein: The second playback module is further used for: Selecting a plurality of candidate first vectors related to the second vector from the vector database; Calculate and obtain the vector similarity and / or vector distance between the second vector and each of the candidate first vectors; A target first vector is determined from the plurality of candidate first vectors according to the vector similarity and / or the vector distance.

15. The device according to claim 14, wherein: The second playback module is further used for: Obtaining a screening parameter corresponding to the second sub-call parameter, wherein the screening parameter includes at least one of a user identifier and a sub-call interface name; A first vector that meets the screening parameter is selected from the vector database as a candidate first vector.

16. The device according to any one of claims 11 to 15, wherein: The traffic recording module is also used for: Receive the recording configuration parameters sent by the configuration center, and wake up the recording service corresponding to the target process; Based on the recording service, calling the recording plug-in according to the recording configuration parameters to perform traffic recording on the target process to obtain recording data; Each of the recording data corresponds to a unique recording identifier, and each of the recording data at least includes a first entry parameter, a first entry parameter return value, a first sub-call parameter and a first sub-call parameter return value.

17. The device according to claim 16, wherein: The traffic recording module is also used for: Acquire the plug-in configuration parameters included in the recording configuration parameters, and perform logic enhancement on the recording plug-in included in the recording service according to the plug-in configuration parameters; Acquire the recording constraint parameters included in the recording configuration parameters, and determine the to-be-recorded traffic corresponding to the target process according to the recording constraint parameters; The recording plug-in is called based on the recording service to record the traffic to be recorded to obtain recording data.

18. The device according to claim 17, wherein: The traffic recording module is also used for: Determining a traffic screening dimension and a recording sampling ratio corresponding to the target process according to the recording constraint parameter, wherein the traffic screening dimension is at least one of a user identification dimension, a domain name dimension, or a process interface dimension; The traffic to be recorded of the target process is determined according to the traffic screening dimension and the recording sampling ratio.

19. The device according to claim 18, wherein: The first playback module is further used for: Receive the playback configuration parameters sent by the configuration center, and wake up the corresponding playback service of the target process in the test environment; Acquire the target recording identifier carried by the playback configuration parameter, and determine the data to be played back from the recorded data according to the target recording identifier; The playback plug-in is called based on the playback service to initiate playback of the data to be played back.

20. The device according to claim 19, wherein The device also includes: The analysis module is used to generate a difference analysis report based on the recording data and the playback result.

21. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images