Traffic recording method, computer device, storage medium and computer program product

Abstract

The application relates to a traffic recording method, computer equipment, a storage medium and a computer program product. The method comprises the following steps: in the case that it is detected that a request flow sent by an upstream service meets a recording condition, performing recording processing on the request flow according to preset recording configuration information, to obtain a recorded request flow; adding a recording identifier to the request flow, and sending the request flow carrying the recording identifier to a corresponding downstream service; performing recording processing on a response flow returned by the downstream service based on the request flow carrying the recording identifier according to the recording configuration information, to obtain a recorded response flow; and sending the recorded request flow and the recorded response flow to a traffic middle platform asynchronously, so that the traffic middle platform converts the recorded request flow and the recorded response flow into visualized traffic. The method can realize zero-invasion service traffic recording and improve traffic recording performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular, to a traffic recording method, a computer device, a storage medium, and a computer program product. BACKGROUND

[0002] Traffic recording refers to recording requests and then forwarding and storing the recorded traffic, so that testers can use the recorded traffic to perform testing. Traffic recording technology is a relatively important technology in the field of software testing.

[0003] However, current traffic recording technology usually uses AOP (Aspect Oriented Programming) to perform aspect processing on requests in traffic to achieve traffic recording. However, this approach needs to intrude into business services, which reduces the security and processing performance of the business services. SUMMARY

[0004] Therefore, it is necessary to provide a traffic recording method, a computer device, a computer readable storage medium, and a computer program product that can achieve zero-intrusion service and improve traffic recording performance.

[0005] In a first aspect, the present application provides a traffic recording method. The method comprises:

[0006] In a case where it is detected that a request stream sent by an upstream service meets recording conditions, recording processing is performed on the request stream according to preset recording configuration information, to obtain a recorded request stream;

[0007] A recording identifier is added to the request stream, and the request stream carrying the recording identifier is sent to a corresponding downstream service;

[0008] According to the recording configuration information, recording processing is performed on a response stream returned by the downstream service based on the request stream carrying the recording identifier, to obtain a recorded response stream;

[0009] The recorded request stream and the recorded response stream are sent to a traffic middle platform asynchronously, so that the traffic middle platform converts the recorded request stream and the recorded response stream into visualized traffic.

[0010] In one of the embodiments, before the recording processing is performed on the request stream according to the preset recording configuration information to obtain the recorded request stream in a case where it is detected that the request stream sent by the upstream service meets the recording conditions, the method further comprises:

[0011] The request stream sent by the upstream service is received;

[0012] According to the protocol filter, network sniffing is performed on the request stream to obtain service communication protocol corresponding to the request stream;

[0013] According to the protocol filter, decoding processing is performed on the service communication protocol to obtain header information of the service communication protocol.

[0014] According to the header information, it is detected whether the request stream meets recording conditions.

[0015] In one of the embodiments, before receiving the request stream sent by the upstream service, the method further comprises:

[0016] The recording configuration information sent by the service mesh component is received, and historical configuration information in the local service cluster is updated according to the recording configuration information to obtain the preset recording configuration information.

[0017] In one of the embodiments, the recording configuration information sent by the service mesh component is obtained from the service center when the service mesh component listens to a change in configuration information of the service center.

[0018] In one of the embodiments, the recorded request stream and the recorded response stream are asynchronously sent to the traffic middle station, comprising:

[0019] According to a preset transmission protocol, the recorded request stream and the recorded response stream are sent to the traffic middle station through bypass; the preset transmission protocol is any one of user datagram protocol and hypertext transfer protocol.

[0020] In one of the embodiments, the traffic recording method further comprises:

[0021] Adding a recording identifier to the service communication protocol in the network;

[0022] In a case where it is detected that the service communication protocol corresponding to the received request stream carries the recording identifier, recording processing is performed on all requests in a request link associated with the request stream according to the recording configuration information to obtain a recorded request link;

[0023] In a case where it is detected that the service communication protocol corresponding to the received response stream carries the recording identifier, recording processing is performed on all responses in a response link associated with the response stream according to the recording configuration information to obtain a recorded response link;

[0024] The recorded request link and the recorded response link are asynchronously sent to the traffic middle station.

[0025] In one of the embodiments, the traffic recording method further comprises:

[0026] generating a test case according to the visualized traffic;

[0027] performing stress testing on the test case according to the simulation server.

[0028] In a second aspect, the present application also provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0029] In a case where it is detected that the request stream sent by the upstream service meets the recording condition, performing recording processing on the request stream according to preset recording configuration information, to obtain a recorded request stream;

[0030] adding a recording identifier to the request stream, and sending the request stream carrying the recording identifier to the corresponding downstream service;

[0031] performing recording processing on a response stream returned by the downstream service based on the request stream carrying the recording identifier according to the recording configuration information, to obtain a recorded response stream;

[0032] asynchronously sending the recorded request stream and the recorded response stream to a traffic middle platform, so that the traffic middle platform converts the recorded request stream and the recorded response stream into visualized traffic.

[0033] In a third aspect, the present application also provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program implements the following steps when executed by a processor:

[0034] In a case where it is detected that the request stream sent by the upstream service meets the recording condition, performing recording processing on the request stream according to preset recording configuration information, to obtain a recorded request stream;

[0035] adding a recording identifier to the request stream, and sending the request stream carrying the recording identifier to the corresponding downstream service;

[0036] performing recording processing on a response stream returned by the downstream service based on the request stream carrying the recording identifier according to the recording configuration information, to obtain a recorded response stream;

[0037] asynchronously sending the recorded request stream and the recorded response stream to a traffic middle platform, so that the traffic middle platform converts the recorded request stream and the recorded response stream into visualized traffic.

[0038] In a fourth aspect, the present application also provides a computer program product. The computer program product comprises a computer program, and the computer program implements the following steps when executed by a processor:

[0039] In a case where it is detected that the request stream sent by the upstream service meets the recording condition, recording processing is performed on the request stream according to preset recording configuration information, to obtain a recorded request stream;

[0040] The recording identification is added to the request stream, and the request stream carrying the recording identification is sent to the corresponding downstream service;

[0041] According to the recording configuration information, recording processing is performed on the response stream returned by the downstream service based on the request stream carrying the recording identification, to obtain a recorded response stream;

[0042] The recorded request stream and the recorded response stream are sent to the traffic middle platform asynchronously, so that the traffic middle platform converts the recorded request stream and the recorded response stream into visualized traffic.

[0043] The traffic recording method, the computer device, the storage medium and the computer program product, in a case where it is detected that the request stream sent by the upstream service meets the recording condition, recording processing is performed on the request stream according to preset recording configuration information, to obtain a recorded request stream; the recording identification is added to the request stream, and the request stream carrying the recording identification is sent to the corresponding downstream service; according to the recording configuration information, recording processing is performed on the response stream returned by the downstream service based on the request stream carrying the recording identification, to obtain a recorded response stream; the recorded request stream and the recorded response stream are sent to the traffic middle platform asynchronously, so that the traffic middle platform converts the recorded request stream and the recorded response stream into visualized traffic, not only realizing the recording of the request stream, but also realizing the paired recording of the response stream corresponding to the request stream through the recording identification, so that the obtained traffic is more complete, without invading the upstream service and the downstream service, and through the asynchronous transmission of the recorded request stream and the recorded response stream, the business processing of the upstream service and the downstream service can be avoided, thereby greatly improving the performance of traffic recording. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 An application environment diagram of the traffic recording method in one embodiment;

[0045] Figure 2 A flowchart of the traffic recording method in one embodiment;

[0046] Figure 3 A flowchart of the step of detecting that the request stream meets the recording condition in one embodiment;

[0047] Figure 4 A flowchart of the traffic recording method in another embodiment;

[0048] Figure 5This is a timing diagram of the traffic recording method in yet another embodiment;

[0049] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] The traffic recording method provided in this application embodiment can be applied to, for example... Figure 1 The application environment is shown. In this environment, the service mesh sidecar 101 connects to the network and traffic platform 102, the service mesh component 103, and the upstream service...

[0052] Service 104 communicates with downstream service 105, and service mesh component 103 communicates with service center 1065 via the network. When service mesh sidecar 101 detects that the request stream sent by upstream service 104 meets the recording conditions, it records the request stream according to preset recording configuration information to obtain a recorded request stream; it adds a recording identifier to the request stream and sends the request stream carrying the recording identifier to the corresponding downstream service 105; according to the recording configuration information, it returns a response to downstream service 105 based on the request stream carrying the recording identifier.

[0053] The response stream is recorded to obtain a recorded response stream; the recorded request stream and the recorded response stream are asynchronously sent to the traffic platform 102 so that the traffic platform 102 can convert the recorded request stream and the recorded response stream into visual traffic.

[0054] In this context, a service mesh sidecar refers to a process deployed alongside the business process; for example, a service mesh sidecar can be implemented using Envoy. The service mesh itself refers to the infrastructure used to handle inter-service communication.

[0055] The infrastructure layer; the service mesh provides five functions: service discovery, load balancing, fault recovery, metric collection, and monitoring; the service mesh can be implemented using Istio. The service mesh component refers to the control plane component within the service mesh, providing service discovery and traffic management functions. For example, the service mesh component could be the pilot module in Istio. The traffic platform refers to the platform that stores, processes, or manages data such as request and response streams.

[0056] In one embodiment, such as Figure 2 As shown, a traffic recording method is provided, which is applied to 0 Figure 1The service mesh sidecar in the service mesh is taken as an example for illustration, including the following steps.

[0057] In step S201, in a case where it is detected that the request stream sent by the upstream service meets the recording condition, the request stream is recorded according to preset recording configuration information, to obtain a recorded request stream.

[0058] The upstream service refers to a sender of the request stream and the like.

[0059] The request stream refers to a request sent by the upstream service to the downstream service through a communication protocol. The request stream can be a binary data stream.

[0060] The recorded request stream refers to the request stream obtained through recording.

[0061] The recording condition refers to a condition for determining whether the request stream needs to be recorded. For example, the recording condition can be set as recording once every 5 seconds, and in a case where the time information in the header information meets the interval of 5 seconds, the request stream corresponding to the header information is recorded.

[0062] The recording configuration information refers to configuration information indicating the recording mode of the request stream, the response stream and the like. The recording configuration information can also be converted into metadata of the service, and the metadata is stored in a database. For example, the recording configuration information can be a recording strategy and a recording strategy parameter, such as sampling recording and a sampling rate thereof, frequency limiting recording and a frequency limiting value thereof, dyeing recording and a dyeing key value thereof, full link recording and the like. The dyeing key value can be a service command word, an instance IP (Internet Protocol), a user identification (UID), a link trace identification (Traceid) and a custom field.

[0063] The recording condition of the data stream can be set in advance, so that the service mesh sidecar records the request stream or the response stream when receiving the request stream or the response stream. Specifically, when there is a business requirement of the upstream service, the upstream service can send a request stream to the downstream service through a remote calling protocol. The request stream is proxied to the service mesh sidecar by the service mesh, and then the service mesh sidecar receives the request stream sent by the upstream service through the remote calling protocol. The service mesh sidecar includes a main thread and a plurality of worker threads. The worker thread in the service mesh sidecar detects whether the request stream meets the recording condition. In a case where the worker thread detects that the request stream meets the recording condition, the worker thread records the request stream according to preset recording configuration information, and takes the recorded request stream as a recorded request stream. In a case where the worker thread detects that the request stream does not meet the recording condition, the request does not need to be recorded.

[0064] Step S202, adding a recording identifier to the request stream, and sending the request stream carrying the recording identifier to the corresponding downstream service.

[0065] The recording identifier refers to identification information used to mark that the data stream (e.g., the request stream, the response stream) needs to be recorded.

[0066] The downstream service refers to the receiver of the request and other data streams.

[0067] Step S203, according to the recording configuration information, recording the response stream returned by the downstream service based on the request stream carrying the recording identifier to obtain the recorded response stream.

[0068] The response stream refers to the response sent by the downstream service to the upstream service through the communication protocol in response to the request sent by the upstream service. The response stream can be a binary data stream (stream). For example, assuming that the request stream is request, the response stream corresponding to the request stream can be response.

[0069] The recorded response stream refers to the response stream obtained by recording.

[0070] Specifically, the service sidecar mesh adds a recording identifier to the request stream to obtain a request stream carrying the recording identifier, and then sends the request stream carrying the recording identifier to the corresponding downstream service. Then, the downstream service receives the request stream carrying the recording identifier, generates a corresponding response stream according to the request stream carrying the recording identifier, and returns the response stream to the upstream service. In the process of returning the response stream, the response stream is proxied by the service mesh to the service mesh sidecar, and then the service mesh sidecar receives the response stream. The working thread in the service mesh sidecar detects the recording identifier and confirms that the response stream is the response of the downstream service to the request stream carrying the recording identifier in step S202. At this time, the working thread records the response stream according to the recording configuration information, and takes the recorded response stream as the recorded response stream, thereby realizing the paired recording of the request stream and the response stream corresponding to the request stream.

[0071] Further, the working thread of the service mesh sidecar can also add a recording identifier to the pipeline of the request stream. Since the request stream and the response stream corresponding to the request stream are transmitted through the same pipeline, the working thread of the service mesh sidecar can also record the response stream returned through the pipeline marked with the recording identifier according to the recording configuration information to obtain the recorded response stream, thereby realizing the paired recording of the request stream and the response stream corresponding to the request stream. The pipeline refers to a communication mechanism in a computer network, rather than a pipeline in the physical sense that transports physical entities such as water and natural gas.

[0072] Step S204, the post-recording request stream and the post-recording response stream are sent to the traffic middle station asynchronously, so that the traffic middle station converts the post-recording request stream and the post-recording response stream into visualized traffic.

[0073] The visualized traffic refers to data obtained by visualizing conversion of the post-recording request stream and the post-recording response stream. The visualized traffic can be traffic in json format. It should be noted that the visualized traffic, the post-recording request stream (or the request stream) and the post-recording response stream (or the response stream) are traffic in different formats.

[0074] Specifically, the service mesh sidecar can send the post-recording request stream and the post-recording response stream to the traffic middle station through an asynchronous transmission mode together after recording the post-recording request stream and the post-recording response stream. Alternatively, the service mesh sidecar can send the post-recording request stream to the traffic middle station through an asynchronous transmission mode after obtaining the post-recording request stream, and then send the post-recording response stream to the traffic middle station through an asynchronous transmission mode after obtaining the post-recording response stream. The traffic middle station receives the post-recording request stream and the post-recording response stream, and stores the post-recording request stream and the post-recording response stream. The traffic middle station can also encode and decode the post-recording request stream and the post-recording response stream in combination with a protocol file to obtain visualized traffic.

[0075] In the traffic recording method, when it is detected that the request stream sent by the upstream service meets the recording condition, the request stream is recorded according to the preset recording configuration information to obtain a post-recording request stream; the recording identifier is added to the request stream, and the request stream carrying the recording identifier is sent to the corresponding downstream service; the response stream returned by the downstream service based on the request stream carrying the recording identifier is recorded according to the recording configuration information to obtain a post-recording response stream; and the post-recording request stream and the post-recording response stream are sent to the traffic middle station asynchronously, so that the traffic middle station converts the post-recording request stream and the post-recording response stream into visualized traffic. Not only the recording of the request stream is realized, but also the paired recording of the response stream corresponding to the request stream is realized through the recording identifier, so that the obtained traffic is more complete without invading the upstream service and the downstream service. The asynchronous transmission of the post-recording request stream and the post-recording response stream can avoid affecting the business processing of the upstream service and the downstream service, thereby greatly improving the performance of traffic recording.

[0076] In one embodiment, as shown in FIG. 1, the traffic recording method further includes the following steps. Figure 3 Before the step S201, when it is detected that the request stream sent by the upstream service meets the recording condition, the request stream is recorded according to the preset recording configuration information to obtain a post-recording request stream, the method further includes the following steps.

[0077] Step S301, receiving a request stream sent by an upstream service.

[0078] In step S302, the request stream is sniffed according to the protocol filter to obtain the service communication protocol corresponding to the request stream.

[0079] In step S303, the service communication protocol is decoded according to the protocol filter to obtain the header information of the service communication protocol.

[0080] In step S304, whether the request stream meets the recording condition is detected according to the header information.

[0081] The service communication protocol refers to a communication protocol used by both parties of a service to complete communication. The service communication protocol can be a private Remote Procedure Call Protocol (RPC protocol), for example, the private RPC protocol can be a Tencent Music Entertainment Protocol, a QZone Application Protocol (QZA), and a Serverbench Plus Plus Remote Call Framework Protocol (SRF) 4-layer protocol, or a Hypertext Transfer Protocol (HTTP), a Google Remote Procedure Call (gRPC), and a HTTP2 7-layer protocol.

[0082] The protocol filter refers to a filter used to filter out the communication protocol of the upstream service. For example, the protocol filter can be a TCP filter, and can also be a protocol filter improved based on the TCP filter, which can sniff out a private RPC protocol.

[0083] The recording condition is stored in the local cache of the service mesh sidecar. The service mesh sidecar needs to determine whether the received request stream meets the recording condition to determine whether the request stream needs to be recorded. Specifically, before the service mesh records the request stream in step S201, the service mesh proxies the request stream transmitted by the upstream service to the downstream service to the service mesh sidecar, and the service mesh sidecar receives the request stream. The worker thread in the service mesh can obtain a TCP filter, sniff out the service communication protocol from the request stream through the TCP filter, and then decode the service communication protocol through the TCP filter again to obtain the header information of the service communication protocol. The header information refers to the string transmitted by the upstream service before the request stream is transmitted to the downstream service through the service communication protocol. The header information includes request sending time, request type, and header field information, so that the worker thread can determine whether the request stream meets the recording condition according to the header information.

[0084] In this embodiment, the request stream sent by the upstream service is received. The request stream is network sniffed according to the protocol filter to obtain the service communication protocol corresponding to the request stream. The service communication protocol is decoded according to the protocol filter to obtain the header information of the service communication protocol. Whether the request stream meets the recording condition is detected according to the header information. Not only is it determined whether the currently received request stream needs to be recorded through the header information of the service communication protocol, but also the upstream service and the downstream service do not need to be invaded, thereby avoiding affecting the business processing of the upstream service and the downstream service, and improving the performance of traffic recording.

[0085] In one embodiment, before the request stream sent by the upstream service is received in step S301, the recording configuration information sent by the service mesh component is also received, and the historical configuration information in the local service cluster is updated according to the recording configuration information to obtain the preset recording configuration information.

[0086] The local service cluster refers to the service cluster in the local cache of the service mesh sidecar.

[0087] The historical configuration information refers to the historical recording configuration stored in the local service cluster of the service mesh.

[0088] Specifically, after the service mesh component obtains the new recording configuration information, the service mesh component sends the recording configuration information to the service mesh sidecar through a channel of the xDS protocol. The service mesh sidecar receives the recording configuration information, and then a main thread in the service mesh sidecar instructs all worker threads in the service mesh sidecar to update the historical configuration information in the local service cluster to the received recording configuration information, and to use the updated recording configuration information in the local service cluster as the preset recording configuration information in step S201. The xDS protocol is a communication protocol between the service mesh and the service mesh component. The xDS protocol is a collection of service discovery and governance rules. For example, the xDS protocol can include an LDS (Listener Discovery Service), a CDS (Cluster Discovery Service), an EDS (Endpoint Discovery Service), and an RDS (Route Discovery Service).

[0089] For example, after the service mesh component obtains the new recording configuration information, the service mesh component can send the recording configuration information to the service mesh sidecar through a CDS channel, and can also send the recording configuration information to the service mesh sidecar through an RDS channel.

[0090] In this embodiment, by receiving the recording configuration information sent by the service mesh component and updating the historical configuration information in the local service cluster according to the recording configuration information to obtain the preset recording configuration information, the updating of the recording configuration information in the service mesh sidecar is realized, so that the service mesh sidecar uses the new recording configuration information to record the request stream and the response stream.

[0091] In one embodiment, the recording configuration information sent by the service mesh component is obtained from the service center by the service mesh component when the service mesh component detects that the configuration information of the service center has changed.

[0092] The service center refers to a platform that stores information associated with a service.

[0093] The configuration information refers to information stored in the service center and indicating a recording method.

[0094] Specifically, in response to a configuration setting operation, the service center obtains recording configuration information corresponding to the configuration setting operation, and then updates the configuration information stored in the service center to the recording configuration information. The service mesh component monitors the service center. When the service mesh component detects that the configuration information of the service center has changed to the recording configuration information, the service mesh component reads the recording configuration information in the service center and converts the recording configuration information into a service model. The service model refers to an object of a service.

[0095] In the embodiment, when it is detected that the configuration information of the service center is changed, the recording configuration information is obtained from the service center through the service mesh component, the self-defined configuration and flexible update of the recording configuration information are realized, and the recording configuration information can be transmitted to the service mesh sidecar, so that the service mesh sidecar executes the subsequent traffic recording step based on the recording configuration information.

[0096] In one embodiment, the step S204 of sending the recorded request stream and the recorded response stream to the traffic middle station asynchronously includes the following content: according to a preset transmission protocol, the recorded request stream and the recorded response stream are sent to the traffic middle station through bypass; the preset transmission protocol is any one of a user datagram protocol and a hypertext transfer protocol.

[0097] The preset transmission protocol refers to a communication protocol for transmitting the recorded data stream, for example, the preset transmission protocol can be a user datagram protocol (UDP) or a hypertext transfer protocol (HTTP).

[0098] The bypass is a communication mode, which can realize remote communication without affecting the original communication network between services.

[0099] Specifically, after obtaining the recorded request stream and the recorded response stream, the service mesh sidecar can send the recorded request stream and the recorded response stream to the traffic middle station through bypass according to UDP or HTTP. It should be noted that if the asynchronous transmission mode such as bypass is not used, but the default main road transmission mode in communication is used, the recorded request stream and the recorded response stream will occupy the resources in the main road, affect the communication and service processing of the service mesh sidecar, and further cause the performance of the service mesh sidecar to decrease.

[0100] In the embodiment, by sending the recorded request stream and the recorded response stream to the traffic middle station through bypass according to the preset transmission protocol, the pair recording of the recorded request stream and the recorded response stream is realized, the obtained traffic is more complete, the quality of traffic recording and the quality of subsequent traffic playback processing can be improved, and by transmitting the recorded request stream and the recorded response stream through bypass, the data processing efficiency of the service mesh sidecar and related services caused by the recorded request stream and the recorded response stream occupying the resources in the main road can be avoided, so that the data processing performance of the service mesh sidecar and related services is further improved.

[0101] In an embodiment, the traffic recording method described above further includes the following: adding a recording identifier to a service communication protocol in the network; in the case where it is detected that the received request stream carries the recording identifier in the corresponding service communication protocol, recording all requests in the request link associated with the request stream according to the recording configuration information to obtain a recorded request link; in the case where it is detected that the received response stream carries the recording identifier in the corresponding service communication protocol, recording all responses in the response link associated with the response stream according to the recording configuration information to obtain a recorded response link; and asynchronously sending the recorded request link and the recorded response link to the traffic platform.

[0102] The request link refers to a transmission path between the transmission of a request from an upstream service to a downstream service.

[0103] The response link refers to a transmission path between the transmission of a response from a downstream service to an upstream service.

[0104] The recorded request link refers to data obtained by recording each request in the request link.

[0105] The recorded response link refers to data obtained by recording each response in the response link.

[0106] If the complete link needs to be recorded, full-link recording information can be set in the recording configuration information, or the corresponding full-link recording instruction can be generated by the service mesh component and sent to the service mesh sidecar, and then the service mesh sidecar adds a recording identifier in the service communication protocol. Specifically, the service mesh sidecar receives a request stream, then performs network sniffing on the request stream according to a protocol filter to obtain a service communication protocol corresponding to the request stream, and in the case where it is detected that the service communication protocol carries the recording identifier, records the request stream according to the recording configuration information to obtain a recorded request stream. Since the communication protocol will be transmitted along the communication link of the request stream, the service communication protocol corresponding to all requests on the request stream link carries the recording identifier, and thus all requests on the request stream link will be recorded. The recorded request corresponding to all requests on the request stream link is taken as a recorded request link. Similarly, since the communication protocol will be transmitted along the communication link of the response stream, the service communication protocol corresponding to all requests on the response stream link carries the recording identifier, and thus all responses on the response stream link will be recorded. The recorded response corresponding to all responses on the response stream link is taken as a recorded response link.

[0107] Further, the post-recording request link can be sent to the traffic middle station asynchronously first after being acquired, and then the post-recording response link can be sent to the traffic middle station asynchronously after being acquired. The post-recording request link and the post-recording response link can also be sent to the traffic middle station asynchronously together after both of them are acquired. The traffic middle station receives and stores the post-recording request link and the post-recording response link, and can also convert the post-recording request link and the post-recording response link into visualized traffic in combination with the protocol file.

[0108] In this embodiment, a recording identifier is added to a service communication protocol in a network. In a case where it is detected that the recording identifier is carried in a service communication protocol corresponding to a received request stream, all requests in a request link associated with the request stream are recorded according to recording configuration information, to obtain a post-recording request link. In a case where it is detected that the recording identifier is carried in a service communication protocol corresponding to a received response stream, all responses in a response link associated with the response stream are recorded according to the recording configuration information, to obtain a post-recording response link. The post-recording request link and the post-recording response link are sent to a traffic middle station asynchronously, so that full-link paired recording of requests and responses is implemented, complete full-link traffic can be obtained, and therefore the quality of traffic recording is greatly improved, and the quality of subsequent traffic playback processing is also improved.

[0109] In one embodiment, the traffic recording method described above further includes the following content: generating a test case according to the visualized traffic; and performing stress testing on the test case according to a simulation server.

[0110] The test case refers to data for performing a test task on a specific software product.

[0111] The simulation server refers to a tool for building a virtual simulation object. For example, the simulation server can be a MockServer.

[0112] Specifically, the service mesh sidecar can use a tool to convert the visualized traffic in the traffic middle station into a test case, send the test case to the simulation server, and then perform stress testing or regression testing in multiple test scenarios through the test case and the simulation server.

[0113] In this embodiment, the test case is generated according to the visualized traffic, and stress testing is performed on the test case according to the simulation server, so that the application of the visualized traffic in the test scenario is implemented. Since the visualized traffic contains complete requests and responses, the test case generated based on the visualized traffic has more accurate test effects, and the accuracy of the test is improved.

[0114] In one embodiment, as Figure 4As shown, another traffic recording method is provided, and the method is applied to Figure 1 The service mesh sidecar in the method is taken as an example for illustration, and the method includes the following steps:

[0115] In step S401, recording configuration information sent by a service mesh component is received, and historical configuration information in a local service cluster is updated according to the recording configuration information to obtain preset recording configuration information.

[0116] In step S402, a request stream sent by an upstream service is received.

[0117] In step S403, network sniffing is performed on the request stream according to a protocol filter to obtain a service communication protocol corresponding to the request stream, and the service communication protocol is decoded according to the protocol filter to obtain header information of the service communication protocol.

[0118] In step S404, whether the request stream meets recording conditions is detected according to the header information.

[0119] In step S405, in a case where it is detected that the request stream sent by the upstream service meets the recording conditions, the request stream is recorded according to the preset recording configuration information to obtain a recorded request stream.

[0120] In step S406, a recording identifier is added to the request stream, and the request stream carrying the recording identifier is sent to a corresponding downstream service.

[0121] In step S407, a response stream returned by the downstream service based on the request stream carrying the recording identifier is recorded according to the recording configuration information to obtain a recorded response stream.

[0122] In step S408, the recorded request stream and the recorded response stream are sent to a traffic middle platform through a bypass according to a preset transmission protocol, so that the traffic middle platform converts the recorded request stream and the recorded response stream into visualized traffic.

[0123] The traffic recording method can achieve the following beneficial effects: not only does it not need to intrude into the upstream service and the downstream service, but also it realizes recording of the request stream and paired recording of the response stream corresponding to the request stream through the recording identifier, so that the obtained traffic is more complete, and asynchronous transmission of the recorded request stream and the recorded response stream can avoid affecting business processing of the upstream service and the downstream service, thereby greatly improving the performance of traffic recording.

[0124] In order to more clearly illustrate the traffic recording method provided by the embodiments of the present disclosure, the traffic recording method is specifically described below with reference to one specific embodiment. As shown in Figure 5As shown, a timing diagram of another traffic recording method is provided, assuming that a service mesh component is implemented through pilot and a service mesh sidecar is implemented through envoy; in the recording configuration information updating phase, the following contents are specifically included:

[0125] (1) pilot listens to update: when the pilot listens to the configuration information update of the service center to the recording configuration information, the recording configuration information is obtained from the service center, and the recording configuration information is converted into a service model.

[0126] (2) xDS protocol push and envoy update cluster information: pilot sends the recording configuration information to the main thread of envoy through the xDS protocol channel.

[0127] (3) The main thread of envoy instructs all worker threads of envoy to update the historical configuration information in the locally cached cluster information.

[0128] In the traffic recording phase, the following contents are specifically included:

[0129] (4) The upstream service sends a request to the downstream service through a private rpc protocol, and the service mesh proxies the request to the worker thread of envoy.

[0130] (5) Process the request through the protocol filter: the worker thread of envoy sniffs out the protocol type of the private rpc protocol through the protocol filter, and encodes and decodes the private rpc to obtain header information.

[0131] (6) Mark the request and send it to the downstream service: the worker thread of envoy determines whether the request meets the recording condition according to the header information; in the case that the request meets the recording condition, the worker thread of envoy adds a recording identifier to the request and sends the request carrying the recording identifier to the downstream service.

[0132] (7) Record the request: in the case that the request meets the recording condition, the worker thread of envoy records the request according to the recording configuration information in the locally cached cluster information, obtains the recorded request, and sends the recorded request to the traffic platform asynchronously.

[0133] (8) Receive response: the downstream service returns the response generated based on the request carrying the recording identifier to the upstream service through the private rpc protocol, the service mesh proxies the response to envoy, and the worker thread of envoy receives the response.

[0134] (9) Recording response: The working thread of the envoy performs recording processing on the response according to the recording configuration information in the locally cached cluster information, obtains a recorded response, and asynchronously sends the recorded response to the traffic middle platform, so that the traffic middle platform encodes and decodes the recorded request and the recorded response to obtain visualized traffic.

[0135] In the embodiment, by recording the request and the response of the private rpc protocol, zero-invasion of the service in the traffic recording process is realized, the traffic recording capability of the service mesh is enhanced, and paired recording of the request stream and the response stream corresponding to the request stream is also realized, so that the obtained traffic is more complete. Since the traffic contains complete requests and responses, the traffic has more accurate test effect, and the test accuracy of the test case generated based on the traffic is improved.

[0136] It should be understood that, although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0137] In one embodiment, a computer device is provided, which can be a service mesh, and an internal structure diagram thereof can be as shown in Figure 6 The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as request stream, response stream, recorded request stream and recorded response stream. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a traffic recording method.

[0138] Those skilled in the art can understand that Figure 6 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] In one embodiment, a computer device is also provided, including a memory and a processor, the memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0140] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0141] In one embodiment, a computer program product is provided, which includes a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0142] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant national and regional laws, regulations and standards.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0144] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0145] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A flow recording method, characterized by, The method comprises: In the case that it is detected that the request stream sent by the upstream service meets the recording condition, recording processing is performed on the request stream according to preset recording configuration information, to obtain a recorded request stream; the recording configuration information is used at least to represent a recording mode; A recording identifier is added to the request stream, and the request stream carrying the recording identifier is sent to the corresponding downstream service; According to the recording configuration information, recording processing is performed on the response stream returned by the downstream service based on the request stream carrying the recording identifier, to obtain a recorded response stream; The recorded request stream and the recorded response stream are sent to a traffic middle station asynchronously through bypass communication, so that the traffic middle station converts the recorded request stream and the recorded response stream into visualized traffic; In the case that the recording mode represented by the recording configuration information is full-link recording, the recording identifier is added to the service communication protocol; the service communication protocol carrying the recording identifier is transmitted through the request link associated with the request stream and the response link associated with the response stream, so that the service communication protocol of all requests in the request link and all responses in the response link carries the recording identifier; in the case that it is detected that the service communication protocol corresponding to the request stream or the response stream carries the recording identifier, recording processing is performed on the request stream or the response stream based on the recording configuration information, to record all requests in the request link and all responses in the response link.

2. The method of claim 1, wherein, Before the recording processing is performed on the request stream according to the preset recording configuration information to obtain a recorded request stream in the case that it is detected that the request stream sent by the upstream service meets the recording condition, the method further comprises: Receiving the request stream sent by the upstream service; According to a protocol filter, network sniffing is performed on the request stream, to obtain a service communication protocol corresponding to the request stream; According to the protocol filter, decoding processing is performed on the service communication protocol, to obtain header information of the service communication protocol; According to the header information, it is detected whether the request stream meets the recording condition.

3. The method of claim 2, wherein, Before the request stream sent by the upstream service is received, the method further comprises: Receiving the recording configuration information sent by a service mesh component, and updating historical configuration information in a local service cluster according to the recording configuration information, to obtain the preset recording configuration information.

4. The method of claim 3, wherein, The recording configuration information sent by the service mesh component is obtained from a service center by the service mesh component when it is detected that the configuration information of the service center changes.

5. The method of claim 1, wherein, The recorded request stream and the recorded response stream are sent to a traffic middle station asynchronously, comprising: According to a preset transmission protocol, the recorded request stream and the recorded response stream are sent to the traffic middle station through bypass; the preset transmission protocol is any one of user datagram protocol and hypertext transfer protocol.

6. The method according to any one of claims 1 to 5, characterized in that, After the recording processing is performed on all requests in the request link, the method further comprises: A recorded request link is obtained. After recording processing all responses in the response link, further comprising: Obtaining the recorded response link; The method further comprises: Asynchronously sending the recorded request link and the recorded response link to the traffic middle station through bypass communication.

7. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: According to the visual traffic, generating a test case; According to the simulation server, performing stress testing on the test case.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to realize the steps of the method in any one of claims 1 to 7.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 7.

Images