Traffic mirroring method, device and equipment

[0019] In order to achieve the purpose of this specification, the embodiments of this specification provide a traffic mirroring method, device, and equipment, which obtain traffic data from the service extension implementation layer of the target data plane proxy. The traffic data includes request traffic data and/or response traffic data. According to the traffic data, determine the target mirroring strategy corresponding to the traffic data, and then, based on the determined target mirroring strategy corresponding to the traffic data, mirror the traffic data to generate mirror data corresponding to the traffic data, and mirror data corresponding to the traffic data Forward to the mirror pipeline, so that the data analysis server obtains the mirrored data from the mirror pipeline, and analyzes and processes the mirrored data. In this way, the traffic data is obtained from the service expansion implementation layer of the target data plane agent, avoiding the above-mentioned process of obtaining traffic data Among them, it is difficult to reorganize and restore data, to extract business data, and cannot be parsed in the case of encrypted communication, thereby reducing the consumption of computing resources in the process of obtaining traffic data.

[0020] In order to enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification in conjunction with the drawings in the embodiments of this specification. Obviously, the described The embodiments are only a part of the embodiments in this specification, rather than all the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this specification.

[0021] Such as figure 1 As shown, the embodiment of this specification provides a traffic mirroring method. The execution subject of the method can be a mirror filter mounted on the Proxy layer processing link on MOSN, and the mirror filter can proxy services from the target data plane. Extend the implementation layer to obtain traffic data, etc. The method can obtain flow data from the service expansion realization layer of the target data plane agent, perform mirror processing on the above flow data, generate mirror data corresponding to the flow data, and forward the mirror data corresponding to the flow data to the mirroring pipeline. The method may specifically include the following steps:

[0022] In step S102, the traffic data is obtained from the service expansion implementation layer of the target data plane agent, and the traffic data includes request traffic data and/or response traffic data.

[0023] Among them, the above-mentioned target data plane agent can be a data plane agent developed using a certain programming language, such as a Service Mesh data plane agent (MOSN, Modular Observable Smart Network) developed using Golang programming language, etc. The function and positioning of the data plane agent can be Similar to Envoy, it aims to provide distributed, modular, observable, and intelligent proxy capabilities. The foregoing service extension implementation layer may be a service extension implementation layer for a certain service set according to actual conditions, and may specifically be a proxy layer. If the target data plane proxy is a Service Mesh data plane proxy developed using the Golang programming language, and the business extension implementation layer is a proxy proxy layer, the above data plane proxy can be deployed in the same container as the application in a sidecar manner to complete the application RPC (Remote Procedure Call Protocol, remote procedure call protocol) communication transparent proxy. The above-mentioned target data plane agent and the application program can communicate locally, and different application programs can carry out network communication through the data plane agent. The above-mentioned target data plane agent may be the data plane agent that provides traffic data in the embodiment of this specification. The above-mentioned target data plane agent may be located in the same container as the server application process, or may be located in the same container as the requesting application process. The embodiment of this specification does not specifically limit which application process the target data plane agent and the application process are set in the same container.

[0024] In the implementation, the implementation logic of the target data plane agent and the business extension implementation layer can be written in advance through a predetermined programming language (such as Golang programming language or C language, etc.). After the writing is completed, the business extension implementation layer of the target data plane agent can be extended Put into use. The business expansion implementation layer of the target data plane agent can collect and process traffic data. When the flow data needs to be mirrored, the flow data can be obtained from the service expansion implementation layer of the target data plane agent.

[0025] In order to better illustrate the above processing process, the following takes the target data plane proxy as the Service Mesh data plane proxy developed using the Golang programming language, and the service extension implementation layer as the proxy proxy layer as an example for detailed description. Such as figure 2 As shown, figure 2 The data plane proxy MOSN proxy application processing process provided in the embodiment of this specification, the data plane proxy MOSN is essentially a 4-7 layer proxy, which can be deployed in the same way as a sidecar and a user program in the form of an independent process. In a physical machine or a VM (Virtual Machine, virtual machine), or in the form of an independent gateway, it can run on a host or a virtual machine alone.

[0026] Such as figure 2 As shown, MOSN 1 can be deployed in the same pod1 with the client application process (such as caller application process), and MOSN2 can be deployed in the same pod2 with the server application process (such as target application process). When sending a request message, a forward request link can include the following steps: the caller application process can send the request information to MOSN1 through local communication. MOSN1 can proxy the service discovery, routing, load balancing and other capabilities of the caller application process mentioned above, and forward the request information of the caller application process to the upstream MOSN2 (such as MOSN2 deployed in the same pod2 as the target application process) and upstream MOSN2 through protocol conversion. Translate the received request through protocol conversion and send it to the agent's target application on process. The reverse link is similar. Through the above proxy process, the data plane proxy MOSN proxy the request between the caller application process and the target application process.

[0027] Such as image 3 As shown, the data plane proxy MOSN in order to forward data, MOSN divides the overall function into four layers, namely NetworkNetwork (network communication layer), Protocol (protocol processing layer), Stream (session flow processing layer) and Proxy (business extension implementation layer) , Each layer can be used to complete independent functions, and layers can cooperate with each other to achieve complete Proxy forwarding.

[0028] Specific, such as image 3 As shown, the MOSN working process can include the following processes: Step A: When MOSN receives data from the application process, that is, when the data arrives at MOSN, MOSN can read the data at the Network layer, and then send the data to the Protocol layer through readfilter. Decoding; Step B: The Protocol layer can call back the decoded data to the stream layer for data creation and encapsulation processing; Step C: The Stream layer calls back the created and encapsulated data to the Proxy layer; Step D: Proxy layer can Send the received data to the stream layer for processing; Step E: The stream layer sends the processed data to the Protocol layer corresponding to the protocol of the above data, and re-encode the data; Step F: The Protocol layer will be encoded The latter data is sent to the Network layer through the write filter, and the data is sent out through the Network layer.

[0029] MOSN provides an extension mechanism for Stream Filter. In the embodiments of this specification, the process of mirroring the acquired traffic data can be implemented at the Proxy layer of MOSN through the Stream filter extension mechanism. The aforementioned Stream filter extension mechanism can be enabled on the server. Alternatively, it can also be turned on on the client side, mirroring the acquired traffic data by mounting a mirror filter on the Proxy layer processing link, and then forwarding the mirrored data obtained after the mirroring process to the message queue, so that the analysis is in progress The station can obtain the mirrored data from the above message queue, and analyze and process the mirrored data.

[0030] In implementation, at present, the analysis of mirrored data obtained by mirroring traffic data plays a very important role in business security risk insights, system security vulnerabilities and intrusion detection, and monitoring of data security asset flows. However, due to the above process of obtaining traffic data, the host system-level traffic mirroring method is adopted. By capturing data packets on the host network interface, it is difficult to reorganize and restore data, difficult to extract business data, and cannot be resolved in the case of encrypted communication. The problem of excessive consumption of host computing resources. In the application gateway-level traffic mirroring method, the seven-layer network traffic mirroring has the problem that the service data limited to the gateway access protocol cannot support the service data that is called and transferred internally. In the communication device-level traffic mirroring method, the four-layer network traffic mirroring, the physical layer/network layer/transport layer data packet exchange data mirroring, data reorganization and restoration is difficult, business data extraction is difficult, and it cannot be resolved in the case of encrypted communication. The problem of huge consumption of equipment computing resources. Therefore, it is necessary to provide a technical solution that consumes less computing resources when acquiring traffic data. For details, refer to the following content.

[0031] In the specific implementation process, the client sends a request to the server, and the server responds to the client as an example. MOSN1 and MOSN2 can be deployed on the client and server respectively, such as Figure 4 As shown, MOSN1 can be deployed in the same pod1 with the user application process (such as the caller application process) in a sidecar manner, and MOSN2 can be deployed in the same pod1 with the server application process (such as the target application process) in a sidecar manner In the same pod2. Among them, the target data plane agent can be MOSN1 or MOSN2. In this embodiment, MOSN2, which is deployed in the same pod2 as the server application process, is the target data plane proxy, and the mirror filter is mounted on the Proxy layer processing link in MOSN2 through the Stream filter extension mechanism, and the mirror filter is passed Obtain the traffic data on the Proxy layer in MOSN2. The traffic data includes request traffic data and/or response traffic data, such as Figure 4 As shown, the above-mentioned mirror filter includes a receiver filter and a sender filter. The receiver filter can be used to receive request traffic data, and the sender filter can receive response traffic data.

[0032] Specific, such as Figure 4 As shown, taking the client sending a request to the server as an example, the caller application process (caller application process) deployed on the client can send an order query request to the server application process (target application process), and the order query request carries With the order ID of the order to be queried, the calling application process sends the order query request through local communication to MOSN1, which is deployed in the same pod as the calling application process. After receiving the order query request, MOSN1 executes the above image 3 As shown in the processing process, the data processed in step F is sent to the Network layer of MOSN2 through the Network layer of MOSN1. When the order request information reaches the Network layer of MOSN2, it passes image 3 The processing process shown, that is, after the Network layer, protocol (protocol processing layer), Stream (session flow processing layer) of MOSN2, and reaching the Proxy (business extension realization layer), on the one hand, it is mounted on the Proxy layer processing link The mirror filter can obtain request traffic data (such as the order query request sent by the caller) through the receiver filter. on the other hand. After the above-mentioned order query request arrives at the Proxy (business extension implementation layer), it will go through the Proxy (business extension implementation layer), Stream (session stream processing layer), and protocol (protocol processing layer) in the downward direction. Until the Network (network communication layer) of MOSN2, the order query request sent by the caller is sent to the server application process through the Network (network communication layer) of MOSN2.

[0033] After the server application process receives the order query request sent by the caller application process, it searches for the order details corresponding to the order request, and then sends the found order details as response information to the same deployment as the server application process via local communication. MOSN2 in a pod, after MOSN2 receives the response information, execute the above image 3 As shown in the processing process, the response information passes through the Network (network communication layer), protocol (protocol processing layer), and Stream (session stream processing layer) to the Proxy (business extension implementation layer) in turn, and the response information reaches the MOSN2 After Proxy (Business Expansion Implementation Layer), on the one hand, the mirror filter mounted on the Proxy layer processing link can obtain response traffic data (such as the order details sent by the server) through the senderfilter. On the other hand, after the above response information arrives at the Proxy (business extension implementation layer), the response information will follow the downward direction, that is, from the Proxy (business extension implementation layer) through Stream (session flow processing layer), Protocol (protocol) The processing layer) is sent to the MOSN2 Network (network communication layer), and then the response information sent by the server is sent to MOSN1 through the MOSN2 Network (network communication layer), and finally, the response information is sent to the calling application through MOSN1 process.

[0034] In step S104, according to the flow data, a target mirroring strategy corresponding to the flow data is determined.

[0035] Among them, the above-mentioned target mirroring strategy may be the protocol type, data type, traffic trace identification, caller application name, server application name, application interface name, traffic data source IP address, traffic data destination IP address, source The mirroring strategy formulated by one or more attributes of port, target port, request/response header, request/response body, and request/response tail. For example, the above target mirroring strategy can be that the protocol type of the traffic data is A1, the calling application name is B1, the server application name is C1, the application interface name is D1, the traffic data source IP address is E1, and the traffic data destination IP address is G1's request flow data and response flow data mirroring strategy.

[0036] In implementation, the mirror filter mounted on the Proxy layer processing link on MOSN2 obtains traffic data from the service expansion implementation layer of the target data plane proxy through the above S102 processing, which can be based on the protocol type and data type of the acquired traffic data , Traffic trajectory identification, caller application name, server application name, application interface name, traffic data source IP address, traffic data destination IP address, source port, target port, request/response header, request/response body, request/response The tail and other information determine the target mirroring strategy corresponding to the traffic data.

[0037] In step S106, based on the determined target mirroring strategy corresponding to the traffic data, mirroring processing is performed on the traffic data to generate mirroring data corresponding to the traffic data.

[0038] In implementation, the mirror filter mounted on the Proxy layer processing link on MOSN2 determines the target mirroring strategy corresponding to the traffic data according to the flow data through the above S104 processing, and can be based on the target mirroring strategy corresponding to the determined traffic data. The data is mirrored to generate mirrored data corresponding to the traffic data.

[0039] Specifically, for example, the protocol type for traffic data is A1, the calling application name is B1, the server application name is C1, the application interface name is D1, the traffic data source IP address is E1, and the traffic data destination IP address is G1 Request traffic data, and the determined target mirroring strategy is X1. For the response flow data whose protocol type is A1, the calling application name is B1, the server application name is C1, the application interface name is D1, the traffic data source IP address is E, and the traffic data destination IP address is G1, confirm The target mirroring strategy is X2. For the request traffic data whose protocol type is A2, the calling application name is B2, the server application name is C2, the application interface name is D2, the traffic data source IP address is E2, the traffic data destination IP address is G2, confirm The target mirroring strategy is X3. Then, based on the target mirroring strategy corresponding to the determined traffic data, mirroring processing can be performed on the traffic data to generate mirroring data corresponding to the traffic data.

[0040] In step S108, the mirror data corresponding to the flow data is forwarded to the mirror pipe, so that the data analysis server obtains the mirror data from the mirror pipe, and analyzes and processes the mirror data.

[0041] Among them, the above-mentioned mirroring pipe may be a message queue or a log pipe.

[0042] In the implementation, such as Figure 4 As shown, after the mirror filter mounted on the Proxy layer processing link on MOSN2 generates the mirrored data corresponding to the traffic data through the above S106 processing, the mirrored data corresponding to the traffic data can be forwarded to the mirror pipe to make the data analysis server Or the analysis center obtains the mirrored data from the mirror pipeline, the data analysis server or the analysis center can perform flink streaming data preprocessing on the acquired mirrored data, and use the processed data for business/system/data security analysis and detection.

[0043] As can be seen from the technical solutions provided by the above embodiments of this specification, the embodiments of this specification obtain traffic data from the service expansion implementation layer of the target data plane agent. The traffic data includes request traffic data and/or response traffic data. The traffic is determined according to the traffic data. The target mirroring strategy corresponding to the data, and then, based on the determined target mirroring strategy corresponding to the traffic data, the traffic data is mirrored, the mirroring data corresponding to the traffic data is generated, and the mirroring data corresponding to the traffic data is forwarded to the mirroring pipeline to make The data analysis server obtains the mirrored data from the mirroring pipeline, and analyzes and processes the mirrored data. In this way, the traffic data is obtained from the service expansion implementation layer of the target data plane agent, which avoids the difficulty of data reorganization and restoration in the process of obtaining the traffic data. Large, business data extraction is difficult, and cannot be analyzed in the case of encrypted communication, thereby reducing the consumption of computing resources in the process of obtaining traffic data.

[0044] Among them, such as Figure 5 As shown, the specific processing methods of the above S104 can be various, and an optional processing method is provided below. For details, please refer to the processing of S1042-S1046 below.

[0045] In step S1042, keywords included in the traffic data are extracted.

[0046] Among them, the above keywords can be the protocol type of traffic data, traffic data type, caller application name, server application name, application interface name, traffic data source IP address, traffic data destination IP address, source port, destination port, request Header or response header, request body or response body, request tail or response tail, etc.

[0047] In step S1044, the keywords are matched with preset keywords included in the mirroring strategy in the predetermined mirroring strategy set.

[0048] Wherein, the aforementioned predetermined mirroring strategy set may be a mirroring strategy set composed of a mirroring strategy formulated in advance for the traffic data that needs to be mirrored according to data analysis requirements. The foregoing predetermined set of mirroring strategies may include at least one mirroring strategy. For example, the above-mentioned mirroring strategy set may include an X mirroring strategy, a Y mirroring strategy, and a Z mirroring strategy. The above-mentioned X mirroring strategy may be a strategy for mirroring a request from interface A, and the Y mirroring strategy may be for mirroring a response from interface B. The Z mirroring strategy may be a mirroring strategy for the response of interface C, etc. The mirroring strategy in the mirroring strategy set in this embodiment may also include the X mirroring strategy, the Y mirroring strategy and the Z mirroring strategy in the above-mentioned mirroring strategy set. After the combination, the E mirroring strategy is formed according to the priority order of the strategy execution. For example, the E mirroring strategy can be the X mirroring strategy first, then the Y mirroring strategy, and the Z mirroring strategy finally.

[0049] The above-mentioned preset keywords may include the protocol type with the above-mentioned mirrored data, the type of mirrored data, the name of the caller application, the name of the server application, the name of the application interface, the source IP address of the mirrored data, the destination IP address of the mirrored data, the source port, and the destination port. , Request header or response header, request body or response body, request tail or response tail, etc.

[0050] The policy attributes of the mirroring strategy in the aforementioned predetermined mirroring strategy set may include one or more of mirroring object, mirroring type, mirroring mode, and mirroring agreement. The aforementioned mirroring object may include one or more of a mirroring application, a mirroring interface, and a mirroring method. The type of mirroring may include supporting request flow data mirroring, or, supporting response flow data mirroring, or, supporting request flow data and response flow data mirroring in pairs. The mirroring mode may include mirroring according to the amount of mirrored traffic within a preset time period, or random mirroring may be performed according to a preset traffic ratio. The mirroring protocol can include http rpc or a custom rpc protocol. The mirroring policy attribute may also include the upper limit of the size of a single flow of mirroring.

[0051] In implementation, after the mirror filter mounted on the Proxy layer processing link on MOSN2 extracts the keywords contained in the traffic data through the above S1042, the extracted keywords can be combined with the preset mirroring strategy included in the predetermined mirroring strategy set. Set keywords to match.

[0052] In step S1046, the mirroring strategy corresponding to the preset keyword matching the keyword is determined as the target mirroring strategy corresponding to the traffic data.

[0053] In implementation, if the mirror filter mounted on the Proxy layer processing link on MOSN2 extracts the keyword contained in the traffic data through the above S1042, the protocol type of the traffic data is A1, the calling application name is B1, and the server application name It is C1, the application interface name is D1, the traffic data source IP address is E1, and the traffic data destination IP address is G1. A certain mirroring strategy X1 in the predetermined mirroring strategy set includes the preset keywords: the protocol type of the mirroring data is A1, the caller application name is B1, the server application name is C1, the application interface name is D1, and the mirroring data source IP address If it is E1 and the target IP address of the mirrored data is G1, it can be determined that the mirroring strategy X1 is the target mirroring strategy corresponding to the above-mentioned traffic data.

[0054] Further, the aforementioned target mirroring strategy includes a mirroring strategy corresponding to the amount of mirrored traffic data within a preset time and a mirroring strategy corresponding to a preset ratio of the preset mirroring traffic amount to the total amount of traffic data. Among them, such as Image 6 As shown, the specific processing methods of S106 above can be various, and an optional processing method is provided below. For details, please refer to the processing of S1062 below.

[0055] In step S1062, the traffic data is mirrored based on a mirroring strategy corresponding to the amount of mirrored traffic data within a preset time, or according to a mirroring strategy corresponding to a preset ratio of the amount of mirrored traffic to the total amount of traffic data To generate mirrored data.

[0056] Among them, the mirror filter mounted on the Proxy layer processing link on MOSN2 can also include a statistics module, which can be used to count the number of request traffic and response traffic entering the mirror filter, and the request traffic that is mirrored The number and the number of response traffic being mirrored.

[0057] In the implementation, the number of request traffic and the number of response traffic entering the mirror filter within the preset time period calculated by the above statistical module, as well as the number of request traffic being mirrored and the number of response traffic being mirrored, can be set to mirror within a predetermined time period. A mirroring strategy corresponding to the amount of requested traffic, or a mirroring strategy corresponding to the amount of mirroring response traffic within a predetermined period of time, or a mirroring strategy corresponding to the amount of random mirroring traffic within a predetermined period of time, the above random amount of mirroring traffic may be random Mirror the mirroring strategy corresponding to the request traffic and or the response traffic, and then mirror the aforementioned traffic data according to the aforementioned mirroring strategy to generate mirrored data.

[0058] In the implementation, the number of request traffic and the number of response traffic entering the mirror filter within the preset time period calculated by the above statistics module, as well as the number of request traffic being mirrored and the number of response traffic being mirrored, can be preset The preset ratio of the amount of mirrored traffic to the total amount of traffic data sets a corresponding mirroring strategy, and then mirrors the above-mentioned traffic data according to the above-mentioned mirroring strategy to generate mirrored data.

[0059] Further, such as Figure 7 As shown, before the above step S1042, the following processing procedures of S110-S112 may also be included.

[0060] In step S110, it is detected whether there is a predetermined mirroring strategy set, and the predetermined mirroring strategy set includes at least one mirroring strategy.

[0061] In step S112, when there is a predetermined mirroring strategy set, the flow data is filtered to filter out the simulated flow and/or the flow data of the pressure measurement flow in the flow data to obtain filtered flow data.

[0062] As can be seen from the technical solutions provided by the above embodiments of this specification, the embodiments of this specification obtain traffic data from the service expansion implementation layer of the target data plane agent. The traffic data includes request traffic data and/or response traffic data. The traffic is determined according to the traffic data. The target mirroring strategy corresponding to the data, and then, based on the determined target mirroring strategy corresponding to the traffic data, the traffic data is mirrored, the mirroring data corresponding to the traffic data is generated, and the mirroring data corresponding to the traffic data is forwarded to the mirroring pipeline to make The data analysis server obtains the mirrored data from the mirroring pipeline, and analyzes and processes the mirrored data. In this way, the traffic data is obtained from the service expansion implementation layer of the target data plane agent, which avoids the difficulty of data reorganization and restoration in the process of obtaining the traffic data. Large, business data extraction is difficult, and cannot be analyzed in the case of encrypted communication, thereby reducing the consumption of computing resources in the process of obtaining traffic data.

[0063] Corresponding to the traffic mirroring method provided in the foregoing embodiment, based on the same technical concept, the embodiment of the present invention also provides a traffic mirroring device. Figure 5 This is a schematic diagram of the first module composition of a traffic mirroring device provided by an embodiment of the present invention. The traffic mirroring device is used to execute Figure 1 to Figure 4 Describe the traffic mirroring method, such as Picture 8 As shown, the traffic mirroring device includes:

[0064] The data acquisition module 801 acquires traffic data from the service expansion implementation layer of the target data plane agent, where the traffic data includes request traffic data and/or response traffic data;

[0065] The strategy determination module 802 determines the target mirroring strategy corresponding to the traffic data according to the traffic data;

[0066] The data generating module 803, based on the determined target mirroring strategy corresponding to the traffic data, performs mirroring processing on the traffic data to generate mirroring data corresponding to the traffic data;

[0067] The data forwarding module 804 forwards the mirrored data corresponding to the traffic data to the mirrored pipe, so that the data analysis server obtains the mirrored data from the mirrored pipe, and analyzes and processes the mirrored data.

[0068] Optionally, the strategy determination module includes:

[0069] An extraction unit to extract keywords contained in the traffic data;

[0070] A matching unit, matching the keywords with preset keywords included in a mirroring strategy in a predetermined mirroring strategy set;

[0071] The determining unit determines the mirroring strategy corresponding to the preset keyword that matches the keyword as the target mirroring strategy corresponding to the traffic data.

[0072] Optionally, the target mirroring strategy includes a mirroring strategy corresponding to the number of mirrored traffic data within a preset time and a mirroring strategy corresponding to a preset ratio of the preset mirroring traffic to the total amount of traffic data;

[0073] The data generation module performs a mirroring strategy on the traffic data based on a mirroring strategy corresponding to the amount of mirrored traffic data within a preset time, or according to a mirroring strategy corresponding to a preset ratio of the amount of mirrored traffic to the total amount of traffic data Mirror, generate mirror data.

[0074] Optionally, the target data plane agent is deployed in the same container as the application in a sidecar manner.

[0075] Optionally, local communication is performed between the target data plane agent and the application program, and network communication is performed between different applications through the target data plane agent.

[0076] Optionally, the device further includes:

[0077] A strategy detection module, before extracting the keywords of the mirror data, detect whether there is a predetermined mirror strategy set, and the predetermined mirror strategy set includes at least one mirror strategy;

[0078] The filtering processing module, when the predetermined mirroring policy set exists, performs filtering processing on the flow data to filter out the flow data of the simulated flow and/or the pressure measurement flow in the flow data, and obtain the filtered Traffic data.

[0079] Optionally, the target data plane proxy is a Service Mesh data plane proxy developed using the Golang programming language, and the service extension implementation layer is a proxy proxy layer.

[0080] Optionally, the policy attributes of the mirroring policy in the predetermined mirroring policy set include one or more of a mirroring object, a mirroring type, a mirroring mode, and a mirroring agreement.

[0081] As can be seen from the technical solutions provided by the above embodiments of this specification, the embodiments of this specification obtain traffic data from the service expansion implementation layer of the target data plane agent. The traffic data includes request traffic data and/or response traffic data. The traffic is determined according to the traffic data. The target mirroring strategy corresponding to the data, and then, based on the determined target mirroring strategy corresponding to the traffic data, the traffic data is mirrored, the mirroring data corresponding to the traffic data is generated, and the mirroring data corresponding to the traffic data is forwarded to the mirroring pipeline to make The data analysis server obtains the mirrored data from the mirroring pipeline, and analyzes and processes the mirrored data. In this way, the traffic data is obtained from the service expansion implementation layer of the target data plane agent, avoiding the difficulty of data reorganization and restoration in the process of obtaining the traffic data. Large, business data extraction is difficult, and cannot be analyzed in the case of encrypted communication, thereby reducing the consumption of computing resources in the process of obtaining traffic data.

[0082] The traffic mirroring device provided in the embodiment of the present specification can implement each process in the embodiment corresponding to the foregoing traffic mirroring method. In order to avoid repetition, details are not described herein again.

[0083] It should be noted that the traffic mirroring device provided in the embodiment of this specification and the traffic mirroring method provided in the embodiment of this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned traffic mirroring method, and the repetition will not be omitted. Repeat.

[0084] Corresponding to the traffic mirroring method provided in the foregoing embodiment, based on the same technical concept, the embodiment of the present invention also provides a traffic mirroring device, such as Picture 9 Shown. Picture 9 This is a schematic diagram of the hardware structure of a traffic mirroring device provided by an embodiment of the present invention. The traffic mirroring device is used to execute Figure 1-Figure 7 In the described traffic mirroring method, the traffic mirroring device can provide a service-side application server or a requesting-side application server for the foregoing embodiment.

[0085] Traffic mirroring devices may have relatively large differences due to different configurations or performances, and may include one or more processors 901 and a memory 902, and the memory 902 may store one or more storage applications or data. Among them, the memory 902 may be short-term storage or persistent storage. The application program stored in the memory 902 may include one or more modules (not shown in the figure), and each module may include a series of computer-executable instructions in the device for assigning tasks. Furthermore, the processor 901 may be configured to communicate with the memory 902, and execute a series of computer executable instructions in the memory 902 on the traffic mirroring device. The traffic mirroring device may also include one or more power supplies 903, one or more wired or wireless network interfaces 904, one or more input and output interfaces 905, and one or more keyboards 906.

[0086] Specifically, in this embodiment, the traffic mirroring device includes a memory and one or more programs. One or more programs are stored in the memory, and the one or more programs may include one or more modules, and each The module may include a series of computer-executable instructions in the device for distributing tasks, and the one or more programs configured to be executed by one or more processors include the following computer-executable instructions:

[0087] Acquiring traffic data from the service expansion implementation layer of the target data plane agent, where the traffic data includes request traffic data and/or response traffic data;

[0088] Determine the target mirroring strategy corresponding to the traffic data according to the traffic data;

[0089] Mirroring the traffic data based on the determined target mirroring strategy corresponding to the traffic data to generate mirroring data corresponding to the traffic data;

[0090] The mirror data corresponding to the flow data is forwarded to the mirror pipe, so that the data analysis server obtains the mirror data from the mirror pipe, and analyzes and processes the mirror data.

[0091] Optionally, the determining the target mirroring strategy corresponding to the traffic data according to the traffic data includes:

[0092] Extract keywords contained in the traffic data;

[0093] Matching the keywords with preset keywords included in a mirroring strategy in a predetermined mirroring strategy set;

[0094] The mirroring strategy corresponding to the preset keyword matching the keyword is determined as the target mirroring strategy corresponding to the traffic data.

[0095] Optionally, the target mirroring strategy includes a mirroring strategy corresponding to the amount of mirrored traffic data within a preset time and a mirroring strategy corresponding to a preset ratio of the preset mirroring traffic amount to the total amount of traffic data. The target mirroring strategy corresponding to the traffic data, performing mirroring processing on the traffic data to generate mirroring data corresponding to the traffic data, includes:

[0096] Based on a mirroring strategy corresponding to the number of mirrored traffic data within a preset time, or according to a mirroring strategy corresponding to a preset ratio of the preset number of mirrored traffic to the total amount of the traffic data, the traffic data is mirrored to generate mirrored data.

[0097] Optionally, the target data plane agent is deployed in the same container as the application in a sidecar manner.

[0098] Optionally, local communication is performed between the target data plane agent and the application program, and network communication is performed between different applications through the target data plane agent.

[0099] Optionally, before extracting the keywords of the mirror data, the method further includes:

[0100] Detecting whether there is a predetermined mirroring strategy set, and the predetermined mirroring strategy set includes at least one mirroring strategy;

[0101] In the case where the predetermined mirroring strategy set exists, filtering processing is performed on the flow data to filter out the flow data of the simulated flow and/or pressure measurement flow in the flow data to obtain filtered flow data.

[0102] Optionally, the target data plane proxy is a Service Mesh data plane proxy developed using the Golang programming language, and the service extension implementation layer is a proxy proxy layer.

[0103] Optionally, the policy attributes of the mirroring policy in the predetermined mirroring policy set include one or more of a mirroring object, a mirroring type, a mirroring mode, and a mirroring agreement.

[0104] As can be seen from the technical solutions provided by the above embodiments of this specification, the embodiments of this specification obtain traffic data from the service expansion implementation layer of the target data plane agent. The traffic data includes request traffic data and/or response traffic data. The traffic is determined according to the traffic data. The target mirroring strategy corresponding to the data, and then, based on the determined target mirroring strategy corresponding to the traffic data, the traffic data is mirrored, the mirroring data corresponding to the traffic data is generated, and the mirroring data corresponding to the traffic data is forwarded to the mirroring pipeline to make The data analysis server obtains the mirrored data from the mirroring pipeline, and analyzes and processes the mirrored data. In this way, the traffic data is obtained from the service expansion implementation layer of the target data plane agent, avoiding the difficulty of data reorganization and restoration in the process of obtaining the traffic data. Large, business data extraction is difficult, and cannot be analyzed in the case of encrypted communication, thereby reducing the consumption of computing resources in the process of obtaining traffic data.

[0105] The foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than in the embodiments and still achieve desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0106] In the 1990s, the improvement of a technology can be clearly distinguished between hardware improvements (for example, improvements in circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements in method flow). However, with the development of technology, the improvement of many methods and processes of today can be regarded as a direct improvement of the hardware circuit structure. Designers almost always get the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be realized by hardware entity modules. For example, a programmable logic device (Programmable Logic Device, PLD) (for example, a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user's programming of the device. It is programmed by the designer to "integrate" a digital system on a PLD without requiring the chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly realized by using "logic compiler" software, which is similar to the software compiler used in program development and writing, but before compilation The original code must also be written in a specific programming language, this is called Hardware Description Language (Hardware Description Language, HDL), and HDL is not only one, but there are many, such as ABEL (Advanced Boolean Expression Language) , AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (RubyHardware Description Language), etc. The most commonly used at present are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. It should also be clear to those skilled in the art that only a little logic programming of the method flow in the above-mentioned hardware description languages ​​and programming into an integrated circuit can easily obtain the hardware circuit that implements the logic method flow.

[0107] The controller can be implemented in any suitable manner. For example, the controller can take the form of, for example, a microprocessor or a processor and a computer-readable medium storing computer-readable program codes (such as software or firmware) executable by the (micro)processor. , Logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicon Labs C8051F320, the memory controller can also be implemented as a part of the memory control logic. Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code manner, it is entirely possible to program the method steps to make the controller use logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded The same function can be realized in the form of a microcontroller, etc. Therefore, such a controller can be regarded as a hardware component, and the devices included in it for implementing various functions can also be regarded as a structure within the hardware component. Or even, the device for realizing various functions can be regarded as both a software module for realizing the method and a structure within a hardware component.

[0108] The systems, devices, modules, or units illustrated in the above embodiments may be specifically implemented by computer chips or entities, or implemented by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cell phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or Any combination of these devices.

[0109] For the convenience of description, when describing the above device, the functions are divided into various units and described separately. Of course, when implementing one or more embodiments of this specification, the functions of each unit may be implemented in the same one or more software and/or hardware.

[0110] Those skilled in the art should understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of this specification may adopt a computer program implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The form of the product.

[0111] The embodiments of this specification are described with reference to the flowcharts and/or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are generated In the process Figure one Process or multiple processes and/or boxes Figure one A device with functions specified in a block or multiple blocks.

[0112] These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device. The device is implemented in the process Figure one Process or multiple processes and/or boxes Figure one Function specified in a box or multiple boxes.

[0113] These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment. Instructions are provided to implement the process Figure one Process or multiple processes and/or boxes Figure one Steps of functions specified in a box or multiple boxes.

[0114] In a typical configuration, the computing device includes one or more processors (CPU), input/output interfaces, network interfaces, and memory.

[0115] The memory may include non-permanent memory in computer readable media, random access memory (RAM) and/or non-volatile memory, such as read only memory (ROM) or flash memory (flash RAM). Memory is an example of computer readable media.

[0116] Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

[0117] It should also be noted that the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, product or equipment including a series of elements not only includes those elements, but also includes Other elements that are not explicitly listed, or include elements inherent to this process, method, commodity, or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, commodity, or equipment that includes the element.

[0118] Those skilled in the art should understand that the embodiments of this specification can be provided as methods, systems or computer program products. Therefore, one or more embodiments of this specification may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of this specification may adopt a computer program implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The form of the product.

[0119] One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification can also be practiced in a distributed computing environment. In these distributed computing environments, tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0120] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiment.

[0121] The above descriptions are only examples of this specification, and are not intended to limit this specification. For those skilled in the art, this specification can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the scope of the claims of this specification.