Data acquisition method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a data acquisition method and device, computer equipment, a storage medium and a program product, the data acquisition method applied to an agent side comprises the following steps: calling a data client side to acquire required result binary data from a data server side based on data and an instruction obtained by decoding; the data server transparently transmits the result binary data to a third encoder of a preset message body protocol through the data client and then transparently transmits the result binary data to a fourth encoder of a preset channel protocol, and the fourth encoder encodes the received data to obtain response binary data; and finally, a second byte stream corresponding to the response binary data is transmitted back to the application client, so that a result value needing to be acquired is transmitted to the application client. And the data client and the third encoder of the preset message body protocol do not perform encoding and decoding processing on the result binary data, so that the transmission time of the data in the data client and the third encoder deployed in the proxy end is shortened, and the data transmission efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of Internet technologies, and in particular, to a data acquisition method, apparatus, computer device, storage medium, and computer program product. Background Art

[0002] With the development of Internet technologies, the application runtime derived from service mesh in the cloud native field has received increasing attention; in related technologies, popular open-source products for application runtime include Dapr and Layotto, both of which use grpc (a type of RPC, Remote Procedure Call, translated into Chinese as Remote Procedure Call) as the standard communication protocol for sidecar and SDK (Software Development Kit). After accessing the sidecar, the overall link will increase the latency of the grpc protocol interaction; where sidecar is a design pattern that separates application functions from the application itself as a separate process.

[0003] In related technologies, an application runtime product of a certain platform is developed based on the open-source Layotto. After the product is launched, it is found that the performance deteriorates significantly after the service accesses the sidecar. Especially when users use the cache redis (Remote Dictionary Server) function of the runtime, which is a scenario that is more sensitive to performance, the latency brought by the sidecar has a more obvious impact on users. Therefore, there is an urgent need to provide a new data acquisition method to solve the problem of poor performance in the data acquisition process. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a data acquisition method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve data acquisition performance.

[0005] In a first aspect, this application provides a data acquisition method, which is applied to an agent side and includes:

[0006] Receiving a first byte stream transmitted by an application client, and processing the first byte stream into request binary data; the first byte stream is used to represent a request keyword input by a user in the application client and a data acquisition instruction matching the request keyword, which are successively encoded into binary data by a first encoder of a preset message body protocol deployed in the application client and a second encoder of a preset channel protocol.

[0007] Successively call the second decoder of the preset channel protocol and the first decoder of the preset message body protocol deployed on the proxy end to successively decode the request binary data, and obtain the parameters of the request keyword and the data acquisition instruction matching the request keyword;

[0008] Call the data client to encode the parameters of the request keyword and the data acquisition instruction based on the preset message body protocol, and send a relevant data request to the data server based on the encoded parameters of the request keyword and the data acquisition instruction; to instruct the data server to query based on the data request and return the result binary data corresponding to the request keyword to the data client, so that the data client transparently transmits the result binary data to the third encoder of the preset message body protocol deployed on the proxy end; wherein, the result binary data includes the result value corresponding to the request keyword in the data server;

[0009] Call the fourth encoder of the preset channel protocol deployed on the proxy end to encode the response object corresponding to the result binary data into response binary data;

[0010] Return the second byte stream corresponding to the response binary data to the application client to implement the transmission of the result value required by the application client.

[0011] In one embodiment, the proxy end includes a Mosn framework;

[0012] The receiving the first byte stream transmitted by the application client and processing the first byte stream into request binary data includes:

[0013] The Mosn framework receives the first byte stream transmitted by the application client and cuts the first byte stream based on the request header length and the request body length in the message format of the preset channel protocol to obtain request binary data.

[0014] In one embodiment, the returning the second byte stream corresponding to the response binary data to the application client includes:

[0015] Call the Mosn framework to return the second byte stream corresponding to the response binary data to the application client through the UDS link established between the application client and the proxy end.

[0016] In one embodiment, the step of successively invoking the second decoder of the preset channel protocol and the first decoder of the preset message body protocol deployed on the proxy end to successively decode the request binary data to obtain the parameters of the request keyword and the data acquisition instruction matching the request keyword includes:

[0017] Invoke the second decoder of the preset channel protocol deployed on the proxy end to decode the request binary data into a corresponding request object;

[0018] Invoke the first decoder of the preset message body protocol deployed on the proxy end to decode the request object into the parameters of the request keyword and the data acquisition instruction matching the request keyword.

[0019] In one embodiment, before invoking the fourth encoder of the preset channel protocol deployed on the proxy end to encode the response object corresponding to the result binary data into response binary data, it further includes:

[0020] Place the result binary data into the response body in the message format of the preset channel protocol, set the status bit of the response frame header in the message format of the preset channel protocol, and set the RequestId in the request frame header of the request object to the response frame header to construct the response object corresponding to the result binary data.

[0021] In one embodiment, the preset channel protocol includes a data transmission layer, a preset protocol architecture layer, and a data layer; wherein, the message format of the preset channel protocol includes a fixed-length frame header, a variable-length request header, and a variable-length request body.

[0022] In a second aspect, the present application further provides a data acquisition method applied to an application client, including:

[0023] Receive a request keyword input by a user, and invoke the keyword method to correspond to a data acquisition instruction matching the request keyword on the data server; wherein, the data server communicates with the application client based on a proxy end;

[0024] Successively invoke the first encoder of the preset message body protocol and the second encoder of the preset channel protocol deployed on the application client to successively encode the request keyword and the data acquisition instruction to obtain request binary data;

[0025] Transmit the first byte stream corresponding to the request binary data to the proxy end and wait to receive the data returned by the proxy end;

[0026] Receive the second byte stream returned by the proxy end, and process the second byte stream into response binary data; wherein, the second byte stream is used to represent the result value corresponding to the request keyword extracted from the data server end, and is encoded into binary data after passing through the third encoder of the preset message body protocol deployed on the proxy end and then passing through the fourth encoder of the preset channel protocol;

[0027] Successively call the fourth decoder of the preset channel protocol and the third decoder of the preset message body protocol deployed on the application client to decode the response binary data into result binary data, so as to enable the application client to obtain the result value associated with the request keyword in the data server end; wherein, the result binary data includes the result value corresponding to the request keyword in the data server end.

[0028] In one embodiment, the successively calling the first encoder of the preset message body protocol and the second encoder of the preset channel protocol deployed on the application client to successively encode the request keyword and the data acquisition instruction to obtain request binary data includes:

[0029] Call the first encoder of the preset message body protocol deployed on the application client, and encode to obtain the binary content corresponding to the data acquisition instruction according to the request keyword in combination with the preset message body protocol;

[0030] Generate a RequestId for multiplexing, set it into the request frame header of the message format of the preset channel protocol, put the requirement field of the proxy end into the request header of the message format, and put the binary content into the request body of the message format, so as to construct a request object associated with the binary content by the preset channel protocol;

[0031] Call the second encoder of the preset channel protocol deployed on the application client to encode the request object into the request binary data.

[0032] In one embodiment, the application client includes a Netty framework;

[0033] Before transmitting the first byte stream corresponding to the request binary data to the proxy end, it further includes:

[0034] Call the send method of the Netty framework to asynchronously send the request binary data, and save the future object returned by the send method into a multiplexed data storage set; wherein, the content associated with the request keyword in the data storage set is the RequestId, and the RequestId is used to associate the future object.

[0035] In one embodiment, transmitting the first byte stream corresponding to the request binary data to the proxy end includes:

[0036] Invoking the Netty framework to transmit the first byte stream corresponding to the request binary data to the proxy end asynchronously through a preset thread via the UDS link established between the application client and the proxy end; wherein, the preset thread is different from the thread where the future object is located.

[0037] In one embodiment, receiving the second byte stream returned by the proxy end and processing the second byte stream into response binary data includes:

[0038] The Netty framework receives the second byte stream returned by the proxy end, and performs packet cutting on the second byte stream based on the request header length and the request body length in the message format of the preset channel protocol to obtain response binary data.

[0039] In one embodiment, sequentially invoking the fourth decoder of the preset channel protocol and the third decoder of the preset message body protocol deployed on the application client to decode the response binary data into result binary data includes:

[0040] Invoking the fourth decoder of the preset channel protocol deployed on the application client to decode the response binary data into a response object;

[0041] Based on the decoding of the response binary data by the fourth decoder, extracting the RequestId, obtaining the corresponding future object based on the RequestId, and releasing the block of the thread where the future object is located, and returning the response object corresponding to the sending method;

[0042] Invoking the third decoder of the preset message body protocol deployed on the application client to decode the response object into result binary data.

[0043] In a third aspect, the present application further provides a data acquisition device, which is applied to the proxy end and includes:

[0044] A data acquisition module, configured to receive the first byte stream transmitted by the application client and process the first byte stream into request binary data; the first byte stream is used to represent the request keyword input by the user on the application client and the data acquisition instruction matching the request keyword, and is sequentially encoded into binary data by the first encoder of the preset message body protocol and the fourth encoder of the preset channel protocol deployed on the application client;

[0045] A data processing module, configured to sequentially call a fourth decoder of the preset channel protocol and a first decoder of the preset message body protocol deployed on the proxy end to sequentially decode the request binary data, so as to obtain parameters of the request keyword and the data acquisition instruction matching the request keyword;

[0046] The data processing module is further configured to call a data client to encode the parameters of the request keyword and the data acquisition instruction based on the preset message body protocol, and send a relevant data request to a data server based on the encoded parameters of the request keyword and the data acquisition instruction; to instruct the data server to query based on the data request and return result binary data corresponding to the request keyword to the data client, so that the data client transparently transmits the result binary data to a third encoder of the preset message body protocol deployed on the proxy end; wherein, the result binary data includes a result value corresponding to the request keyword in the data server;

[0047] The data processing module is further configured to call a fourth encoder of the preset channel protocol deployed on the proxy end to encode a response object corresponding to the result binary data into response binary data;

[0048] A data interaction module, configured to return a second byte stream corresponding to the response binary data to the application client, so as to implement transmitting the result value required by the application client.

[0049] Fourthly, the present application further provides a data acquisition device, which is applied to an application client and includes:

[0050] A data acquisition module, configured to receive a request keyword input by a user and call a keyword method to correspond to a data acquisition instruction matching the request keyword in a data server; wherein, the data server communicates with the application client based on a proxy end;

[0051] A data processing module, configured to sequentially call a first encoder of the preset message body protocol and a second encoder of the preset channel protocol deployed on the application client to sequentially encode the request keyword and the data acquisition instruction, so as to obtain request binary data;

[0052] A data interaction module, configured to transmit a first byte stream corresponding to the request binary data to the proxy end and wait to receive the data returned by the proxy end;

[0053] The data interaction module is further configured to receive a second byte stream transmitted back by the proxy end and process the second byte stream into response binary data; wherein, the second byte stream is used to represent the result value corresponding to the request keyword extracted from the data server, and is encoded into binary data after passing through the third encoder of the preset message body protocol deployed on the proxy end and then passing through the fourth encoder of the preset channel protocol;

[0054] The data processing module is further configured to sequentially call the fourth decoder of the preset channel protocol and the third decoder of the preset message body protocol deployed on the application client to decode the response binary data into result binary data, so as to enable the application client to obtain the result value associated with the request keyword in the data server; wherein, the result binary data includes the result value corresponding to the request keyword in the data server.

[0055] In a fifth aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, it implements the data acquisition method according to any one of the first aspect or the second aspect.

[0056] In a sixth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the data acquisition method according to any one of the first aspect or the second aspect.

[0057] In a seventh aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the data acquisition method according to any one of the first aspect or the second aspect.

[0058] The above data acquisition method, device, computer device, storage medium, and computer program product receive a first byte stream transmitted by an application client at an agent end, process it into request binary data, and then sequentially call a second decoder of a preset channel protocol and a first decoder of a preset message body protocol deployed at the agent end to sequentially decode the request binary data. Then, a data client is called to obtain the required result binary data from a data server based on the decoded data and instructions. Subsequently, the data server directly passes the result binary data to a third encoder of the preset message body protocol deployed at the agent end, and the third encoder further passes it to a fourth encoder of the preset channel protocol deployed at the agent end. The fourth encoder encodes the received data to obtain response binary data, and finally, the second byte stream corresponding to the response binary data is sent back to the application client to transmit the result value required by the application client. By setting that neither the data client deployed at the agent end nor the third encoder of the preset message body protocol encodes and / or decodes the result binary data, but directly passes it to the fourth encoder of the preset channel protocol deployed at the agent end, the time for data transmission in the data client and the third encoder deployed at the agent end is reduced, which is beneficial to improving the efficiency of data transmission, and further improving the efficiency of the application client obtaining data from the data server through the agent end. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0060] Figure 1 It is a schematic diagram of the overall architecture of an application runtime product in related technologies;

[0061] Figure 2 It is a schematic flowchart of a data acquisition method applied to an agent end in an embodiment provided by the present application;

[0062] Figure 3 It is a schematic diagram of the message format of a preset channel protocol provided by an embodiment of the present application;

[0063] Figure 4 It is a schematic flowchart of a data acquisition method applied to an application client in an embodiment provided by the present application;

[0064] Figure 5 It is a schematic flowchart of a data acquisition method provided by an embodiment of the present application;

[0065] Figure 6 A structural block diagram of a data acquisition device in an embodiment provided by this application;

[0066] Figure 7 Another structural block diagram of a data acquisition device in an embodiment provided by this application;

[0067] Figure 8 The internal structure diagram of a computer device in an embodiment provided by this application. Detailed implementation manners

[0068] In order to make the objectives, technical solutions and advantages of this application clearer, the following further elaborates on this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0069] As described in the background art, in the related art, the application operation products of a certain platform are developed based on the open-source Layotto. After the products are launched, it is found that the performance loss is relatively large after the business access sidecar is connected. Especially when users use the cache redis (Remote Dictionary Server) function during runtime, which is a performance-sensitive scenario, the latency brought by the sidecar has a more obvious impact on users. After research, it is found that the performance loss is mainly concentrated in the grpc communication and encoding / decoding between the sidecar and the SDK. To solve this performance problem, this application proposes to design a set of high-performance private protocols more suitable for application runtime, and replace the traditional standard grpc protocol with the newly designed high-performance private protocol.

[0070] Figure 1 For a schematic diagram of the overall architecture of an application operation product in the related art, please refer to Figure 1 , this architecture is mainly divided into four modules: console service, access machine, sidecar, and standard SDK.

[0071] Users set runtime-related configurations on the interface of the platform where the console service is located, and then the configurations are written into redis through the console service. Then, the access machine periodically reads the configurations from redis and distributes them to the sidecar (i.e., Layotto) through the xDS protocol (an information transmission protocol). Layotto, as the base of the runtime, has the capabilities of multiple middleware, such as push, cache, message queue, etc.

[0072] In order to call these capabilities of sidecar, users need to access a standard SDK provided by the platform where the console service is located. This SDK integrates the standard API (Application Programming Interface) of the above middleware. Generally, it does not need to be upgraded because it is just a simple encapsulation of the API. The main logic of the middleware client has been sunk to sidecar, and subsequent upgrades are completed by sidecar without perception. Next, when the user processes the request, the standard SDK is used to call the interface provided by sidecar through the grpc protocol, and then sidecar calls the corresponding middleware server, and finally returns the grpc response.

[0073] according to Figure 1 It can be seen from the request link that compared with the traditional SDK access, the runtime sidecar access method has an extra hop of grpc communication time. In order to observe the performance loss brought by sidecar to users, the inventor of the present application performed a performance stress test on the GET command (a data acquisition command) of the redis scenario that users are most concerned about, and found that accessing the sidecar will increase the latency by 0.24ms, but the native SDK's GET command takes only 0.52ms at a time, so the sidecar will bring a performance loss of 0.28ms (more than 50%), which may not be acceptable to users who are sensitive to performance.

[0074] In order to analyze the performance bottleneck of sidecar, the inventor of the present application used the performance analysis tool pprof (performance debugging tool) provided by Go language (an open source programming language) to sample and analyze the CPU (Central Processing Unit) of sidecar during stress testing, and found that the overall CPU consumption is mainly concentrated in three areas: the system call overhead caused by network packet sending and receiving, the overhead of grpc protocol processing, and the overhead of protobuf (Protocol Buffer, a mixed language data standard) encoding and decoding.

[0075] Figure 2 A flow chart of a data acquisition method applied to an agent in an embodiment of the present application is provided, which aims at the above-mentioned problems existing in the related technology, such as Figure 2As shown in the figure, the present application provides a data acquisition method. In this embodiment, the method is exemplified by being applied to a terminal. The terminal may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and Internet of Things devices. The Internet of Things devices may be, for example, smart TVs and in-vehicle intelligent devices. It can be understood that this method can also be applied to a server. The server may be an independent server or a server cluster composed of multiple servers, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server.

[0076] In this embodiment, the terminal to which the data acquisition method is applied may specifically include an agent, and the relevant data acquisition method may include the following steps 101 to 105, where:

[0077] Step 101: Receive the first byte stream transmitted by the application client and process the first byte stream into request binary data. The first byte stream is used to represent the request keyword input by the user in the application client and the data acquisition instruction matching the request keyword, which are successively encoded into binary data by the first encoder of the preset message body protocol deployed in the application client and the second encoder of the preset channel protocol.

[0078] Specifically, the architecture provided by the present application for data communication includes an application client, an agent that has established a communication relationship with the application client, and a data server that has further established a communication relationship with the agent. Based on this, step 101 in the data acquisition method applied to the agent is executed. The agent receives the first byte stream transmitted by the application client and processes the first byte stream into binary data that can be recognized by the subsequent decoder. The processed binary data is specifically request binary data. The "request" in the "request binary data" here is only a naming limitation for the binary data, used to distinguish it from the binary data obtained after being processed through other steps in the present application.

[0079] It should be added that the user inputs a request keyword in the application client, and the request keyword is matched with a corresponding data acquisition instruction. The request keyword and the corresponding data acquisition instruction are successively encoded into binary data by the first encoder of the preset message body protocol deployed in the application client and the second encoder of the preset channel protocol. The first byte stream is used to represent the binary data, that is, the first byte stream is a form of the binary data and includes the content corresponding to the binary data.

[0080] It should be added that the application client in the present application may specifically be a client SDK, and the agent may specifically be Sidecar or Layotto.

[0081] Step 102: Call the second decoder of the preset channel protocol and the first decoder of the preset message body protocol deployed on the proxy end in sequence to decode the request binary data successively, and obtain the parameters of the request keyword and the data acquisition instruction matching the request keyword.

[0082] Specifically, after the proxy end receives the first byte stream transmitted by the application client and processes it into request binary data, step 102 can be further executed. First, call the second decoder of the preset channel protocol deployed on the proxy end, and perform the first decoding on the request binary data through the second decoder. Then, call the first decoder of the preset message body protocol deployed on the proxy end, and perform the second decoding on the request binary data that has been decoded for the first time through the first decoder. That is, after performing two successive decodings on the request binary data through the second decoder and the first decoder, the parameters of the request keyword and the data acquisition instruction matching the request keyword can be obtained.

[0083] It should be noted that since the user enters the request keyword and the data acquisition instruction matching the request keyword in the application client, and they are transmitted to the proxy end after two encodings in the application client, the proxy end needs to perform two decodings after receiving the first byte stream to obtain the parameters of the request keyword entered by the user in the application client and the data acquisition instruction matching the request keyword.

[0084] Step 103: Call the data client to encode the parameters of the request keyword and the data acquisition instruction based on the preset message body protocol, and send a relevant data request to the data server based on the encoded parameters of the request keyword and the data acquisition instruction; to instruct the data server to query based on the data request and return the result binary data corresponding to the request keyword to the data client, and then enable the data client to transparently transmit the result binary data to the third encoder of the preset message body protocol deployed on the proxy end; where the result binary data includes the result value corresponding to the request keyword in the data server.

[0085] Specifically, after decoding the parameters of the requested keyword and the data acquisition instruction matching the requested keyword through the second decoder and the first decoder deployed on the proxy side, step 103 can be further executed. Call the data client deployed on the proxy side to encode the parameters of the requested keyword and the data acquisition instruction based on the preset message body protocol to obtain data that can be recognized by the data server. Then, send a relevant data request to the data server based on the encoded parameters of the requested keyword and the data acquisition instruction, instructing the data server to query the data requested by the client and return the result binary data corresponding to the requested keyword to the data client. Among them, the result binary data includes the result value corresponding to the requested keyword in the data server. In step 103 provided by this application, after the data client receives the result binary data corresponding to the requested keyword returned by the data server, it does not parse the binary data, but calls the data client to directly pass through the result binary data to the third encoder of the preset message body protocol deployed on the proxy side, and further process the result binary data through the third encoder.

[0086] Since in the data acquisition method provided by this application, the data client does not parse the result binary data returned by the data server, one data parsing process in the data acquisition process is reduced. Specifically, it avoids the consumption of decoding the data once, which is beneficial to improving the transmission efficiency of data in the proxy side.

[0087] Step 104, call the fourth encoder of the preset channel protocol deployed on the proxy side to encode the response object corresponding to the result binary data into response binary data.

[0088] Specifically, after the third encoder deployed on the proxy side receives the result binary data, step 104 can be further executed. Call the fourth encoder of the preset channel protocol deployed on the proxy side to further encode the response object received from the third encoder side. Here, the response object is the data obtained after the result binary data undergoes certain data processing steps based on the preset channel protocol. Specifically, the third encoder encodes the response object into response binary data, that is, encodes the data it receives into a data format that can be further transmitted to the application client through the third encoder to ensure the smooth transmission of data from the proxy side to the application client.

[0089] It should be added that the "response" in the "response binary data" is only a naming limitation for the binary data, used to distinguish it from the "request binary data" that has appeared in this application. Obviously, the request binary data and the response binary data are obtained through different data processing steps.

[0090] Step 105: Send back the second byte stream corresponding to the response binary data to the application client to transmit the result value that the application client needs to obtain.

[0091] Specifically, after obtaining the response binary data in step 104, step 105 can be further executed to process the response binary data into the corresponding second byte stream, and then transmit the second byte stream to the application client, so as to transmit the result value that the application client needs to obtain in the data server for communication based on the proxy end.

[0092] Among them, the second byte stream is used to represent the response binary data, that is, the second byte stream is a form of the response binary data and includes the content corresponding to the response binary data.

[0093] It should be added that the application client and the proxy end provided in this application can be optionally set on the same machine; however, this is only an optional implementation manner provided in this application, and this application does not make specific limitations on this.

[0094] It should be added that in the data acquisition method provided in this application, the preset channel protocol used is a private protocol provided in this application, and the preset message body protocol used can optionally be the Resp protocol. Among them, the Resp protocol is a protocol used when the Redis client communicates with the Redis server, and its full name is REDis SerializationProtocol, that is, the redis serialization protocol.

[0095] It should be noted first that the grpc protocol adopted in the related technology is an open-source RPC protocol. It is carried on top of HTTP2 and generally uses protobuf as the encoding protocol for the service content. Its overall architecture is divided into the TCP (Transmission Control Protocol) transport layer, the TLS (Transport Layer Security) encryption layer, the HTTP2 (HyperText Transfer Protocol - 2) application layer from bottom to top, then the framework layer of the grpc protocol, and finally the data layer of the service content encoded by protobuf. From the layering of grpc, it can be seen that compared with the general seven-layer protocol, the grpc protocol will inevitably increase the encoding and decoding time of the HTTP2 protocol, so it has a certain impact on the overall performance.

[0096] In view of the above performance issues of the gRPC protocol, this application provides a private protocol, namely a preset channel protocol, which has an optimized architecture compared to the gRPC protocol. This application does not specifically limit the optimized architecture of the preset channel protocol compared to the gRPC protocol, as long as the optimized preset channel protocol can cooperate to implement the data acquisition method of steps 101 - 105 provided by this application.

[0097] Obviously, in view of the problem that the overall CPU consumption in the related technology mainly concentrates in three places, in the data acquisition method provided by this application, a private protocol (preset channel protocol) is used to replace the gRPC protocol, that is, the related channel protocol is optimized; and the resp protocol is used to replace the protobuf protocol (in the redis scenario), that is, the message body protocol is optimized; in addition, the client side in the redis scenario is also optimized. Specifically, the data client deployed on the proxy side does not parse the query data returned by the data server it receives, but directly passes it through to the third encoder of the preset message body protocol.

[0098] In the above data acquisition method, in this application, after receiving the first byte stream transmitted by the application client at the proxy side and processing it into request binary data, the second decoder of the preset channel protocol and the first decoder of the preset message body protocol deployed on the proxy side are sequentially called to decode the request binary data. Then, the data client is called to obtain the required result binary data from the data server based on the decoded data and instructions; then the data server directly passes through the result binary data to the third encoder of the preset message body protocol deployed on the proxy side, and the third encoder then passes it through to the fourth encoder of the preset channel protocol deployed on the proxy side. The received data is encoded by the fourth encoder to obtain response binary data, and finally the second byte stream corresponding to the response binary data is sent back to the application client to transmit the result value it needs to obtain to the application client. That is, by setting that neither the data client deployed on the proxy side nor the third encoder of the preset message body protocol encodes and / or decodes the result binary data, but directly passes it through to the fourth encoder of the preset channel protocol deployed on the proxy side, the time for data transmission in the data client and the third encoder deployed on the proxy side is reduced, which is beneficial to improving the efficiency of data transmission, and further improving the efficiency of the application client obtaining data from the data server through the proxy side.

[0099] Please continue to refer to Figure 2 , in an exemplary embodiment, the proxy side includes the Mosn framework; for step 101 performed above, receiving the first byte stream transmitted by the application client and processing the first byte stream into request binary data, the specific execution can be:

[0100] The Mosn framework receives the first byte stream transmitted by the application client, and performs packet splitting on the first byte stream based on the request header length and the request body length in the message format of the preset channel protocol to obtain the request binary data.

[0101] Specifically, an alternative implementation provided in this application is that the Mosn framework is deployed in the proxy end. The proxy end can receive the first byte stream transmitted by the application client through the Mosn framework, and then perform packet splitting on the first byte stream based on the request header length and the request body length in the message format of the preset channel protocol to obtain the request binary data corresponding to the first byte stream; that is, the first byte stream is processed into binary data that can be recognized by the subsequent decoder based on the message format of the preset channel protocol, thereby ensuring the transmission effect of the data in the proxy end. Among them, MOSN is a full-stack network proxy.

[0102] Please refer to Figure 2 , in an exemplary embodiment, step 105 performed above, returning the second byte stream corresponding to the response binary data to the application client, can be specifically implemented as:

[0103] Call the Mosn framework to return the second byte stream corresponding to the response binary data to the application client through the UDS link established between the application client and the proxy end.

[0104] Specifically, since the application client and the proxy end provided in this application can be optionally set on the same machine, an alternative implementation provided in this application is that the communication connection between the application client and the proxy end is specifically implemented through a UDS (unix domain socket) link; then, the proxy end can communicate with the application client through the UDS link.

[0105] Based on the Mosn framework deployed in the proxy end, the content executed in step 105 can specifically be that the proxy end calls the Mons framework to transmit the second byte stream corresponding to the response binary data to the application client based on the UDS link.

[0106] Among them, UDS is a very high-performance IPC communication method. It does not need to go through the network protocol stack, does not need to pack and unpack, calculate checksums, maintain sequence numbers and responses, etc. It just copies the data packet from one process to another in the kernel state, so the efficiency is extremely high. Among them, IPC (inter-process communication) is inter-process communication.

[0107] Please refer to Figure 2, in an exemplary embodiment, for the above-mentioned step 102 that is executed, the second decoder of the preset channel protocol deployed on the proxy side and the first decoder of the preset message body protocol are sequentially called to sequentially decode the request binary data, and the parameters of the request keyword and the data acquisition instruction matching the request keyword are obtained. Specifically, it can be executed as steps 121 and 122, where:

[0108] Step 121: Call the second decoder of the preset channel protocol deployed on the proxy side to decode the request binary data into a corresponding request object;

[0109] Step 122: Call the first decoder of the preset message body protocol deployed on the proxy side to decode the request object into the parameters of the request keyword and the data acquisition instruction matching the request keyword.

[0110] Specifically, for step 121, first call the second decoder of the preset channel protocol deployed on the proxy side to decode the request binary data obtained by executing step 101 to obtain a request object corresponding to the request binary data; then execute step 122, call the first decoder of the preset message body protocol deployed on the proxy side to decode the request object obtained by executing step 121 to obtain the parameters of the request keyword corresponding to the request object and the data acquisition instruction matching the request keyword.

[0111] Since the data client cannot recognize binary data, therefore, the parameters of the request keyword corresponding to the request binary data and the data acquisition instruction matching the request keyword are decoded through the second decoder and the first decoder, so as to facilitate the data client to recognize and process the data, and to achieve the smooth transmission of data in the proxy side.

[0112] Please refer to Figure 2 , in an exemplary embodiment, for the above-mentioned step 104 that is executed, before calling the fourth encoder of the preset channel protocol deployed on the proxy side to encode the response object corresponding to the result binary data into response binary data, step 106 is further included; execute step 106, put the result binary data into the response body in the message format of the preset channel protocol, set the status bit of the response frame header in the message format of the preset channel protocol, and set the RequestId in the request frame header of the request object to the response frame header to construct a response object corresponding to the result binary data.

[0113] Specifically, after step 103 is executed and the data client transparently transmits the result binary data to the third encoder of the preset message body protocol deployed on the proxy side, step 106 can be further executed to construct a response object of the preset channel protocol. The specific execution content is to place the result binary data into the response body in the message format of the preset channel protocol, set the status bit of the response frame header in the message format of the preset channel protocol, and at the same time set the RequestId in the request frame header of the request object to the response frame header to facilitate multiplexing by the client. Through these steps, the construction of the response object corresponding to the result binary data is achieved.

[0114] Please refer to Figure 2 , in an exemplary embodiment, the preset channel protocol includes a data transmission layer, a preset protocol architecture layer, and a data layer; wherein, the message format of the preset channel protocol includes a fixed-length frame header, a variable-length request header, and a variable-length request body.

[0115] Specifically, an alternative architecture mode provided by this application for the preset channel protocol is that the preset channel protocol is a three-layer protocol, including a data transmission layer, a preset protocol architecture layer, and a data layer from bottom to top; that is, compared with the grpc protocol in the related art, the preset channel protocol provided by this application does not require a TLS encryption layer. At the same time, in order to reduce the parsing consumption of the seven-layer protocol, HTTP2 is no longer used, but a private protocol (preset channel protocol) is directly developed on top of TCP (data transmission layer). This application provides an alternative named runtime-protocol for the preset channel protocol, which means a protocol dedicated to the application runtime; then on top of the private protocol is the business data it carries, and the business data can use any encoding and decoding protocol. For the scenario of running redis at runtime, the resp protocol can be optionally used, and for other scenarios, the protobuf protocol can be optionally used.

[0116] Another alternative implementation provided by this application is to set the message format of the preset channel protocol to include a fixed-length frame header, a variable-length request header, and a variable-length request body.

[0117] Figure 3 For a schematic diagram of the message format of the preset channel protocol provided by the embodiment of this application, please refer to Figure 2 and Figure 3 , specifically, the message format of the preset channel protocol is divided into three parts: a fixed-length frame header (18 bytes), a variable-length header, and a variable-length body. The header part stores the headers required during application runtime, and the body part is used to encapsulate the binary content of other protocols.

[0118] First, the fixed-length header from front to back includes the following content: magic identifier: protocol magic number identifier, 1 byte, corresponding to the ASCII character ^. flag field: 1 byte, which needs to be parsed according to the 8 bits in it. The first bit of the high order represents whether it is a request or a response, 0 for request, 1 for response; the second bit represents the content protocol type, 0 represents the RESP protocol. The third bit represents the message type, 1 represents the heartbeat message. status field: 2 bytes, representing the response status code. requestId: 8 bytes, request ID, used for multiplexing. header length: 2 bytes, representing the length of the variable-length header, used for packet splitting. body length: 4 bytes, representing the length of the variable-length body, used for packet splitting.

[0119] Following the fixed-length header is the variable-length header. Each header format is as follows: a fixed 1 byte represents the length of the following string, and the length cannot exceed 256 bytes. The string format is {key:value}, and the key is specified not to contain a colon.

[0120] The commonly used headers in gRPC are shown in Table 1 below. The commonly used headers in the private protocol are shown in Table 2 below. Compared with the gRPC protocol, the headers used in the private protocol are more concise, lacking many fixed HTTP headers, such as scheme, method, content-Type, etc. Therefore, the overall message header of the private protocol is more delicate and occupies less network bandwidth. Among them, the commonly used headers in gRPC are generally more than 100 bytes, and the commonly used headers in the private protocol plus the fixed-length header generally do not exceed 50 bytes.

[0121] Table 1

[0122] header Name Meaning :scheme HTTP mode, such as http / https :method Request method, such as GET / POST :authority Request host header content-type Content type, such as application / grpc user-agent Client identifier content-coding Compression algorithm …… ……

[0123] Table 2

[0124] header Name Meaning RN Resource name, ResourceName CI Client ID, ClientId TI Link ID, TraceId

[0125] After the variable-length header is the variable-length body. For the Redis scenario during the runtime of applications with high performance requirements, this application selects Redis's private protocol, the RESP protocol, to replace the standard Protobuf protocol. The RESP protocol features good parsing performance and is intuitive and easy to read. In contrast, the Protobuf protocol has worse parsing performance than the RESP protocol, and the data after Protobuf encoding is pure binary and not highly readable. This application implements the codec for the RESP protocol by referring to the high-performance Redis client Lettuce in Java. The format of the RESP protocol (version 2.0) is as follows. The transmitted data is divided into five types, and each line ends with a carriage return and line feed character (\r\n).

[0126] Among them, the specific details of the five types of transmitted data are as follows. A simple string (single-line string, Simple Strings) starts with the '+' symbol. A bulk string (multi-line string, Bulk Strings) starts with the '$' symbol, followed by the length of the string. An integer (Integers) starts with the ':' symbol, followed by the string form of the integer. An error (Errors) starts with the '-' symbol. An array (Arrays) starts with the '*' symbol, followed by the length of the array.

[0127] Thanks to the simple nature of the RESP protocol, the corresponding encoding and decoding logic is relatively easy to implement and also has good performance. This application conducts a performance benchmark (Benchmark) in the Go language for the scenario of request decoding. The request of a Redis GET command is decoded using the RESP protocol and the Protobuf protocol respectively. By changing the size of the key of the GET command, the performance differences under different key sizes are observed. As shown in Table 3 below, it shows the comparison of the decoding time per word for the Redis GET command request under different key sizes, where the unit is nanoseconds (ns). Obviously, in the small packet scenario (below 1k), it can be seen that the performance of the RESP protocol is more than 30% higher than that of the Protobuf protocol.

[0128] Table 3

[0129]

[0130] Furthermore, the present application also has optimizations at the transport layer. Since the client and sidecar (server) during application runtime are on the same machine, the Unix domain socket (UDS for short) can be used to replace the TCP socket. UDS is a high-performance IPC communication method that does not need to go through the network protocol stack, does not require packet packing and unpacking, calculating checksums, maintaining sequence numbers and acknowledgments, etc. It only copies data packets from one process to another in the kernel state, so the efficiency is extremely high.

[0131] Based on the same inventive concept, an embodiment of the present application also provides a data acquisition method. In this embodiment, it is exemplified that this method is also applied to a terminal. The terminal to which this data acquisition method is applied may specifically include an application client. Figure 4 FIG. 5 is a schematic flowchart of a data acquisition method applied to an application client in the embodiment provided by the present application. The related data acquisition method may include the following steps 201 to step 205, where:

[0132] Step 201, receive a request keyword input by a user, and call a keyword method to correspond to a data acquisition instruction matching the request keyword on the data server; wherein, the data server communicates with the application client based on the proxy.

[0133] Specifically, the architecture provided by the present application for data communication includes an application client, a proxy that has established a communication relationship with the application client, and a data server that has further established a communication relationship with the proxy. Based on this, step 201 in the data acquisition method applied to the application client is executed. The application client receives the request keyword input by the user, and the application client can call the keyword method to correspond to the data acquisition instruction matching the request keyword on the data server according to the request keyword input by the user. That is, by executing step 201, the request keyword input by the user and the data acquisition instruction matching the request keyword can be obtained to know what content the user needs to extract from the data server.

[0134] Step 202, sequentially call the first encoder of the preset message body protocol and the second encoder of the preset channel protocol deployed on the application client to sequentially encode the request keyword and the data acquisition instruction to obtain request binary data.

[0135] Specifically, after obtaining the request keyword input by the user and the data acquisition instruction matching the request keyword in step 201, step 202 can be further executed. First, the first encoder of the preset message body protocol deployed on the application client is called to perform the first encoding on the request keyword and the data acquisition instruction. Then, the second encoder of the preset channel protocol deployed on the application client is called to perform the second encoding on the request keyword and the data acquisition instruction that have been first-encoded, so as to obtain the request binary data carrying the request keyword and the data acquisition instruction. Through this step, the request keyword and the data acquisition instruction are encoded into data that can be transmitted between the application client and the proxy end through the communication protocol.

[0136] Step 203: Transmit the first byte stream corresponding to the request binary data to the proxy end and wait to receive the data returned by the proxy end.

[0137] Specifically, in step 203, the request binary data is processed into the corresponding first byte stream to realize the transmission of binary data between the application client and the proxy end. The first byte stream can support more data transmission formats, which is conducive to enhancing the diversification of data transmission between the application client and the proxy end. Then, the application client waits to receive the relevant data queried by the proxy end from the data server and returned.

[0138] Step 204: Receive the second byte stream returned by the proxy end and process the second byte stream into response binary data. The second byte stream is used to represent the result value corresponding to the request keyword extracted from the data server, which is encoded into binary data after passing through the third encoder of the preset message body protocol deployed on the proxy end and then through the fourth encoder of the preset channel protocol.

[0139] Specifically, after the proxy end extracts the result value corresponding to the request keyword from the data server with which it communicates, the result value will pass through the third encoder of the preset message body protocol deployed on the proxy end and then through the fourth encoder of the preset channel protocol in sequence. The fourth encoder will encode the data carried by the result value into (response) binary data, and the response binary data will be processed into (the second) byte stream and then returned to the application client. Therefore, the content executed in step 204 is to receive the second byte stream returned by the proxy end, and then the second byte stream can be processed into response binary data so that the subsequent decoder can recognize it.

[0140] Step 205: Call the fourth decoder of the preset channel protocol and the third decoder of the preset message body protocol deployed on the application client in sequence to decode the response binary data into result binary data, so as to realize the application client's acquisition of the result value associated with the request keyword in the data server. The result binary data includes the result value corresponding to the request keyword in the data server.

[0141] Specifically, when performing step 204 to obtain the response binary data associated with the result value for execution, step 205 can be executed. First, call the fourth decoder of the preset channel protocol deployed on the application client to perform the first decoding on the response binary data, and then call the third decoder of the preset message body protocol to perform the second decoding on the response binary data that has been first decoded, to obtain the decoded result binary data. The result binary data includes the result value corresponding to the request keyword in the data server, so as to realize the application client's acquisition of the result value associated with the request keyword in the data server. Then, the result value can be provided to the user, so as to realize that after the user inputs the request keyword in the application client, the application client extracts the result value associated with the request keyword from the data server through the proxy.

[0142] It should be added that in the proxy, the data client performs one encoding on the data output by the first decoder of the preset message body protocol it receives. The data client does not perform encoding and decoding processing on the data transmitted from the data server, and the third encoder of the preset message body protocol also does not perform encoding and decoding processing on the received data. However, the fourth encoder of the preset channel protocol performs one encoding process on the received data; that is, the data is encoded twice during the process of being transmitted from the first decoder of the preset message body protocol to the fourth encoder of the preset channel protocol. Therefore, after the data that has been encoded twice is transmitted back to the application client, it needs to be first decoded by the fourth decoder of the preset channel protocol and then decoded a second time by the third decoder of the preset message body protocol before the result value that can be output to the user can be obtained.

[0143] Please continue to refer to Figure 4 , in an exemplary embodiment, for the above-mentioned step 202, the first encoder of the preset message body protocol and the second encoder of the preset channel protocol deployed on the application client are called in sequence to encode the request keyword and the data acquisition instruction in sequence, to obtain the request binary data. Specifically, it can be executed as steps 221 - 223, where:

[0144] Step 221, call the first encoder of the preset message body protocol deployed on the application client, and according to the request keyword, in combination with the preset message body protocol, encode to obtain the binary content corresponding to the data acquisition instruction;

[0145] Step 222, generate a RequestId for multiplexing, set it into the request frame header in the message format of the preset channel protocol, put the required fields of the proxy into the request header in the message format, and put the binary content into the request body in the message format, to construct a request object associated with the binary content in the preset channel protocol;

[0146] Step 221: Invoke the second encoder of the preset channel protocol deployed on the application client to encode the request object into request binary data.

[0147] Specifically, after performing step 201 to obtain the request keyword input by the user and the data acquisition instruction matching the request keyword, step 221 can be executed first. Invoke the first encoder of the preset message body protocol deployed on the application client, and according to the request keyword, in combination with the preset message body protocol, encode to obtain the binary content corresponding to the data acquisition instruction. Then perform step 222 to construct a request object of the private protocol. The construction process includes generating a RequestId for multiplexing, setting it into the request frame header of the message format of the preset channel protocol, placing the necessary fields required by the proxy side into the request header of the message format, and at the same time placing the binary content into the request body of the message format to construct a request object of the preset channel protocol associated with the binary content. Finally, perform step 221 to invoke the second encoder of the preset channel protocol deployed on the application client to encode the request object into request binary data, so as to encode the request keyword and the data acquisition instruction into data that can be transmitted between the application client and the proxy side through the communication protocol.

[0148] Please continue to refer to Figure 4 , in an exemplary embodiment, the application client includes a Netty framework; before performing step 203 above to transmit the first byte stream corresponding to the request binary data to the proxy side, it further includes step 206. Perform step 206 to asynchronously send the request binary data by invoking the send method of the Netty framework, and save the future object returned by the send method to the multiplexed data storage set; where the content associated with the request keyword in the data storage set is the RequestId, and the RequestId is used to associate the future object.

[0149] Specifically, an optional setting method provided by this application is that there is a Netty framework deployed. After obtaining the request binary data after two encodings of the request keyword and the data acquisition instruction in step 202, step 206 can be executed first. Invoke the send method of the Netty framework to asynchronously send the request binary data. This method will return a Java Future object (future object), and save the future object returned by the send method to the dedicated multiplexed data storage set. The corresponding key is the requestId generated in step 222. Subsequently, the Future object can be found through the requestId, and then the Future can be notified to complete the entire asynchronous operation.

[0150] Among them, Netty is an asynchronous event-driven network application framework used for quickly developing maintainable high-performance protocol servers and clients.

[0151] Please continue to refer to Figure 4 In an exemplary embodiment, for step 203 executed above, which is to transmit the first byte stream corresponding to the request binary data to the proxy end, it can be alternatively executed as follows:

[0152] Call the Netty framework to transmit the first byte stream corresponding to the request binary data to the proxy end asynchronously through the UDS link established between the application client and the proxy end by a preset thread; wherein, the preset thread and the thread where the future object is located are different.

[0153] Specifically, a setting method available for the application client provided in this application is that the Netty framework is deployed therein. Therefore, for step 203 executed, where the application client transmits the first byte stream corresponding to the request binary data to the proxy end with which it has a communication connection, it can be specifically executed as follows: Call the Netty framework to transmit the first byte stream corresponding to the request binary data to the proxy end asynchronously through the UDS link established between the application client and the proxy end by a preset thread different from the thread where the future object is located.

[0154] Please continue to refer to Figure 4 In an exemplary embodiment, for step 204 executed above, which is to receive the second byte stream returned by the proxy end and process the second byte stream into response binary data, it can be alternatively executed as follows:

[0155] The Netty framework receives the second byte stream returned by the proxy end, and performs packet splitting on the second byte stream based on the request header length and the request body length in the message format of the preset channel protocol to obtain the response binary data.

[0156] Specifically, a setting method available for the application client provided in this application is that the Netty framework is deployed therein. Therefore, for the content that in step 204, the application client receives the second byte stream returned by the proxy end and processes the second byte stream into response binary data, it can be specifically executed as follows: Receive the second byte stream returned by the proxy end through the Netty framework deployed in the application client, and also perform packet splitting processing on the second byte stream according to the request header length and the request body length in the message format of the preset channel protocol to obtain the response binary data corresponding to the second byte stream, which is convenient for subsequent decoders to identify and process the data.

[0157] Please continue to refer to Figure 4 In an exemplary embodiment, for step 205 executed above, which is to sequentially call the fourth decoder of the preset channel protocol and the third decoder of the preset message body protocol deployed in the application client to decode the response binary data into result binary data, it can be specifically executed through steps 251 - 253, where:

[0158] Step 251, call the fourth decoder of the preset channel protocol deployed on the application client to decode the response binary data into a response object;

[0159] Step 252, based on the decoding of the response binary data by the fourth decoder, extract the RequestId, obtain the corresponding future object based on the RequestId, unblock the thread where the future object is located, and return the response object corresponding to the sending method;

[0160] Step 253, call the third decoder of the preset message body protocol deployed on the application client to decode the response object into result binary data.

[0161] Specifically, execute Step 251, call the fourth decoder of the preset channel protocol deployed on the application client to perform the first decoding on the response binary data, specifically decode the response binary data into a response object through the fourth decoder, and then execute Step 252. Based on the decoding of the response binary data by the fourth decoder, extract the RequestId, obtain the corresponding future object based on the RequestId, unblock the thread where the future object is located at the same time, and return the response object corresponding to the sending method; then execute Step 253, call the third decoder of the preset message body protocol deployed on the application client to perform one-side decoding processing on the response object, and decode the response object into result binary data; the result binary data includes the result value corresponding to the request keyword in the data server, so as to realize the application client's acquisition of the result value associated with the request keyword in the data server, and then the result value can be provided to the user, so as to realize that after the user inputs the request keyword in the application client, the application client extracts the result value associated with the request keyword from the data server through the proxy.

[0162] Figure 5 The following is a schematic flowchart of a data acquisition method provided in an embodiment of the present application. Please refer to Figure 5 , and specifically provide an overall communication process of a client SDK (application client) and a sidecar (proxy), including Steps S1 - S19:

[0163] Step S1. First, when the runtime client SDK is initialized, a UDS link between the client and the Sidecar will be established.

[0164] Step S2. After the initialization is completed, the user calls the String get(String key) method through the client SDK, corresponding to the GET command of redis.

[0165] Step S3, RespEncoder is the encoder of the resp protocol. It encodes the binary content corresponding to the redis GET command according to the key input by the user and the format of the resp protocol.

[0166] Step S4: Next, construct the request object of the private protocol, generate the RequestId for multiplexing, and set it in the frame header. Then put the necessary fields required by the sidecar side into the header, put the binary content of Resp into the request body, and finally use the encoder of the private protocol to encode the request object into binary data.

[0167] Step S5: After getting the binary data of the request body, prepare to call Netty's Send() method to send the data asynchronously. This method will return a Java Future object, which will be saved in a multiplexing-specific Map (Map collection), and the corresponding key is the requestId generated above. The Future object can be found later through the requestId, and then Future is notified to complete the entire asynchronous operation.

[0168] Step S6: After calling the above method, Netty will send the byte stream asynchronously through another thread.

[0169] Step S7: On the other side, the network framework Mosn at the bottom of the sidecar receives the byte stream and cuts the packet according to the header length and body length to solve the packet sticking problem. After obtaining the complete request body binary data, the decoder of the private protocol is called to decode the binary data into the corresponding request object, but the request body data (resp protocol data) has not yet been decoded.

[0170] Step S8: Call the Resp protocol decoder RespDecoder to decode the request body to obtain the corresponding Redis command (GET command) and key parameters.

[0171] Step S9: Then, the Redis command and parameters are passed in, and the corresponding method of the Go-redis client is called.

[0172] Step S10, after a series of processing, the Go-redis client initiates a request to the Redis server, during which the Resp protocol request is encoded once.

[0173] Step S11, after receiving the command and finding the specified key, the Redis server returns a response of Resp.

[0174] Step S12, the Go-redis client receives the Resp response, but does not parse the response anymore, but directly returns it as a method result. This is an optimization point of the present application. The present application modifies the open source Go-redis source code, shields the original logic of parsing the Resp response, and allows it to directly return the binary data of the Resp response, thus avoiding the consumption of one Resp protocol decoding.

[0175] Step S13: Since the Go-redis client returns binary data, RespEncoder does not need to decode it again, but directly puts the binary data into the response body of the private protocol.

[0176] Step S14: Then construct a response object of the private protocol, set the status bit in the fixed frame header, and also set the requestId in the request frame header to the response frame header to facilitate multiplexing by the client. Then use the private protocol encoder Encoder to encode the response object into binary data.

[0177] Step S15: After obtaining the encoded response data, call the Mosn related method to send a binary byte stream to Netty through the UDS link.

[0178] Step S16, Netty receives the byte stream, and also cuts the packet according to the header length and body length, and then passes the complete response content to the decoder Decoder of the private protocol for decoding to obtain the response object Response of the private protocol.

[0179] Step S17, after the above decoding process, the RequestId in the frame header can be obtained, and then the RequestId is used to obtain the corresponding Future object, and the thread where the Future is located is notified to unblock, and the response result of the Send() method above is returned, that is, the Response object.

[0180] Step S18: The response body content in the Response object has not been decoded yet and needs to be processed by the decoder of the Resp protocol and decoded into the execution result of the redis command.

[0181] Step S19: Finally, make a judgment based on the execution result of Redis. If successful, return the Value obtained by the GET command; if failed, throw an exception.

[0182] To verify the optimization effect of the data acquisition solution provided in this application, performance stress testing was also conducted in this application. The stress testing specifications are as follows: the business container is configured with 4 cores and 8GB of memory, the sidecar container is configured with 4 cores and 2GB of memory, the stress testing concurrency is 6 concurrent requests, the stress testing tool used is Jmeter in Java, and the stress testing interface is the GET command of Redis.

[0183] When the business container only uses the native Redis SDK, the stress testing results are shown in Table 4, and the average latency is 0.48 ms.

[0184] Table 4

[0185]

[0186] Table 5

[0187]

[0188] When the business uses the optimized Sidecar during application runtime, the stress testing results are shown in Table 5, and the average latency is 0.55 ms.

[0189] It can be seen that after using the sidecar for access, the overall average latency loss is 0.07 ms, while the latency loss before optimization is 0.28 ms, and the optimization effect reaches more than 70%.

[0190] Therefore, the solution to optimize the communication performance during application runtime based on the private protocol can greatly reduce the latency loss brought by accessing the application runtime, achieving the effect of cost reduction and efficiency improvement.

[0191] It can be seen that this application uses the self-developed private protocol runtime-protocol to replace the grpc protocol as the communication protocol during application runtime; uses Uds instead of tcp communication to optimize the performance of the transport layer; in the Redis scenario, uses the resp protocol to replace the protobuf protocol as the content encoding protocol. In the Redis scenario, the Go-Redis client is modified so that it does not parse the Resp protocol response, which is beneficial to reducing performance loss.

[0192] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the indications of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0193] Based on the same inventive concept, an embodiment of the present application further provides a data acquisition device for implementing the data acquisition method involved above. The implementation solution provided by this device to solve the problem of poor performance in the data acquisition process is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more embodiments of the data acquisition device provided below can refer to the limitations on the data acquisition method in the foregoing text, and will not be repeated here.

[0194] Figure 6 The following is a structural block diagram of the data acquisition device in the embodiment provided by the present application. Please refer to Figure 2 Refer to Figure 6 , in an exemplary embodiment, as Figure 6 shown, a data acquisition device 200 is provided. This device is applied to the proxy side and includes: a data acquisition module 81, a data processing module 82, and a data interaction module 83, where:

[0195] The data acquisition module 81 is used to receive the first byte stream transmitted by the application client and process the first byte stream into request binary data; the first byte stream is used to represent the request keyword input by the user in the application client and the data acquisition instruction matching the request keyword, which are successively encoded into binary data by the first encoder of the preset message body protocol deployed in the application client and the second encoder of the preset channel protocol;

[0196] The data processing module 82 is used to sequentially call the second decoder of the preset channel protocol and the first decoder of the preset message body protocol deployed in the proxy side to sequentially decode the request binary data to obtain the parameters of the request keyword and the data acquisition instruction matching the request keyword;

[0197] The data processing module 82 is further configured to call a data client to encode the parameters of the request keyword and the data acquisition instruction based on a preset message body protocol, and send a relevant data request to the data server based on the encoded parameters of the request keyword and the data acquisition instruction; to instruct the data server to query based on the data request and return the result binary data corresponding to the request keyword to the data client, so that the data client transparently transmits the result binary data to a third encoder of the preset message body protocol deployed on the proxy end; wherein, the result binary data includes the result value corresponding to the request keyword in the data server.

[0198] The data processing module 82 is further configured to call a fourth encoder of a preset channel protocol deployed on the proxy end to encode the response object corresponding to the result binary data into response binary data.

[0199] The data interaction module 83 is configured to return a second byte stream corresponding to the response binary data to the application client, so as to transmit the result value required by the application client.

[0200] Specifically, the data acquisition module 81 is configured to receive a first byte stream transmitted by the application client to the proxy end, and process the first byte stream into binary data that can be recognized by subsequent decoders. The processed binary data is specifically request binary data; the "request" in the "request binary data" here is only a naming limitation for the binary data, used to distinguish it from the binary data obtained after other steps in this application.

[0201] It should be added that the user inputs a request keyword in the application client, and the request keyword is matched with a corresponding data acquisition instruction. The request keyword and the corresponding data acquisition instruction are encoded into binary data by a first encoder of a preset message body protocol and a second encoder of a preset channel protocol deployed on the application client in sequence. The first byte stream is used to represent the binary data, that is, the first byte stream is a form of the binary data and includes the content corresponding to the binary data.

[0202] It should be added that the application client in this application can specifically be a client SDK, and the proxy end can specifically be Sidecar or Layotto.

[0203] The data processing module 82 is configured to first call a second decoder of a preset channel protocol deployed on the proxy end to perform a first decoding on the request binary data through the second decoder, and then call a first decoder of a preset message body protocol deployed on the proxy end to perform a second decoding on the request binary data that has been first decoded through the first decoder. That is, after the request binary data is decoded twice in sequence through the second decoder and the first decoder, the parameters of the request keyword and the data acquisition instruction matching the request keyword can be obtained.

[0204] It should be noted that since the user inputs the request keyword and the data acquisition instruction matching the request keyword in the application client, and only after two encodings in the application client is it transmitted to the proxy end. Therefore, after receiving the first byte stream, the proxy end needs to go through two decodings to obtain the parameter of the request keyword input by the user in the application client and the data acquisition instruction matching the request keyword.

[0205] The data processing module 82 is further configured to call the data client deployed on the proxy end to encode the parameter of the request keyword and the data acquisition instruction based on a preset message body protocol, so as to obtain data that can be recognized by the data server. Furthermore, based on the encoded parameter of the request keyword and the data acquisition instruction, it sends a relevant data request to the data server to instruct the data server to query the data requested by the client based on the data request encoded by the data client and return the result binary data corresponding to the request keyword to the data client. The result binary data includes the result value corresponding to the request keyword in the data server. After receiving the result binary data corresponding to the request keyword returned by the data server, the data client does not parse the binary data, but calls the data client to directly pass through the result binary data to the third encoder of the preset message body protocol deployed on the proxy end, and the third encoder further processes the result binary data.

[0206] In this application, the data client does not parse the result binary data returned by the data server, reducing one data parsing process in the data acquisition process. Specifically, it avoids the consumption of decoding the data once, which is beneficial to improving the transmission efficiency of data in the proxy end.

[0207] The data processing module 82 is further configured to call the fourth encoder of the preset channel protocol deployed on the proxy end to further encode the response object received from the side of the third encoder. Here, the response object is the data obtained after the result binary data undergoes certain data processing steps based on the preset channel protocol. Specifically, the third encoder encodes the response object into response binary data, that is, encodes the data it receives into a data format that can be further transmitted to the application client to ensure the smooth transmission of data from the proxy end to the application client.

[0208] It should be added that the "response" in the "response binary data" is only a naming limitation for the binary data, used to distinguish it from the "request binary data" that has appeared in this application. Obviously, the request binary data and the response binary data are obtained through different data processing steps.

[0209] The data interaction module 83 is used to process the response binary data into a corresponding second byte stream, and then transmit the second byte stream to the application client, so as to implement the transmission of the result value that the application client needs to obtain in the data server for communication based on the proxy end to the application client.

[0210] Among them, the second byte stream is used to represent the response binary data, that is, the second byte stream is a form of the response binary data and includes the content corresponding to the response binary data.

[0211] In an exemplary embodiment, the proxy end includes a Mosn framework; the data acquisition module 81 receives the first byte stream transmitted by the application client and processes the first byte stream into request binary data, including that the Mosn framework receives the first byte stream transmitted by the application client and performs packet cutting on the first byte stream based on the request header length and request body length in the message format of the preset channel protocol to obtain the request binary data. For details, please refer to Figure 2 , and the above description about Figure 2 .

[0212] In an exemplary embodiment, the data interaction module 83 transmits the second byte stream corresponding to the response binary data back to the application client, including: calling the Mosn framework to transmit the second byte stream corresponding to the response binary data back to the application client through the UDS link established between the application client and the proxy end. For details, please refer to Figure 2 , and the above description about Figure 2 .

[0213] In an exemplary embodiment, the data processing module 82 sequentially calls the second decoder of the preset channel protocol and the first decoder of the preset message body protocol deployed on the proxy end to sequentially decode the request binary data to obtain the parameters of the request keyword and the data acquisition instruction matching the request keyword, including: calling the second decoder of the preset channel protocol deployed on the proxy end to decode the request binary data into a corresponding request object; calling the first decoder of the preset message body protocol deployed on the proxy end to decode the request object into the parameters of the request keyword and the data acquisition instruction matching the request keyword. For details, please refer to Figure 2 , and the above description about Figure 2 .

[0214] In an exemplary embodiment, before the data processing module 82 executes the fourth encoder that invokes the preset channel protocol deployed on the proxy side to encode the response object corresponding to the result binary data into response binary data, optionally, the data processing module 82 is used to put the result binary data into the response body in the message format of the preset channel protocol, set the status bit of the response frame header in the message format of the preset channel protocol, and set the RequestId in the request frame header of the request object to the response frame header, so as to construct the response object corresponding to the result binary data. For details, please refer to Figure 2 , as well as the above description about Figure 2 .

[0215] In an exemplary embodiment, the preset channel protocol includes a data transmission layer, a preset protocol architecture layer, and a data layer; wherein, the message format of the preset channel protocol includes a fixed-length frame header, a variable-length request header, and a variable-length request body.

[0216] Figure 7 Another structural block diagram of the data acquisition device in the embodiments provided by this application is shown in Figure 4 Please refer to Figure 7 . In an exemplary embodiment, as shown in Figure 7 , a data acquisition device 300 is provided. This device is applied to an application client and includes: a data acquisition module 91, a data processing module 92, and a data interaction module 93, where:

[0217] The data acquisition module 91 is used to receive a request keyword input by a user and call the keyword method to correspond to a data acquisition instruction that matches the request keyword on the data server; wherein, the data server communicates based on the proxy side and the application client.

[0218] The data processing module 92 is used to sequentially call the first encoder of the preset message body protocol deployed on the application client and the second encoder of the preset channel protocol to encode the request keyword and the data acquisition instruction in sequence to obtain request binary data.

[0219] The data interaction module 93 is used to transmit the first byte stream corresponding to the request binary data to the proxy side and wait to receive the data returned by the proxy side.

[0220] The data interaction module 93 is further used to receive the second byte stream returned by the proxy side and process the second byte stream into response binary data; wherein, the second byte stream is used to represent the result value corresponding to the request keyword extracted from the data server and encoded into binary data after being transparently transmitted through the third encoder of the preset message body protocol deployed on the proxy side to the fourth encoder of the preset channel protocol.

[0221] The data processing module 92 is further configured to sequentially call the fourth decoder of the preset channel protocol deployed on the application client and the third decoder of the preset message body protocol to decode the response binary data into result binary data, so as to enable the application client to obtain the result value associated with the request keyword in the data server; wherein, the result binary data includes the result value corresponding to the request keyword in the data server.

[0222] Specifically, the data acquisition module 91 is configured to receive the request keyword input by the user through the application client, and the application client can call the keyword method to map the request keyword input by the user to the data acquisition instruction in the data server that matches the request keyword. That is, the data acquisition module 91 can obtain the request keyword input by the user and the data acquisition instruction that matches the request keyword, so as to know what content the user needs to extract from the data server.

[0223] The data processing module 92 is configured to first call the first encoder of the preset message body protocol deployed on the application client to perform the first encoding on the request keyword and the data acquisition instruction, and then call the second encoder of the preset channel protocol deployed on the application client to perform the second encoding on the request keyword and the data acquisition instruction that have been first encoded, so as to obtain the request binary data carrying the request keyword and the data acquisition instruction. Through this step, the request keyword and the data acquisition instruction are encoded into data that can be transmitted between the application client and the proxy through the communication protocol.

[0224] The data interaction module 93 is configured to process the request binary data into a corresponding first byte stream to achieve the transmission of binary data between the application client and the proxy. The first byte stream can transmit more data formats, which is beneficial to enhancing the diversity of data transmission between the application client and the proxy; then, the application client waits to receive the relevant data queried from the data server and returned by the proxy.

[0225] After the proxy extracts the result value corresponding to the request keyword from the data server communicating with it, the result value will be transparently transmitted through the third encoder of the preset message body protocol deployed on the proxy to the fourth encoder of the preset channel protocol in sequence. The fourth encoder will encode the data carried by the result value into (response) binary data, and the response binary data will be processed into a (second) byte stream and then returned to the application client; therefore, the data interaction module 93 is further configured to receive the second byte stream returned by the proxy, and then can process the second byte stream into response binary data so that the subsequent decoder can recognize it.

[0226] The data processing module 92 is also configured to first call the fourth decoder of the preset channel protocol deployed on the application client to perform a first decoding on the response binary data, and then call the third decoder of the preset message body protocol to perform a second decoding on the response binary data that has been first decoded, so as to obtain the decoded result binary data. The result binary data includes the result value corresponding to the request keyword in the data server, so as to implement the application client's acquisition of the result value associated with the request keyword in the data server. Then, the result value can be provided to the user, so as to implement that after the user inputs the request keyword in the application client, the application client extracts from the data server through the proxy the result value associated with the request keyword.

[0227] In an exemplary embodiment, the data processing module 92 successively calls the first encoder of the preset message body protocol and the second encoder of the preset channel protocol deployed on the application client to successively encode the request keyword and the data acquisition instruction, and obtain the request binary data, including: calling the first encoder of the preset message body protocol deployed on the application client, and encoding according to the request keyword and in combination with the preset message body protocol to obtain the binary content corresponding to the data acquisition instruction; generating a RequestId for multiplexing, setting it into the request frame header of the message format of the preset channel protocol, putting the requirement field of the proxy into the request header of the message format, and putting the binary content into the request body of the message format, so as to construct a request object associated with the binary content of the preset channel protocol; calling the second encoder of the preset channel protocol deployed on the application client to encode the request object into the request binary data. For details, reference can be made to Figure 4 , and the above description about Figure 4 .

[0228] In an exemplary embodiment, the application client includes a Netty framework; before the data interaction module 93 transmits the first byte stream corresponding to the request binary data to the proxy, the data processing module 92 is also configured to call the send method of the Netty framework to asynchronously send the request binary data, and save the future object returned by the send method into the multiplexed data storage set; wherein, the content associated with the request keyword in the data storage set is the RequestId, and the RequestId is used to associate the future object. For details, reference can be made to Figure 4 , and the above description about Figure 4 .

[0229] In an exemplary embodiment, the first byte stream corresponding to the request binary data executed by the data interaction module 93 includes: calling the Netty framework to establish a UDS link between the application client and the proxy end, and asynchronously transmitting the first byte stream corresponding to the request binary data to the proxy end through a preset thread; wherein, the preset thread and the thread where the future object is located are different. For details, refer to Figure 4 , and the above description about Figure 4 .

[0230] In an exemplary embodiment, the data interaction module 93 executes receiving the second byte stream returned by the proxy end and processing the second byte stream into response binary data, including: the Netty framework receives the second byte stream returned by the proxy end, and performs packet splitting on the second byte stream based on the request header length and the request body length in the message format of the preset channel protocol to obtain the response binary data. For details, refer to Figure 4 , and the above description about Figure 4 .

[0231] In an exemplary embodiment, the data processing module 92 executes successively calling the fourth decoder of the preset channel protocol and the third decoder of the preset message body protocol deployed on the application client to decode the response binary data into result binary data, including: calling the fourth decoder of the preset channel protocol deployed on the application client to decode the response binary data into a response object; based on the decoding of the response binary data by the fourth decoder, extracting the RequestId, obtaining the corresponding future object based on the RequestId, and releasing the block of the thread where the future object is located, and returning the response object corresponding to the sending method; calling the third decoder of the preset message body protocol deployed on the application client to decode the response object into result binary data. For details, refer to Figure 4 , and the above description about Figure 4 .

[0232] Each module in the above data acquisition device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to the above respective modules.

[0233] Figure 8 The following is the internal structure diagram of the computer device in the embodiment provided by this application. In an exemplary embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 8As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program, when executed by the processor, is used to implement a data acquisition method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0234] Those skilled in the art can understand that Figure 8 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0235] Based on the same inventive concept, this application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the aforementioned data acquisition method, and the data acquisition method is any one of the data acquisition methods mentioned in the embodiments of this application. For related embodiments, refer to the above text.

[0236] Based on the same inventive concept, this application also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the aforementioned data acquisition method, and the data acquisition method is any one of the data acquisition methods mentioned in the embodiments of this application. For related embodiments, refer to the above text.

[0237] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0238] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., and are not limited thereto.

[0239] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0240] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A data acquisition method, characterized in that, Applied to the proxy side, including: Receiving a first byte stream transmitted by an application client, and processing the first byte stream into request binary data; the first byte stream is used to represent a request keyword input by a user in the application client and a data acquisition instruction matching the request keyword, and is encoded into binary data by a first encoder of a preset message body protocol and a second encoder of a preset channel protocol deployed in the application client in sequence; Sequentially calling a second decoder of the preset channel protocol and a first decoder of the preset message body protocol deployed on the proxy side to sequentially decode the request binary data to obtain parameters of the request keyword and the data acquisition instruction matching the request keyword; Calling a data client to encode the parameters of the request keyword and the data acquisition instruction based on the preset message body protocol, and sending a relevant data request to a data server based on the encoded parameters of the request keyword and the data acquisition instruction; to instruct the data server to query based on the data request and return result binary data corresponding to the request keyword to the data client, so that the data client transparently transmits the result binary data to a third encoder of the preset message body protocol deployed on the proxy side; wherein, the result binary data includes a result value corresponding to the request keyword in the data server; Calling a fourth encoder of the preset channel protocol deployed on the proxy side to encode a response object corresponding to the result binary data into response binary data; Returning a second byte stream corresponding to the response binary data to the application client to implement transmitting the result value required by the application client.

2. The data acquisition method according to claim 1, wherein The proxy side includes a Mosn framework; The receiving the first byte stream transmitted by the application client and processing the first byte stream into request binary data includes: The Mosn framework receives the first byte stream transmitted by the application client, and cuts the first byte stream based on the request header length and request body length in the message format of the preset channel protocol to obtain request binary data.

3. The data acquisition method according to claim 2, wherein The returning the second byte stream corresponding to the response binary data to the application client includes: Calling the Mosn framework to return the second byte stream corresponding to the response binary data to the application client through a UDS link established between the application client and the proxy side.

4. The data acquisition method according to claim 1, wherein The sequentially calling the second decoder of the preset channel protocol and the first decoder of the preset message body protocol deployed on the proxy side to sequentially decode the request binary data to obtain the parameters of the request keyword and the data acquisition instruction matching the request keyword includes: Calling the second decoder of the preset channel protocol deployed on the proxy side to decode the request binary data into a corresponding request object; Invoke the first decoder of the preset message body protocol deployed on the proxy end to decode the request object into the parameters of the request keyword and the data acquisition instruction matching the request keyword.

5. The data acquisition method according to claim 1, wherein before invoking the fourth encoder of the preset channel protocol deployed on the proxy end to encode the response object corresponding to the result binary data into response binary data, further comprising: Put the result binary data into the response body in the message format of the preset channel protocol, set the status bit of the response frame header in the message format of the preset channel protocol, and set the RequestId in the request frame header of the request object to the response frame header to construct the response object corresponding to the result binary data.

6. The data acquisition method according to any one of claims 1-5, wherein the preset channel protocol includes a data transmission layer, a preset protocol architecture layer, and a data layer; wherein, the message format of the preset channel protocol includes a fixed-length frame header, a variable-length request header, and a variable-length request body.

7. A data acquisition method, characterized in that, Applied to an application client, it includes: Receive a request keyword input by a user, and call the keyword method to correspond to a data acquisition instruction matching the request keyword on the data server; wherein, the data server communicates with the application client based on a proxy end. Successively invoke the first encoder of the preset message body protocol and the second encoder of the preset channel protocol deployed on the application client to successively encode the request keyword and the data acquisition instruction to obtain request binary data. Transmit the first byte stream corresponding to the request binary data to the proxy end and wait to receive the data returned by the proxy end. Receive the second byte stream returned by the proxy end and process the second byte stream into response binary data; wherein, the second byte stream is used to represent the result value corresponding to the request keyword extracted from the data server, and is successively encoded into binary data after passing through the third encoder of the preset message body protocol deployed on the proxy end and transmitted to the fourth encoder of the preset channel protocol. Successively invoke the fourth decoder of the preset channel protocol and the third decoder of the preset message body protocol deployed on the application client to decode the response binary data into result binary data, so as to realize the acquisition of the result value associated with the request keyword in the data server by the application client; wherein, the result binary data includes the result value corresponding to the request keyword in the data server.

8. The data acquisition method according to claim 7, wherein the successively invoking the first encoder of the preset message body protocol and the second encoder of the preset channel protocol deployed on the application client to successively encode the request keyword and the data acquisition instruction to obtain request binary data includes: Calling a first encoder of a preset message body protocol deployed on the application client, encoding the binary content corresponding to the data acquisition instruction according to the request keyword and the preset message body protocol; Generate a RequestId for multiplexing, and set it to the request frame header of the message format of the preset channel protocol, and put the requirement field of the proxy end into the request header of the message format, and put the binary content into the request body of the message format to construct a request object of the preset channel protocol associated with the binary content; A second encoder of a preset channel protocol deployed on the application client is called to encode the request object into the request binary data.

9. The data acquisition method according to claim 8, wherein The application client includes a Netty framework; Before transmitting the first byte stream corresponding to the request binary data to the proxy end, the method further includes: The sending method of the Netty framework is called to asynchronously send the request binary data, and the future object returned by the sending method is saved in a multiplexed data storage set; wherein the content associated with the request keyword in the data storage set is RequestId, and RequestId is used to associate with the future object.

10. The data acquisition method according to claim 9, characterized in that: The transmitting the first byte stream corresponding to the request binary data to the proxy end includes: The Netty framework is called to establish a UDS link between the application client and the proxy end, and a first byte stream corresponding to the requested binary data is asynchronously transmitted to the proxy end through a preset thread; wherein the preset thread and the thread where the future object is located are different.

11. The data acquisition method according to claim 10, characterized in that: The receiving the second byte stream returned by the proxy end and processing the second byte stream into response binary data includes: The Netty framework receives the second byte stream returned by the proxy end, cuts the second byte stream into packets based on the request header length and the request body length in the message format of the preset channel protocol, and obtains response binary data.

12. The data acquisition method according to claim 11, characterized in that: The sequentially calling the fourth decoder of the preset channel protocol and the third decoder of the preset message body protocol deployed on the application client to decode the response binary data into result binary data includes: Calling a fourth decoder of the preset channel protocol deployed on the application client to decode the response binary data into a response object; Extracting RequestId based on decoding of the response binary data by the fourth decoder, acquiring the corresponding future object based on RequestId, unblocking the thread where the future object is located, and returning the response object corresponding to the sending method; A third decoder of the preset message body protocol deployed on the application client is called to decode the response object into result binary data.

13. A data acquisition device, characterized in that, Applied to the agent side, including: A data acquisition module, configured to receive a first byte stream transmitted by an application client, and process the first byte stream into request binary data; the first byte stream is used to represent a request keyword input by a user in the application client and a data acquisition instruction matching the request keyword, which are successively encoded into binary data by a first encoder of a preset message body protocol and a second encoder of a preset channel protocol deployed in the application client. A data processing module, configured to successively call a second decoder of the preset channel protocol and a first decoder of the preset message body protocol deployed in the proxy end to successively decode the request binary data, so as to obtain parameters of the request keyword and the data acquisition instruction matching the request keyword. The data processing module is further configured to call a data client to encode the parameters of the request keyword and the data acquisition instruction based on the preset message body protocol, and send a relevant data request to a data server based on the encoded parameters of the request keyword and the data acquisition instruction; to instruct the data server to query based on the data request and return result binary data corresponding to the request keyword to the data client, so that the data client transparently transmits the result binary data to a third encoder of the preset message body protocol deployed in the proxy end; wherein the result binary data includes a result value corresponding to the request keyword in the data server. The data processing module is further configured to call a fourth encoder of the preset channel protocol deployed in the proxy end to encode a response object corresponding to the result binary data into response binary data. A data interaction module, configured to send back a second byte stream corresponding to the response binary data to the application client, so as to transmit the result value required by the application client.

14. A data acquisition device, characterized in that, Applied to an application client, including: A data acquisition module, configured to receive a request keyword input by a user, and call a keyword method to correspond to a data acquisition instruction matching the request keyword in a data server; wherein the data server communicates with the application client based on a proxy end. A data processing module, configured to successively call a first encoder of a preset message body protocol and a second encoder of a preset channel protocol deployed in the application client to successively encode the request keyword and the data acquisition instruction, so as to obtain request binary data. A data interaction module, configured to transmit a first byte stream corresponding to the request binary data to the proxy end, and wait to receive the data sent back by the proxy end. The data interaction module is further configured to receive a second byte stream sent back by the proxy end, and process the second byte stream into response binary data; wherein the second byte stream is used to represent a result value corresponding to the request keyword extracted from the data server, which is successively encoded into binary data after being transparently transmitted from a third encoder of a preset message body protocol deployed in the proxy end to a fourth encoder of a preset channel protocol. The data processing module is further configured to sequentially call a fourth decoder of the preset channel protocol and a third decoder of the preset message body protocol deployed on the application client to decode the response binary data into result binary data, so as to implement the application client's acquisition of the result value associated with the request keyword in the data server; wherein, the result binary data includes the result value corresponding to the request keyword in the data server.

15. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 or claims 7 to 12 are implemented.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6, or claims 7 to 12 are implemented.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6, or claims 7 to 12 are implemented.

Images