A microservice-based data access method and system

Abstract

The application relates to a micro-service-based data access method, which comprises the following steps: S1, acquiring a first request server corresponding to a data access request, and sending a client data access request to the first request server; S2, judging whether the first request server satisfies a local execution condition, and if yes, the first request server locally executes the data access request, and sends a request result to the client; the application can effectively reduce memory and CPU usage, and greatly improves data access efficiency.

Description

Technical Field

[0001] This invention belongs to the field of human-computer interaction technology, and in particular relates to a data access method and system based on microservices. Background Technology

[0002] Making human-computer interaction more convenient and efficient has always been a focus of attention in academia and industry. On the one hand, with the development of mobile internet technology, human-computer terminals have gradually expanded from traditional human-computer workstations to mobile terminal devices such as tablets and mobile phones. Achieving interactive interfaces with low resource consumption and support for unified display across multiple terminals, providing users with data access, is an important direction for current human-computer research. On the other hand, user experience requirements are becoming increasingly demanding. Existing systems' human-computer interfaces, in terms of interaction methods, intelligence levels, and display effects, can only keep pace with user requirements and meet the ever-growing demands for user experience. With the continuous improvement of terminal capabilities, in order to reduce server pressure and fully utilize client resources, some human-computer architectures typically perform a large amount of screen display processing and complex service calls on the client side during design, with caching concentrated on the foreground client, resulting in significant resource consumption. Others exercise complete control on the server side. The server is not the initiator of the request, nor does it belong exclusively to a single user. Therefore, allowing the server, or even multiple servers, to directly control the completion of the entire request, especially complex data requests, will bring more problems. In this context, how to reduce client resource requirements without compromising user experience is an urgent problem to be solved. This invention effectively reduces memory and CPU usage, significantly improving data access efficiency. Based on a microservice framework, it allocates microservice requests, ensuring the microservice framework resides in the memory of these request servers, preventing excessive server idle time and repeated initialization and destruction of the microservice framework, thus maintaining the server's hot state. A neural network model utilizes large datasets to find the relationship between the number of microservice types and request characteristics to locate the optimal server. Through a microservice graph, its decomposition, and node labeling, it supports the rational flow of microservice requests between servers, maximizing the multi-server utilization of microservices. Summary of the Invention

[0003] To address the aforementioned problems in the prior art, this invention proposes a microservice-based data access method and system, the method comprising:

[0004] Step S1: Obtain the first request server corresponding to the data access request, and send the client data access request to the first request server; the first request server is one of the request servers that can provide data access requests; select a first request server based on the status of the request server, and direct the data access request to the first request server;

[0005] Step S2: Determine whether the first request server meets the localized execution conditions. If so, the first request server executes the data access request locally and sends the request result to the client; otherwise, the data access request is split into a set of microservices; the execution of the data access request is controlled based on the set of microservices on the first request server or on other servers.

[0006] Furthermore, the state of the server is the server's busy level and / or the vector of the number of microservice types in the request server environment, MT = [mti].

[0007] Furthermore, the step of selecting a first request server based on the status of the request server specifically means: selecting the least busy request server from among the request servers as the first request server.

[0008] Furthermore, determining that the first request server meets the localization execution conditions specifically means: when the busy level of the first request server indicates that the first request server is idle, it is determined that the first request server meets the localization execution conditions.

[0009] Furthermore, the local execution of the data access request specifically involves: breaking down the execution of the data access request into microservice requests, and completing the corresponding microservice requests based on the local microservice framework.

[0010] A microservice-based data access system, the system comprising: a client and a server.

[0011] Furthermore, the client can be one or more.

[0012] Furthermore, the client is used to initiate data access requests and receive the request results from the server; the client is also used to control the data access requests based on the microservice graph and ultimately obtain the request results.

[0013] Furthermore, there may be one or more servers.

[0014] Furthermore, the server is used to receive data access requests and return the results of the data access requests to the client; the data access requests are broken down into microservices for execution, so that more requests can be shared by the microservice framework. The microservice framework itself located on the server is not destroyed when a microservice is completed, but continues to execute the next microservice.

[0015] The beneficial effects of this invention include: (1) The microservice framework enables the sharing of resources under a lightweight structure. Since the services have been lightweighted, the possibility of sharing is greatly increased; (2) Through the allocation of microservice requests, the microservice framework resides in the memory of these request servers, ensuring that the servers do not have a large amount of idle time and that the initialization and destruction of the microservice framework are repeated, thus maintaining the hot state of the servers; (3) Starting from fine granularity, the request server most likely to satisfy the request locally is selected as the first request server, and the request service is localized as much as possible; furthermore, by dividing the input layer into blocks, the attributes of each dimension can quickly converge, improving the training and usage efficiency of the model; through the neural network model, the relationship between the number of microservice types and request characteristics is found using big data samples to locate the optimal server; (4) By matching requests with the site, a large amount of framework reconstruction overhead is saved; through the microservice graph and its splitting and node labeling, the reasonable flow of microservice requests between servers is supported, maximizing the multi-server utilization of microservices. Attached Figure Description

[0016] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to unduly limit the invention. In the drawings:

[0017] Figure 1 This is a schematic diagram of the microservice-based data access method of the present invention. Detailed Implementation

[0018] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The illustrative embodiments and descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0019] The present invention relates to a microservice-based data access system, comprising: a client and a server; wherein: the client may be one or more; the client is used to initiate a data access request and receive the request result of the server access request from the server; the client is also used to control the data access request based on the microservice graph and finally obtain the request result;

[0020] There are one or more servers. Servers receive data access requests and return the results to the client. Data access requests are broken down into microservices for execution, allowing more requests to share the microservice framework. The microservice framework itself on the server is not destroyed when one microservice is completed, but continues to execute the next microservice. The microservice framework includes microservice programs, hardware and software environments, etc., used to provide microservices. Preferably, the microservice framework resides in memory. Different types of microservice frameworks serve corresponding types of microservices. Therefore, under the microservice framework, data flow is reduced accordingly, and data interaction can be achieved through parameter passing or data communication between frameworks.

[0021] Preferred approach: When a microservice framework is not used for a period exceeding a threshold, the microservice framework will be destroyed, and the various types of resources corresponding to the microservice framework will be released and used to build other types of microservice frameworks. This allows for the reuse of resources under a lightweight structure. Since the services have been lightweighted, the possibility of sharing resources will be greatly increased, and the reuse of resources will be better.

[0022] The server includes request servers and non-request servers. Request servers receive data access requests from clients and determine whether to execute them locally. In the case of local execution, the request server completes the data access request and returns the result to the client, maximizing access efficiency and simplifying access control. In the case of non-local execution, the client controls the execution of the request through a microservice graph. This invention differentiates the control rights for completing access requests based on the completion complexity of microservices, minimizing the client's execution complexity. In the case of non-local execution, data access requests can be completed by either the request server or the non-request server. For different types of data access requests, the same server may... It can be designated as a request server or a non-request server; one or more request servers correspond to the same type of data access request; through the corresponding settings of the request servers, specific types of data access requests can appear regularly or frequently on the corresponding request servers, so that the corresponding microservice framework resides in the memory of these request servers. When the execution of these request servers allows, the requests are completed locally on the corresponding request server first, ensuring that the server does not have a lot of idle time and repeated initialization and destruction of the microservice framework, so that the server keeps the corresponding microservice in a hot state; at the same time, the setting of the request server ensures that the selection of the first request server is within a small range, so as not to bring too much complexity to the selection of requests;

[0023] Preferred: Microservices include specific types of data reading, specific types of data writing, data rendering, data concatenation, and specific operations on data blocks;

[0024] The microservice-based data access method claimed in this invention specifically includes the following steps:

[0025] Step S1: Obtain the first request server corresponding to the data access request, and send the client data access request to the first request server; the first request server is one of the request servers that can provide data access requests; depending on the type of data access request, there may be one or more request servers, select a first request server based on the status of the request server, and direct the data access request to the first request server;

[0026] Preferably, the server status is the server's busy level and / or the vector of the number of microservice types in the request server's field MT = [mti], where: mti is the number of the i-th microservice type in the field;

[0027] The step of selecting a first request server based on the status of the request server specifically means: selecting the least busy request server from among the request servers as the first request server;

[0028] Based on the introduction of microservices, this invention selects the request server most likely to satisfy the request locally as the first request server from a fine-grained perspective, and uses microservice type vectors to characterize the situation, so as to localize the request service as much as possible.

[0029] Therefore, an alternative approach is to: obtain the multi-dimensional features of the current data access request, obtain the microservice type quantity vector of the requesting server, and select the requesting server corresponding to the microservice type quantity vector that matches the multi-dimensional attributes as the first requesting server; a simple approach is to obtain the matching microservice type quantity vector by querying the correspondence table between multi-dimensional attributes and microservice type quantity vectors; different types of data access requests have their own attributes, and access requests can be characterized by attributes. The number of corresponding microservice types and access requests of that type often have a certain correlation, so it can be obtained directly by querying the relationship table; the multi-dimensional attributes include the type of access request, the amount of data involved in the access request, etc.; the relationship table is about some attributes, such as the type of request;

[0030] Preferably, the multi-dimensional attributes include environmental information of the access request, such as hardware and software resource information, user characteristics of the requester, access request type, amount of data and computation involved in the access request, etc.; these different dimensions of information together constitute the multi-dimensional attributes of the data access request; each attribute dimension contains one or more attributes;

[0031] Preferred method: Select the first request server using a multi-layer neural network model; select the request server with the highest matching degree as the first request server;

[0032] The multi-layer neural network model has 5 layers. The first layer is the first input layer, which contains m input parts, where m is the number of dimensions of the multi-dimensional attributes. Each input part corresponds to one attribute dimension. Each input part contains Ik input neurons, where the number of Ik corresponds to the number of attributes in the k-th attribute dimension; k = [1-m]. The input of a neuron in each input part is one attribute in the corresponding dimension; that is, each specific attribute corresponds to the input of a neuron in the input part. The second layer is the feature layer, which contains m neurons. Each neuron corresponds to one attribute dimension, and the inputs of the neurons are the outputs of the corresponding input parts of the first layer. The third layer is the first fully connected layer, which contains m neurons. The input of each neuron is the output of all neurons in the second layer.

[0033] By setting the first to third layers independently, the features of each dimension can form an independent convergent structure. The fully connected layer enables the model to reflect the cross-correlation between features of different dimensions. At the same time, the training complexity and sample requirements are reduced.

[0034] The fourth layer is the adjustment layer, which contains m neurons. The input of each neuron is the output of the corresponding neuron in the third layer and the adjustment value. That is, the adjustment values ​​in the adjustment layer constitute the second input layer of the neural network model. The adjustment value is a vector of the number of microservice types. The fifth layer is the output layer, which contains one neuron. The input of this neuron is the output of all neurons in the fourth layer. The output value is the matching value, which represents the degree of matching between the multidimensional attribute value and the vector of the number of microservice types. The matching value is a value between 0 and 1.

[0035] This invention improves the training and usage efficiency of the model by dividing and layering the input layer into blocks, enabling the attributes of each dimension to converge quickly and allowing complex attributes to be integrated with the actual situation. It also uses a neural network model to find the relationship between the number of microservice types and request characteristics using big data samples to locate the optimal server.

[0036] Step S2: Determine whether the first request server meets the localized execution conditions. If so, the first request server executes the data access request locally and sends the request result to the client; otherwise, the data access request is split into a set of microservices; the execution of the data access request is controlled based on the set of microservices on the first request server or on other servers.

[0037] The determination that the first request server meets the localized execution conditions specifically means: when the busy level of the first request server indicates that the first request server is idle, it is determined that the first request server meets the localized execution conditions;

[0038] Alternative: Obtain the on-site data of the first request server, fill the quantity vector of microservice types based on the on-site data, input the quantity vector of microservice types into a multi-layer neural network model to obtain the matching degree between the data access request and the first request server, and determine that the first request server meets the localization execution conditions when the matching degree is greater than the first matching degree threshold.

[0039] The local execution of the data access request specifically involves: breaking down the execution of the data access request into microservice requests, and completing the corresponding microservice requests based on the local microservice framework; when there is no corresponding type of microservice framework locally, initializing the corresponding type of microservice framework to generate the corresponding type of microservice framework.

[0040] The process of splitting data access requests into microservice requests specifically involves: splitting the execution flow of data access requests according to microservice templates to obtain an ordered set of microservices; the ordered set of microservices contains one or more microservice requests that are the same or different.

[0041] The execution of data access requests is controlled based on a microservice collection; specifically, it includes the following steps:

[0042] Step SE1: Represent the microservices using an ordered microservice graph, where each node in the graph represents a microservice and directed edges indicate the order in which the microservices are executed.

[0043] Step SE2: Starting from the beginning node, traverse the microservice graph. If the current node can be executed by the local first request server, mark the current node as locally executed. If the current node cannot be executed by the local first request server, calculate the minimum communication overhead for the microservice corresponding to the current node to be executed by other servers. If the minimum communication overhead is less than the local initialization overhead, add the current node to the pending set. Otherwise, create a microservice framework based on the microservice type of the current node and mark the current node as locally executed. Continue traversing the next node until all nodes have been traversed.

[0044] Preferably, the traversal is a preorder traversal;

[0045] The calculation of the minimum communication overhead for the current node to be executed by other servers is specifically as follows: among the non-first request servers, find other servers of the microservice type that can directly execute the current node, and take the non-first request server with the minimum communication overhead that satisfies the microservice corresponding to the current node as the minimum communication overhead;

[0046] Preferred: Local initialization overhead is the initialization overhead required to rebuild the microservice framework for executing microservices locally;

[0047] Step SE3: Organize all nodes in the set to be processed into one or more sub-microservice graphs according to the order relationship in the microservice graph; that is, reorganize the edges in the microservice graph that are related to the nodes in the set to be processed and only related to the nodes in the set to be processed, and all nodes in the set to be processed into one or more sub-graphs.

[0048] Step SE4: For each microservice subgraph, first select another server that can satisfy the microservice type of all nodes in the microservice subgraph, and label all nodes; if there is no such other server, use the method in Step SE2 to traverse the microservice subgraph again from the starting node, label the nodes in the microservice subgraph and split the microservice subgraph, until all nodes have been labeled; after processing all microservice subgraphs, proceed to the next step;

[0049] An alternative approach is to ensure that all microservices are distributed across no more than two layers. In this case, if no other server can satisfy the microservice type of all nodes in the microservice subgraph, then select the server that can satisfy at most the microservice types of all nodes in the microservice subgraph and directly assign all nodes in this microservice subgraph to that server. For microservice types that cannot be satisfied, perform the corresponding type initialization and mark all nodes in the microservice subgraph. Similarly, the number of distributed layers can be further expanded, and can be set to be related to the size of the graph involved in the request. When the microservice graph or microservice subgraph is large, the number of distributed layers is larger, and vice versa.

[0050] Step SE5: Send the microservice graph and node labeling results to the client. The client controls the execution of the data access request based on the microservice graph and node labeling results. Specifically: According to the sequence of microservices in the microservice graph, after the microservice of the preceding node is completed, the client retains the microservice result data locally or sends it to the subsequent node and waits for the microservice of the subsequent node to complete. After the microservice graph is completed, the client concatenates the microservice result data to form the request result data of the data access request.

[0051] Preferred method: The client establishes a handshake relationship with the server corresponding to the marked node by sending a notification. When the microservice of the corresponding node is completed, a trigger response is sent to control the continuation of data interaction requests.

[0052] Preferred: The client controls the execution of data access requests based on Petri nets;

[0053] When data access requests are split, too many control nodes and control transfers can reduce server execution efficiency. In this case, giving control directly to the client can unify control and reduce server overhead.

[0054] In existing technologies, the differences between server environments are not considered when providing microservices. These differences are related to the current external environment and the server's own execution capabilities. This invention saves a significant amount of framework reconstruction overhead by matching requests with the environment. Through the microservice graph, its decomposition, and node labeling, it supports the reasonable flow of microservice requests between servers, maximizing the multi-server utilization of microservices.

[0055] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0057] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0058] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0059] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

[0060] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0061] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0062] Those skilled in the art will understand that all or part of the steps in the above method implementation can be accomplished by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A microservice-based data access method, characterized in that, The method includes: Step S1: Obtain the first request server corresponding to the data access request, and send the client's data access request to the first request server; The first requesting server is one of the requesting servers that can provide data access requests; A first request server is selected based on the status of the request server, and the data access request is directed to the first request server; Step S2: Determine whether the first request server meets the localized execution conditions. If so, the first request server executes the data access request locally and sends the request result to the client; otherwise, the data access request is split into a set of microservices. Based on the microservice collection, data access requests are executed on the first request server local or on other servers. The server status is the server's busy level and / or the vector of the number of microservice types in the request server's field MT=[mti], where: mti is the number of the i-th type of microservice in the field; The process of splitting data access requests into microservice requests specifically involves: splitting the execution flow of data access requests according to microservice templates to obtain an ordered set of microservices; the ordered set of microservices contains one or more microservice requests that are the same or different. The execution of data access requests is controlled based on a microservice collection; specifically, it includes the following steps: Step SE1: Represent the microservices using an ordered microservice graph, where each node in the graph represents a microservice and directed edges indicate the order in which the microservices are executed. Step SE2: Starting from the beginning node, traverse the microservice graph. If the current node can be executed by the local first request server, mark the current node as locally executed. If the current node cannot be executed by the local first request server, calculate the minimum communication overhead for the microservice corresponding to the current node to be executed by other servers. If the minimum communication overhead is less than the local initialization overhead, add the current node to the pending set. Otherwise, create a microservice framework based on the microservice type of the current node and mark the current node as locally executed. Continue traversing the next node until all nodes have been traversed. Step SE3: Organize all nodes in the set to be processed into one or more sub-microservice graphs according to the order relationship in the microservice graph; that is, reorganize the edges in the microservice graph that are related to the nodes in the set to be processed and only related to the nodes in the set to be processed, and all nodes in the set to be processed into one or more sub-graphs. Step SE4: For each microservice subgraph, first select another server that can satisfy the microservice type of all nodes in the microservice subgraph, and label all nodes; if there is no such other server, use the method in Step SE2 to traverse the microservice subgraph again from the starting node, label the nodes in the microservice subgraph and split the microservice subgraph, until all nodes have been labeled; after processing all microservice subgraphs, proceed to the next step; Step SE5: Send the microservice graph and node labeling results to the client. The client controls the execution of the data access request based on the microservice graph and node labeling results. Specifically: Based on the sequence of microservices in the microservice graph, after the preceding node's microservice is completed, the client retains the microservice result data locally or sends it to the subsequent node and waits for the subsequent node's microservice to complete. After the microservice graph is completed, the client concatenates the microservice result data to form the request result data of the data access request.

2. The microservice-based data access method according to claim 1, characterized in that, The step of selecting a first request server based on the status of the request server specifically means selecting the least busy request server from among the request servers as the first request server.

3. The microservice-based data access method according to claim 2, characterized in that, The determination that the first request server meets the localized execution conditions specifically means: when the busy level of the first request server indicates that the first request server is idle, it is determined that the first request server meets the localized execution conditions.

4. The microservice-based data access method according to claim 3, characterized in that, The local execution of the data access request specifically involves: breaking down the execution of the data access request into microservice requests, and completing the corresponding microservice requests based on the local microservice framework.

5. A microservice-based data access system employing the microservice-based data access method of any one of claims 1-4, characterized in that, The system includes: a client and a server.

6. The microservice-based data access system according to claim 5, characterized in that, The client can be one or more.

7. The microservice-based data access system according to claim 6, characterized in that, The client is used to initiate data access requests and receive the request results from the server; the client is also used to control the data access requests based on the microservice graph and finally obtain the request results.

8. The microservice-based data access system according to claim 7, characterized in that, There may be one or more servers.

9. The microservice-based data access system according to claim 8, characterized in that, The server is used to receive data access requests and return the results of the data access requests to the client. Data access requests are broken down into microservices for execution, so that more requests can share the microservice framework. The microservice framework itself located on the server is not destroyed when a microservice is completed, but continues to execute the next microservice.

Images