A multi-language data access proxy device based on gRPC

Abstract

The present invention belongs to the field of database technology, and specifically relates to a multi-language data access proxy device based on gRPC, including an access module, a definition module, a sinking module, a deployment module, a switching module and a central control module. The data proxy service in the present invention is stateless and can provide horizontal scaling capabilities according to business needs. It only needs to implement a set of dao data access layer logic to provide efficient and secure database table access for business projects in different programming languages. In addition, the dao data access layer is completely decoupled from the business project code, and can even complete the switching of different databases in the data access proxy layer without the business caller's perception. At the same time, a spring actuator plug-in is introduced to observe the call status and time consumption of each gRPC interface in real time, providing a reference guide for dao interface performance optimization.

Description

Technical Field

[0001] The present invention belongs to the field of database technology, and in particular relates to a multilingual data access proxy device based on gRPC. Background Art

[0002] In the era of rapid internet informatization, diverse business needs have led to a proliferation of different implementation languages ​​for business projects. This often requires projects written in multiple programming languages ​​to access the same database tables. The current common solution is to load different SQL drivers to connect to the database. However, projects written in different programming languages ​​also require implementing multiple DAO data access layers, which is not scalable.

[0003] In the existing technology, for business projects developed in different languages ​​​​to access the same database table, business projects in different programming languages ​​​​need to load different database drivers to implement multiple sets of dao data access logic. Users need to maintain multiple sets of dao data access layer code logic, which easily leads to the loss or omission of dao interface implementation. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-language data access proxy device based on gRPC, which can easily realize multi-instance deployment of proxy nodes and horizontally scale proxy nodes according to business needs.

[0005] The technical solutions adopted by the present invention are as follows:

[0006] A multi-language data access proxy device based on gRPC, including an access module, a definition module, a sinking module, a deployment module, a switching module and a central control module;

[0007] The access module is used to access the data access proxy device through a communication interface, wherein the communication interface includes a gRPC port and an http port;

[0008] The definition module is used to obtain the dao data access layer in the business project and define the gRPC interface service through ProtoBuf;

[0009] The sinking module is used to realize the logic sinking of the dao data access layer interface in the business project and send it to the data access agent device to complete the interaction and access with the database;

[0010] The deployment module is used to deploy the edge proxy component Envoy and access the data access proxy device through the edge proxy component Envoy;

[0011] The switching module is used to transform the logic of the dao data access layer into a gRPC client call mode, and switch the database access of each business process dao interface to a gRPC client accessing the gRPC interface mode;

[0012] The central control module is used to send and receive flow information between the access module, definition module, sinking module, deployment module and switching module.

[0013] In a preferred solution, the gRPC port is used to provide the gRPC interface service of the dao data access layer, and the http port is used to obtain gRPC interface monitoring information through the spring actuator.

[0014] In a preferred solution, the gRPC interface provides external access to the database and supports multiple programming languages.

[0015] In a preferred embodiment, an alarm module is further included, which is used to collect and summarize gRPC interface monitoring information in real time, wherein the gRPC interface monitoring information includes the name, method and type of input parameters and output parameters;

[0016] The alarm module is provided with a standard data set, which is used for real-time comparison with the gRPC interface monitoring information;

[0017] If the gRPC interface monitoring information does not belong to the standard data set, it indicates that the gRPC interface is abnormal;

[0018] If the gRPC interface monitoring information belongs to the standard data set, it indicates that the gRPC interface is normal.

[0019] In a preferred embodiment, the gRPC interface monitoring information further includes interface call information, wherein the interface call information includes allocated interfaces and unallocated interfaces;

[0020] After the allocated interface is determined, the response time of the allocated interface is obtained in real time, and the response time is marked as a parameter to be evaluated, and the parameter to be evaluated is summarized into a data set to be evaluated;

[0021] The parameters to be evaluated are input into an evaluation model to obtain the allowable response time of the unassigned interface.

[0022] In a preferred embodiment, the evaluation model is executed as follows:

[0023] Obtaining parameters to be evaluated from the data set to be evaluated;

[0024] Obtain the response time of the newly assigned interface and use it as the benchmark parameter;

[0025] An evaluation function is called from the evaluation model, and the parameters to be evaluated and the benchmark parameters are input into the evaluation function, and the output result is calibrated as the allowable response time.

[0026] In a preferred embodiment, after the allowable response time is output, the demand response time of the newly added business process is collected, and the demand response time is compared with the allowable response time;

[0027] If the demand response time is greater than or equal to the allowed response time, it indicates that the data access demand of the newly added business process can be executed;

[0028] If the demand response time is less than the allowed response time, it indicates that the data access of the newly added business process cannot be executed, and an alarm signal is generated simultaneously.

[0029] In a preferred solution, when all the gRPC interfaces are occupied, the execution priorities of the newly added business processes and the allocated business processes are obtained and compared in real time;

[0030] If the execution priority of the newly added business process is higher than the execution priority of the assigned business process, the execution priority of the assigned business process is stopped, and data access of the newly added business process is executed;

[0031] If the execution priority of the newly added business process is less than or equal to the execution priority of the assigned business process, the data access of the assigned business process continues to be executed, and the newly added business process is added to the waiting queue.

[0032] In a preferred solution, when the sinking module is executed, the code of the dao data access layer and the business service layer are decoupled and isolated, wherein the data access proxy device is a stateless design, used to implement the deployment of multiple proxy nodes, and can horizontally expand and shrink the proxy nodes according to business needs.

[0033] The present invention also provides a multi-language data access proxy terminal based on gRPC, comprising:

[0034] at least one processor;

[0035] and a memory communicatively coupled to the at least one processor;

[0036] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned multi-language data access proxy device based on gRPC.

[0037] The technical effects achieved by the present invention are:

[0038] The data proxy service in this invention is stateless and can provide horizontal scaling capabilities according to business needs. It only needs to implement a set of dao data access layer logic to provide efficient and secure database table access for business projects in different programming languages. In addition, the dao data access layer is completely decoupled from the business project code, and can even complete switching between different databases in the data access proxy layer without the business caller's awareness. At the same time, a spring actuator plug-in is introduced to observe the call status and time consumption of each gRPC interface in real time, providing a reference guide for dao interface performance optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a structural diagram of the device provided by the present invention. DETAILED DESCRIPTION

[0040] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0042] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it constitute a separate or selective embodiment that is mutually exclusive of other embodiments.

[0043] See also Figure 1 As shown, the present invention provides a multi-language data access proxy device based on gRPC, including an access module, a definition module, a sinking module, a deployment module, a switching module and a central control module;

[0044] The access module is used to access the data access proxy device through a communication interface, wherein the communication interface includes a gRPC port and an http port;

[0045] The definition module is used to obtain the dao data access layer in the business project and define the gRPC interface service through ProtoBuf;

[0046] The sinking module is used to realize the logic sinking of the dao data access layer interface in the business project and send it to the data access agent device to complete the interaction and access with the database;

[0047] The deployment module is used to deploy the edge proxy component Envoy and access the data access proxy device through the edge proxy component Envoy;

[0048] The switching module is used to transform the logic of the dao data access layer into a gRPC client call method, and switch the database access of each business process dao interface to accessing the gRPC interface through the gRPC client;

[0049] The central control module is used to send and receive flow information between the access module, definition module, sinking module, deployment module and switching module.

[0050] In the present invention, with the rapid development of Internet technology, the demand for data interaction between various application systems is increasing. In order to realize data access and communication between different languages, different platforms, and different systems, people have adopted a variety of technical means, such as RESTful API, SOAP, XML, etc. However, these technologies have certain limitations in practical applications, such as performance bottlenecks, insufficient cross-language support, etc. Therefore, research and development of an efficient, flexible, and scalable data access proxy device has become a hot issue in the current technical field. In this embodiment, the access module is first used to access the data access proxy device through the communication interface. For example, the data access proxy device can be written in Java language, and the gRPC interface service of the dao data access layer can be provided through the gRPC port. The gRPC port is used to provide the gRPC interface service of the dao data access layer, and the http port is used to access the data access proxy device through the spring The actuator is used to obtain gRPC interface monitoring information. The gRPC interface provides external database access and is compatible with multiple programming languages. The definition module then obtains the dao data access layer in the business project. The sinking module then sinks the dao data access layer to the data access proxy device, achieving code decoupling and isolation between the business service layer and the dao data access layer. The deployment module then deploys the edge proxy component, Envoy. Envoy is a L7 proxy and communication bus designed for large-scale modern SOA (service-oriented architecture) architectures. It is compact and high-performance, providing network functions such as routing, load balancing, health monitoring, exception retry, and failover. The switching mode is then used to transform the dao data access layer logic into a gRPC client call method. The specific implementation of database access for each business dao interface is switched to accessing the gRPC interface through the gRPC client. This can meet the access requirements of business systems in different programming languages ​​for the same database table. Business callers only need to encapsulate gRPC interface parameters and implement database access requirements through the gRPC client. This is convenient and fast, eliminating the need to implement multiple sets of dao data access logic on the business caller side.

[0051] In a preferred embodiment, an alarm module is further included, which is used to collect and summarize gRPC interface monitoring information in real time, wherein the gRPC interface monitoring information includes the name, method, and type of input parameters and output parameters;

[0052] The alarm module is equipped with a standard data set, which is used for real-time comparison with the gRPC interface monitoring information;

[0053] If the gRPC interface monitoring information does not belong to the standard data set, it indicates that the gRPC interface is abnormal;

[0054] If the gRPC interface monitoring information belongs to the standard data set, it indicates that the gRPC interface is normal.

[0055] In this embodiment, the system also includes an alarm module. When the alarm module is executed, it will count and summarize the monitoring information of the gRPC interface, and a standard data set is preset in the alarm module. The standard data set is used for real-time comparison with the gRPC interface monitoring information. When the gRPC interface monitoring information does not belong to the standard data set, it indicates that the gRPC interface is abnormal, and Binghui synchronously issues an alarm signal. Otherwise, it indicates that the gRPC interface is normal and data access operations can continue. In this way, when the interface is abnormal, the management personnel can be reminded to perform maintenance operations in time to avoid the occurrence of access data disorder.

[0056] In a preferred embodiment, the gRPC interface monitoring information also includes interface call information, wherein the interface call information includes allocated interfaces and unallocated interfaces;

[0057] After the allocated interface is determined, the response time of the allocated interface is obtained in real time and marked as a parameter to be evaluated, and the parameters to be evaluated are summarized into a data set to be evaluated;

[0058] The parameters to be evaluated are input into the evaluation model to obtain the allowable response time of the unassigned interface.

[0059] In this embodiment, after the gRPC interface monitoring information is output, its interface call information is classified into allocated interfaces and unallocated interfaces. After the allocated interface is determined, the response time of the allocated interface will be obtained in real time. This embodiment will mark it as a parameter to be evaluated, and then input the parameter to be evaluated into the evaluation model to obtain the allowable response time of the unallocated interface, wherein the effective response time of the data access agent device is preset in the evaluation model.

[0060] In a preferred embodiment, the evaluation model is executed as follows:

[0061] S1. Obtain the parameters to be evaluated from the dataset to be evaluated;

[0062] S2. Obtain the response time of the newly assigned interface and calibrate it as a benchmark parameter;

[0063] S3. Call the evaluation function from the evaluation model, input the parameters to be evaluated and the benchmark parameters into the evaluation function, and calibrate its output result as the allowable response time.

[0064] As described in steps S1-S3 above, when the evaluation model is executed, the parameters to be evaluated are first called from the data set to be evaluated, and then the evaluation function is called from the evaluation model, where the expression of the evaluation function is: Where, Tx Indicates the allowed response time, T z Indicates the effective response time, n indicates the number of benchmark parameters, f i and the baseline parameters representing adjacent responses, and then comparing the allowed response time with the required response time of the new business process to determine whether the new business process can be executed.

[0065] In a preferred embodiment, after the allowed response time is output, the demand response time of the newly added business process is collected, and the demand response time is compared with the allowed response time;

[0066] If the required response time is greater than or equal to the allowed response time, it indicates that the data access requirement of the new business process can be executed;

[0067] If the required response time is less than the allowed response time, it indicates that the data access of the newly added business process cannot be executed, and an alarm signal is generated simultaneously.

[0068] In this embodiment, after the allowed response time is output, the execution status of the allocated interface will be monitored in real time, and before the new business process is executed, the allowed response time will be added to the occupancy time of the allocated interface. Then, the demand response time of the new business process will be collected and compared with the allowed response time. When the demand response time is less than the allowed response time, an alarm signal will be issued. In this state, it indicates that the new business process cannot be executed. Accordingly, it will be added to the waiting queue to facilitate subsequent execution of data access for the new business process. Otherwise, it indicates that the data access of the new business process can be executed. At this time, the unallocated interface can be directly assigned to the new business process.

[0069] In a preferred embodiment, when all gRPC interfaces are occupied, the execution priorities of the newly added business processes and the assigned business processes are obtained and compared in real time;

[0070] If the execution priority of the newly added business process is higher than the execution priority of the assigned business process, the execution priority of the assigned business process will be stopped, and the data access of the newly added business process will be executed;

[0071] If the execution priority of the newly added business process is less than or equal to the execution priority of the assigned business process, the data access of the assigned business process will continue to be executed, and the newly added business process will be added to the waiting queue.

[0072] In this embodiment, during the data access process, there is a situation where all gRPC interfaces are occupied. At this time, when adding a new business process, it will be determined whether the new business process will be added to the waiting queue based on the execution priority of the new business process and the assigned business process. This method is also applicable to the situation where the allowed allocation time of the unassigned interface is insufficient, so it will not be repeated here.

[0073] In a preferred embodiment, when the sinking module is executed, the code of the dao data access layer and the business service layer are decoupled and isolated. The data access proxy device is stateless in design, used to implement the deployment of multiple proxy nodes, and can horizontally scale the proxy nodes according to business needs. Only one set of dao data access layer logic needs to be implemented, which can provide efficient and secure database table access for business projects in different programming languages. In addition, the dao data access layer is completely decoupled from the business project code, and can complete the switching of different databases in the data access proxy layer without the business caller's perception.

[0074] The present invention also provides a multi-language data access proxy terminal based on gRPC, comprising:

[0075] at least one processor;

[0076] and a memory communicatively coupled to the at least one processor;

[0077] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the above-mentioned multi-language data access proxy device based on gRPC.

[0078] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0079] The foregoing is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained herein shall, unless otherwise specified or limited, be implemented in accordance with conventional means in the art.

Claims

1. A multi-language data access proxy device based on gRPC, comprising an access module, a definition module, a sinking module, a deployment module, a switching module, and a central control module, characterized in that: The access module is used to access the data access proxy device through a communication interface, wherein the communication interface includes a gRPC port and an http port; The definition module is used to obtain the dao data access layer in the business project and define the gRPC interface service through ProtoBuf; The sinking module is used to realize the logic sinking of the dao data access layer interface in the business project and send it to the data access agent device to complete the interaction and access with the database; The deployment module is used to deploy the edge proxy component Envoy and access the data access proxy device through the edge proxy component Envoy; The switching module is used to transform the logic of the dao data access layer into a gRPC client call mode, and switch the database access of each business process dao interface to a gRPC client accessing the gRPC interface mode; The central control module is used to send and receive flow information between the access module, definition module, sinking module, deployment module and switching module.

2. A multi-language data access proxy device based on gRPC according to claim 1, characterized in that: The gRPC port is used to provide the gRPC interface service of the dao data access layer, and the http port is used to obtain gRPC interface monitoring information through springactuator.

3. A multi-language data access proxy device based on gRPC according to claim 1, characterized in that: The gRPC interface provides external access to the database and supports multiple programming languages.

4. A multi-language data access proxy device based on gRPC according to claim 1, characterized in that: It also includes an alarm module, which is used to count and summarize gRPC interface monitoring information in real time, wherein the gRPC interface monitoring information includes the name, method and type of input parameters and output parameters; The alarm module is provided with a standard data set, which is used for real-time comparison with the gRPC interface monitoring information; If the gRPC interface monitoring information does not belong to the standard data set, it indicates that the gRPC interface is abnormal; If the gRPC interface monitoring information belongs to the standard data set, it indicates that the gRPC interface is normal.

5. The multi-language data access proxy device based on gRPC according to claim 2, characterized in that: The gRPC interface monitoring information also includes interface call information, wherein the interface call information includes allocated interfaces and unallocated interfaces; After the allocated interface is determined, the response time of the allocated interface is obtained in real time, and the response time is marked as a parameter to be evaluated, and the parameter to be evaluated is summarized into a data set to be evaluated; The parameters to be evaluated are input into an evaluation model to obtain the allowable response time of the unassigned interface.

6. A multi-language data access proxy device based on gRPC according to claim 5, characterized in that: The execution steps of the evaluation model are as follows: Obtaining parameters to be evaluated from the data set to be evaluated; Obtain the response time of the newly assigned interface and use it as the benchmark parameter; An evaluation function is called from the evaluation model, and the parameters to be evaluated and the benchmark parameters are input into the evaluation function, and the output result is calibrated as the allowable response time.

7. The multi-language data access proxy device based on gRPC according to claim 5, characterized in that: After the allowed response time is output, the demand response time of the newly added business process is collected, and the demand response time is compared with the allowed response time; If the demand response time is greater than or equal to the allowed response time, it indicates that the data access demand of the newly added business process can be executed; If the demand response time is less than the allowed response time, it indicates that the data access of the newly added business process cannot be executed, and an alarm signal is generated simultaneously.

8. The multi-language data access proxy device based on gRPC according to claim 5, characterized in that: When all gRPC interfaces are occupied, the execution priorities of the newly added business processes and the assigned business processes are obtained and compared in real time; If the execution priority of the newly added business process is higher than the execution priority of the assigned business process, the execution priority of the assigned business process is stopped, and data access of the newly added business process is executed; If the execution priority of the newly added business process is less than or equal to the execution priority of the assigned business process, the data access of the assigned business process continues to be executed, and the newly added business process is added to the waiting queue.

9. The multi-language data access proxy device based on gRPC according to claim 1, characterized in that: When the sinking module is executed, the codes of the dao data access layer and the business service layer are decoupled and isolated. The data access proxy device is a stateless design, which is used to realize the deployment of multiple proxy nodes and can horizontally expand and shrink the proxy nodes according to business needs.

10. A multi-language data access proxy terminal based on gRPC, characterized by: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the multi-language data access proxy device based on gRPC as described in any one of claims 1 to 9.

Images