Remote procedure call device, distributed system and electronic device

By integrating gRPC, Netty and Hadoop RPC protocols in the remote procedure call device, providing unified interfaces and adapters, the problem of RPC protocol incompatibility is solved, and efficient distributed system communication and simplified system maintenance is achieved.

CN117834701BActive Publication Date: 2024-11-05TIANYUN RONGCHUANG DATA TECH BEIJING CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311864606.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-29
Publication Date
2024-11-05
Estimated Expiration
2043-12-29

AI Technical Summary

Technical Problem

The existing RPC protocols are incompatible, making it difficult to realize inter-service communication in distributed systems within one framework.

Method used

Provides a remote procedure call device, including a general interface, protocol adapter, dynamic proxy module and data processing module, supports the integration of gRPC, Netty and Hadoop RPC protocols, simplifies system maintenance and expansion through unified interfaces and adapters, utilizing HTTP/2 and Asynchronous event-driven technology improves communication efficiency.

Benefits of technology

It realizes seamless communication between different systems, improves communication efficiency and response speed, simplifies system maintenance and expansion, and reduces resource consumption.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117834701B_ABST
    Figure CN117834701B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses a remote procedure call device, a distributed system and an electronic device, which are applied to the field of remote communication technology and can solve the problem of how to integrate all RPC protocols in a framework to realize inter-service communication in a distributed system. The remote procedure call device includes: a universal interface for providing a unified call interface for each remote procedure call RPC protocol; a protocol adapter for providing a specific protocol call for each RPC protocol; a dynamic proxy module for dynamically selecting a target RPC protocol to implement the call of a remote service; a data processing module for processing data through serialization technology and deserialization technology; wherein, the RPC protocols include: gRPC protocol, Netty protocol and Hadoop RPC protocol.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of remote communication technology, and in particular to a remote procedure call device, a distributed system and an electronic device. Background Art

[0002] In modern distributed systems, Remote Procedure Call Protocol (RPC) frameworks play a vital role, allowing efficient communication between different services. When building large-scale distributed systems, different services and applications may also use different RPC protocols to achieve communication. gRPC, Netty, and Hadoop RPC are three popular RPC protocols, each with its own unique advantages and suitable for different scenarios and needs. Currently, these RPC protocols are incompatible with each other. Therefore, how to integrate all RPC protocols into a framework to achieve inter-service communication in distributed systems has become an urgent problem that needs to be solved. Summary of the invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present application provide a remote procedure call device, a distributed system and an electronic device to solve the problem of how to integrate all RPC protocols into a framework to realize inter-service communication in a distributed system.

[0004] In order to achieve the above purpose, the technical solutions provided by the embodiments of the present application are as follows:

[0005] In a first aspect, an embodiment of the present application provides a remote procedure call device, the remote procedure call device comprising: a universal interface, the universal interface is used to provide a unified call interface for each remote procedure call RPC protocol;

[0006] A protocol adapter, wherein the protocol adapter is used to provide a specific protocol call for each RPC protocol;

[0007] A dynamic proxy module, which is used to dynamically select a target RPC protocol to implement a remote service call;

[0008] A data processing module, wherein the data processing module is used to process data through serialization technology and deserialization technology;

[0009] The RPC protocols include: gRPC protocol, Netty protocol and Hadoop RPC protocol.

[0010] As an optional implementation manner, in the first aspect of the embodiment of the present application, the protocol adapter specifically includes:

[0011] gRPC adapter, used to handle cross-language data communication based on HTTP / 2 protocol;

[0012] Netty adapter, used to handle asynchronous event-driven data communication based on TCP / UDP;

[0013] Hadoop adapter that handles data communication within the Hadoop ecosystem.

[0014] As an optional implementation, in the first aspect of the embodiment of the present application, the dynamic proxy module is specifically used to receive a service call request sent by a client, and select the target RPC protocol according to the service call request.

[0015] As an optional implementation, in the first aspect of the embodiment of the present application, the data processing module is specifically used to determine the serialization and deserialization formats corresponding to the target RPC protocol according to the target RPC protocol, so as to encode and decode the data.

[0016] As an optional implementation manner, in the first aspect of the embodiment of the present application, the remote procedure call apparatus further includes:

[0017] A network communication unit, wherein the network communication unit is used to implement TCP communication, UDP communication and file transfer based on a network communication framework.

[0018] As an optional implementation manner, in the first aspect of the embodiment of the present application, the remote procedure call apparatus further includes:

[0019] The security unit is used to implement data encryption and identity authentication.

[0020] As an optional implementation manner, in the first aspect of the embodiment of the present application, the remote procedure call apparatus further includes:

[0021] An error handling and monitoring unit is used to provide an error handling mechanism, as well as monitoring and logging functions.

[0022] As an optional implementation manner, in the first aspect of the embodiment of the present application, the remote procedure call apparatus further includes:

[0023] The service discovery and load balancing unit is used to integrate the service discovery and load balancing mechanisms so as to allocate corresponding service instances to the client's requests.

[0024] In a second aspect, an embodiment of the present application provides a distributed system, which includes: the remote procedure call device in the first aspect of the embodiment of the present application.

[0025] In a third aspect, an embodiment of the present application provides an electronic device, wherein the electronic device is configured with the distributed system in the second aspect of the embodiment of the present application.

[0026] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0027] The embodiment of the present application provides a remote procedure call device, a distributed system and an electronic device, wherein the remote procedure call device includes: a general interface for providing a unified call interface for each remote procedure call RPC protocol; a protocol adapter for providing a specific protocol call for each RPC protocol; a dynamic proxy module for dynamically selecting a target RPC protocol to implement the call of a remote service; a data processing module for processing data through serialization technology and deserialization technology; wherein the RPC protocol includes: gRPC protocol, Netty protocol and Hadoop RPC protocol. Through this device, the gRPC protocol, Netty protocol and Hadoop RPC protocol can be integrated into a framework, supporting multiple programming languages ​​and platforms, making the communication between different systems more flexible and seamless, and utilizing the HTTP / 2 feature of gRPC and the high-performance network processing capability of Netty to improve the overall communication efficiency. In addition, since a unified interface and adapter are provided, the maintenance and future expansion of the system are simplified, and through asynchronous and event-driven methods, resource consumption is reduced, and the response speed and throughput of the system are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 This is a schematic diagram of the structure of a remote procedure call device provided in an embodiment of the present application. Figure 1 ;

[0031] Figure 2 It is a general interface diagram of a remote procedure call device provided in an embodiment of the present application;

[0032] Figure 3 This is a schematic diagram of the structure of a remote procedure call device provided in an embodiment of the present application. Figure 2 ;

[0033] Figure 4 It is a dynamic proxy schematic diagram of a remote procedure call device provided in an embodiment of the present application;

[0034] Figure 5 This is a schematic diagram of the structure of a remote procedure call device provided in an embodiment of the present application. Figure 3 ;

[0035] Figure 6 It is a structural diagram of a distributed system provided in an embodiment of the present application;

[0036] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] The terms “first”, “second” and the like in the description and claims of the present application are used to distinguish different objects rather than to describe a specific order of the objects.

[0039] The terms "including" and "having" and any variations thereof in the embodiments of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0040] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0041] In modern distributed systems, remote procedure call (RPC) frameworks play a vital role, allowing efficient communication between different services. When building large-scale distributed systems, different services and applications may adopt different RPC protocols to achieve communication. gRPC, Netty, and Hadoop RPC are three widely used RPC protocols, each with its own unique advantages and suitable for different scenarios and needs. gRPC is a high-performance, cross-language RPC framework developed by Google. It is based on the HTTP / 2 protocol and supports multiple programming languages, making communication between services more efficient and reliable. Netty is an asynchronous event-driven network application framework that provides support for TCP, UDP, and file transfer, and is used as the underlying communication technology by many high-performance RPC frameworks. Hadoop RPC is designed specifically for the Hadoop ecosystem, allowing efficient communication between Hadoop components.

[0042] When designing a framework that supports gRPC, Netty, and Hadoop RPC protocols at the same time, the technical content involves several key aspects. First, gRPC is a modern, open source, high-performance RPC framework that can run in any environment, effectively connect services within and across data centers, and provide pluggable support for load balancing, call chain tracing, health checks, and authentication. Netty is a high-performance, asynchronous event-driven NIO framework that provides support for TCP, UDP, and file transfer. As an asynchronous NIO framework, all IO operations of Netty are asynchronous and non-blocking. Through the Future-Listener mechanism, users can easily actively obtain or obtain IO operation results through the notification mechanism.

[0043] In order to solve some or all of the above technical problems, the embodiment of the present application provides a remote procedure call device, a distributed system and an electronic device, wherein the remote procedure call device includes: a general interface for providing a unified call interface for each remote procedure call RPC protocol; a protocol adapter for providing a specific protocol call for each RPC protocol; a dynamic proxy module for dynamically selecting a target RPC protocol to implement the call of a remote service; a data processing module for processing data through serialization technology and deserialization technology; wherein the RPC protocol includes: gRPC protocol, Netty protocol and Hadoop RPC protocol. Through this device, the gRPC protocol, Netty protocol and Hadoop RPC protocol can be integrated into a framework, supporting multiple programming languages ​​and platforms, making the communication between different systems more flexible and seamless, and utilizing the HTTP / 2 features of gRPC and the high-performance network processing capabilities of Netty to improve the overall communication efficiency. In addition, due to the provision of a unified interface and adapter, the maintenance and future expansion of the system are simplified, and through asynchronous and event-driven methods, resource consumption is reduced, and the response speed and throughput of the system are improved.

[0044] like Figure 1 As shown, an embodiment of the present application provides a remote procedure call device, which may specifically include:

[0045] The general interface 11 is used to provide a unified calling interface for each remote procedure call (RPC) protocol.

[0046] The protocol adapter 12 is used to provide a specific protocol call for each RPC protocol.

[0047] The dynamic proxy module 13 is used to dynamically select the target RPC protocol to implement the call of the remote service.

[0048] The data processing module 14 is used to process data through serialization technology and deserialization technology.

[0049] Among them, the RPC protocols include: gRPC protocol, Netty protocol and Hadoop RPC protocol; the target RPC protocol is any one of the gRPC protocol, Netty protocol and Hadoop RPC protocol.

[0050] In some embodiments, the remote procedure call device needs to define a universal RPC interface, which will provide a unified calling method for all three protocols. This interface needs to be abstract enough to adapt to the characteristics of different protocols. The RPC interface is a universal interface 11, which can abstract the details of different RPC protocols so that the details of different protocols are transparent to the end user, which means that the user can seamlessly switch protocols without affecting the use of the service.

[0051] In the RPC protocol framework, a common interface is a common concept. A common interface defines a set of common methods that can be implemented by different services. By using a common interface, clients can call methods of different service providers without having to worry about the specific service implementation details.

[0052] A generic interface usually defines a set of standard method signatures, such as the type of input parameters, the type of return value, etc. Service providers need to implement these methods and perform serialization and communication in the agreed format. Clients interact with different service providers by calling these methods.

[0053] The advantage of a common interface is that it can improve the maintainability and scalability of the system. By defining a set of standard method signatures, different service providers can be easily replaced without modifying the client code. In addition, a common interface can also easily add new methods to meet new business needs.

[0054] By using common interfaces, services can be abstracted, making the client and server more decoupled and improving the flexibility and scalability of the system.

[0055] like Figure 2 As shown, the general interface can implement asynchronous requests, synchronous requests and exception handling.

[0056] In some embodiments, the general interface is one of the key components for implementing asynchronous requests. Asynchronous requests allow the client to obtain a response at a later time point without having to wait for the response from the server after initiating a request, which can improve the concurrency performance and response speed of the system. The general interface is an interface specification shared by the client and the server, which is used to define callable remote procedures. The general interface may include methods and corresponding parameters for asynchronous requests. The client sends an asynchronous request to the server by calling the method defined in the general interface without having to wait for the response from the server. After receiving the asynchronous request, the server can put it in a queue. The server can use multi-threading or asynchronous processing to process the requests in the queue. After the server processes the asynchronous request, it can return the response to the client through a callback function or event notification, and the client can obtain the response when needed.

[0057] Specifically, asynchronous requests can be made through CompletableFuture <clientreply>sendRequestAsync(ClientRequest request) function implementation, CompletableFuture <clientreply>is the return type of the function. CompletableFuture is a Java class used to represent the result of an asynchronous computation that may be executed in another thread. ClientReply is the type of the result of this asynchronous computation. sendRequestAsync is the name of the function. ClientRequest request is the parameter list of the function. This function receives a parameter named request of type ClientRequest.

[0058] In some embodiments, the general interface is also one of the key components for implementing synchronous requests. Synchronous requests require that the client must wait for the response of the server after initiating the request before continuing to perform subsequent operations. Similar to asynchronous requests, the general interface is an interface specification shared by the client and the server, which is used to define callable remote procedures. The general interface may include methods and corresponding parameters for synchronous requests. The client sends a synchronous request to the server by calling the method defined in the general interface, and blocks and waits for the response of the server. After receiving the synchronous request, the server processes it and returns a response to the client. The server should complete the processing and return the response as soon as possible to reduce the waiting time of the client. The server returns the processing result to the client through the general interface. After receiving the response, the client continues to perform subsequent operations.

[0059] It should be noted that when implementing synchronous requests, in order to avoid performance problems caused by long client waits, a reasonable timeout should be set. When the timeout is exceeded, the client can choose to retry or abandon the request. Both the server and the client should perform exception handling to ensure that appropriate processing can be performed when an exception occurs, such as logging, returning error codes, etc. When there are multiple service instances providing the same service, a load balancing algorithm should be used to distribute requests to different service instances to improve the scalability and reliability of the system. When transmitting data between the client and the server, serialization and deserialization operations are required. Choosing an appropriate serialization method can improve the efficiency and compatibility of data transmission.

[0060] Specifically, the synchronous request can be implemented through the ClientReply sendRequest (ClientRequest request) function, where ClientReply is the return type of the function, indicating that the function will return a ClientReply object, sendRequest is the name of the function, and ClientRequest request is the parameter list of the function. The function receives a parameter of type ClientRequest named request.

[0061] In some embodiments, exception handling of the general interface is an important part of ensuring system stability and reliability. Exception handling involves both the client and the server, and the purpose is to be able to perform appropriate processing when an exception occurs to avoid system crash or data loss. In the general interface, a set of exception classes can be defined to represent possible error conditions, and these exception classes should inherit from the general exception base class for unified processing. When the server encounters an exception when processing a request, it should throw a corresponding exception class, which can contain detailed information about the error, such as error code, error message, and stack trace. When calling the general interface, the client should capture the exception returned from the server. The client can determine whether an error has occurred by checking the return value or exception object, and perform corresponding processing according to the specific situation. Both the client and the server should implement appropriate exception handling logic. On the client, you can log, notify the user, or perform other recovery operations after catching the exception. On the server, you can log, roll back transactions, or perform other cleanup operations. In addition to throwing exceptions, the server can also pass error information to the client by returning error codes and error messages. The client can perform corresponding error processing based on the error code and message.

[0062] In some embodiments, the remote procedure call device can implement the configuration of an adapter for each RPC protocol, and these adapters are responsible for converting calls of the general interface into calls of the specific protocol.

[0063] In some embodiments, Figure 3 As shown, the protocol adapter 12 specifically includes:

[0064] The gRPC adapter 121 is used to process cross-language data communication based on the HTTP / 2 protocol.

[0065] The Netty adapter 122 is used to process asynchronous event-driven data communications based on TCP / UDP.

[0066] The Hadoop adapter 123 is used to process data communication in the Hadoop ecosystem.

[0067] It should be noted that since the RPC protocol can generally include the above three protocols, namely the gRPC protocol, the Netty protocol and the Hadoop RPC protocol, the protocol adapter can also be configured separately for the three RPC protocols, that is, the protocol adapter includes: a gRPC adapter corresponding to the gRPC protocol, a Netty adapter corresponding to the Netty protocol and a Hadoop adapter corresponding to the Hadoop RPC protocol.

[0068] In some embodiments, a protocol adapter is responsible for converting calls of a generic interface into calls of a specific protocol.

[0069] Specifically, the gRPC adapter is used to convert calls to the general interface into calls to the gRPC protocol, the Netty adapter is used to convert calls to the general interface into calls to the Netty protocol, and the Hadoop adapter is used to convert calls to the general interface into calls to the Hadoop RPC protocol.

[0070] In some embodiments, the dynamic proxy module is specifically used to receive a service call request sent by a client, and select a target RPC protocol according to the service call request.

[0071] The target RPC protocol is any one of the gRPC protocol, the Netty protocol, and the Hadoop RPC protocol.

[0072] It should be noted that the dynamic proxy module can be used to implement a dynamic proxy mechanism, allowing the client to call a remote service just like calling a local service. The dynamic proxy module can dynamically select an appropriate RPC protocol based on the configuration or request content.

[0073] Furthermore, since the protocol adapter has configured a corresponding protocol adapter for each RPC protocol, the dynamic proxy module will also select the protocol adapter corresponding to the target RPC protocol after selecting the appropriate target RPC protocol.

[0074] In some embodiments, in the RPC protocol framework, dynamic proxy is a commonly used technology for implementing communication between the client and the server, such as Figure 4 As shown in the figure, the main function of the dynamic proxy is to provide an intermediate layer to forward the client's request to the server and return the server's response to the client.

[0075] The implementation of a dynamic proxy usually involves the following steps:

[0076] Interface definition: First, you need to define the interfaces that both the server and the client follow. These interfaces define the remote procedures that the client can call, as well as the parameters and return values ​​required by these procedures.

[0077] Proxy generation: On the client side, the dynamic proxy framework automatically generates a proxy class based on the interface definition. This proxy class contains methods for communicating with the server, which forward requests to the server when called.

[0078] Request forwarding: When the client calls the method of the proxy class, the proxy class will package the request into the format specified by the protocol and send it to the server through the network. This process usually involves serialization and network communication.

[0079] Response return: After receiving the request, the server will perform the corresponding operation and package the result into the format specified by the protocol and return it to the client. The client's proxy class will receive this response and parse it into a usable form and return it to the client.

[0080] Through dynamic proxy, the client can call the remote procedure just like calling the local method without worrying about the underlying communication details. This greatly simplifies the implementation of remote calls and makes it easier to decouple the client and server.

[0081] In some embodiments, the data processing module is specifically used to determine the serialization and deserialization formats corresponding to the target RPC protocol according to the target RPC protocol, so as to encode and decode the data.

[0082] In some embodiments, serialization and deserialization is an important technology used to convert data into a format that can be transmitted over a network, or to convert received data back into its original format.

[0083] Serialization is the process of converting a data structure or object into a binary string. In the RPC protocol framework, serialization is the process of converting the state information of an object into a form that can be stored or transmitted, such as converting the state information of an object into a byte stream or XML format. This process usually involves extracting the field values ​​of the object and organizing them in a certain format so that they can be sent over the network or stored on disk.

[0084] Deserialization is the opposite process, which is the process of converting a binary string back into a data structure or object. In the RPC protocol framework, deserialization is the process of converting the received byte stream or XML format data back into the form of an object. This process requires reorganizing the received data into a structured form according to the original format specification for use by the application.

[0085] The purpose of serialization and deserialization is to achieve cross-platform and cross-language data sharing. By converting data into a unified format, different applications and systems can easily exchange and communicate data. At the same time, since data can be encrypted and verified during transmission, serialization and deserialization also help improve data security and reliability.

[0086] The various RPC protocol frameworks mentioned above can support multiple serialization formats, such as Protocol Buffers, JSON, XML, etc., so that the appropriate protocol can be selected for data transmission and storage according to actual needs.

[0087] In some embodiments, Figure 5 As shown, the remote procedure call device may further include: a network communication unit 15, which is used to implement TCP communication, UDP communication and file transmission based on a network communication framework.

[0088] It should be noted that the network communication unit can use Netty as the underlying network communication framework because it provides support for TCP, UDP and file transfer and is asynchronous event-driven, which helps to improve the overall performance and scalability.

[0089] In network communication, the RPC protocol framework mainly involves communication between the client and the server. The core principle of the RPC protocol framework is that the client and the server perform serialized network communication according to the agreed communication protocol. In network communication, the client and the server need to establish a reliable transmission link to ensure reliable data transmission. Common transport layer protocols include TCP and UDP. TCP is a connection-oriented protocol that can ensure orderly and reliable data transmission, but may sacrifice some transmission efficiency. UDP is a connectionless protocol with high transmission efficiency, but it cannot guarantee reliable data transmission. Therefore, when choosing a transport layer protocol, you need to weigh it according to actual needs.

[0090] In network communication, you also need to consider the choice of network IO model. Common network IO models include blocking IO, non-blocking IO, IO multiplexing, asynchronous IO, etc. Different IO models have different characteristics and application scenarios, and you need to choose according to actual needs. For example, for applications that need to handle a large number of concurrent connections, you can choose to use IO multiplexing or asynchronous IO models to improve the system's concurrent processing capabilities.

[0091] In addition, the security and reliability of data also need to be considered in network communications. To ensure data security, encryption algorithms can be used to encrypt data to prevent data theft or tampering. To ensure data reliability, verification mechanisms can be used to verify data to ensure data integrity and consistency.

[0092] In short, network communication in the RPC protocol framework involves many aspects, including the selection of transport layer protocols, the selection of network IO models, and the consideration of data security and reliability. In practical applications, it is necessary to select and configure according to specific needs to achieve the best network communication effect.

[0093] In some embodiments, Figure 5 As shown, the remote procedure call device may further include: a security unit 16, which is used to implement data encryption and identity authentication.

[0094] It should be noted that the security unit can implement security mechanisms, including SSL / TLS encryption, authentication, and authorization, to ensure the security of RPC communications.

[0095] In some embodiments, security in the RPC protocol framework involves multiple aspects, such as data privacy, authentication, authorization, and transmission security.

[0096] Data privacy: Ensure that sensitive data is not illegally obtained during RPC communication. Protect data privacy through encryption and hashing technologies.

[0097] Authentication: By verifying the identity of the client and the server, it ensures that only legitimate clients can access the service. Common authentication methods include using usernames and passwords, digital signatures, or using security tokens.

[0098] Authorization: Access control is performed on clients to ensure that they can only access services that they are authorized to use. This can be achieved by using mechanisms such as access control lists (ACLs) or role-based access control (RBAC).

[0099] Transmission security: Ensure the communication security between the client and the server. By using encryption technologies such as SSL / TLS, the data in the communication process is protected from being tampered with or stolen.

[0100] In addition, some other security issues need to be considered, such as preventing denial of service attacks (DoS attacks) and preventing malicious code injection.

[0101] In order to improve the security of the RPC framework, a series of security measures can be taken, such as using strong password policies, regularly updating and reinforcing security components, and implementing security audits and monitoring. At the same time, it is necessary to constantly pay attention to the latest security threats and vulnerabilities, and update and upgrade the RPC framework in a timely manner to ensure the security of the system.

[0102] In some embodiments, Figure 5 As shown, the remote procedure call apparatus may further include: an error handling and monitoring unit 17, which is used to provide an error handling mechanism, as well as monitoring and logging functions.

[0103] It should be noted that the error handling and monitoring unit can provide an error handling mechanism, as well as monitoring and logging functions, to facilitate problem diagnosis and performance optimization.

[0104] In the RPC protocol framework, error handling and monitoring are important components that help improve the reliability and maintainability of the system.

[0105] Error handling: The RPC protocol framework should provide an appropriate error handling mechanism so that errors can be properly handled and recovered when they occur. Common error handling methods include exception handling, retry, fault tolerance, and transaction management. Exception handling refers to avoiding program crashes or data inconsistencies by capturing and handling exceptions. Retry refers to retrying to call a remote procedure when an error occurs to reduce the impact of network jitter or temporary errors. Fault tolerance refers to improving system availability and reliability by designing redundant or backup systems. Transaction management refers to ensuring data consistency and integrity by ensuring the atomicity, consistency, isolation, and persistence of a series of operations.

[0106] Monitoring: In order to promptly discover and solve problems in the system, the RPC protocol framework should provide monitoring functions. Monitoring can help administrators and developers understand the system's operating status, performance bottlenecks, and potential errors. Common monitoring indicators include: request success rate, response time, throughput, error rate, and resource utilization. The RPC protocol framework should support real-time monitoring and historical data query so that administrators can quickly locate and solve problems. In addition, monitoring data can also be used to analyze and optimize system performance to improve system stability and reliability.

[0107] In summary, error handling and monitoring in the RPC protocol framework are crucial to ensuring the reliability and stability of the system. Through appropriate error handling mechanisms and monitoring functions, errors and problems in the system can be effectively reduced, and the availability and performance of the system can be improved.

[0108] In some embodiments, Figure 5 As shown, the remote procedure call apparatus may further include: a service discovery and load balancing unit 18, which is used to integrate service discovery and load balancing mechanisms so as to allocate corresponding service instances to client requests.

[0109] It should be noted that the service discovery and load balancing unit can integrate the service discovery and load balancing mechanisms to ensure that requests can be assigned to appropriate service instances, which may involve integration with an existing service registry.

[0110] In the RPC protocol framework, service discovery and load balancing are two key parts that help improve the availability and performance of the system.

[0111] Among them, service discovery is a mechanism that allows clients to dynamically find and connect to available service instances at runtime. In a microservice architecture, service instances may be started and stopped dynamically, so clients need a way to discover currently available service instances.

[0112] Service discovery is usually implemented through a service registry, which maintains the network location information of currently available services. When a service instance starts, it registers its network location information with the service registry. The client obtains the network location information of available services by querying the service registry and establishes a connection with the service instance.

[0113] In addition, load balancing is a technology used to distribute client requests to multiple service instances to achieve efficient resource utilization and improve system performance. In the RPC protocol framework, load balancing can help distribute requests, avoid overloading a single service instance, and ensure that all service instances can be reasonably utilized.

[0114] Common load balancing strategies include Round Robin, Random, Least Connections, etc. These strategies can be selected according to different scenarios and requirements. For example, the Round Robin strategy distributes requests to each service instance in sequence, while the Random strategy randomly selects a service instance to handle the request. The Least Connections strategy sends the request to the service instance with the least number of current connections.

[0115] To achieve load balancing, the RPC protocol framework usually introduces a load balancer component on the client or server side. This component is responsible for selecting the appropriate service instance to process the client's request based on the configured load balancing strategy.

[0116] The combination of service discovery and load balancing can further improve the scalability and availability of the system. Through service discovery, the client can dynamically perceive the available service instances and distribute the requests to these instances through load balancing, thereby achieving balanced utilization of resources and high-performance operation of the system.

[0117] The embodiment of the present application provides a remote procedure call device, which includes: a general interface for providing a unified call interface for each remote procedure call RPC protocol; a protocol adapter for providing a specific protocol call for each RPC protocol; a dynamic proxy module for dynamically selecting a target RPC protocol to implement the call of a remote service; a data processing module for processing data through serialization technology and deserialization technology; wherein the RPC protocol includes: gRPC protocol, Netty protocol and Hadoop RPC protocol. Through this device, the gRPC protocol, Netty protocol and Hadoop RPC protocol can be integrated into a framework, supporting multiple programming languages ​​and platforms, making the communication between different systems more flexible and seamless, and utilizing the HTTP / 2 feature of gRPC and the high-performance network processing capability of Netty to improve the overall communication efficiency. In addition, since a unified interface and adapter are provided, the maintenance and future expansion of the system are simplified, and through asynchronous and event-driven methods, resource consumption is reduced, and the response speed and throughput of the system are improved.

[0118] like Figure 6 As shown, an embodiment of the present application provides a distributed system, which may include: Figure 1 The remote procedure call device shown, or Figure 3 The remote procedure call device shown, or Figure 5 The remote procedure call device shown.

[0119] like Figure 7 As shown, an embodiment of the present application further provides an electronic device, which may include: Figure 6 The distributed system shown.

[0120] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media that include computer-usable program code.

[0121] In several embodiments provided by the present application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] In the present application, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0123] In this application, memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0124] In this application, a person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the computer-readable medium includes permanent and non-permanent, removable and non-removable storage media. The storage medium can implement information storage by any method or technology, and the information can be a computer-readable instruction, a data structure, a module of a program, or other data. Examples of computer storage media include, but are not limited to, Parallel Random Access Memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Programmable Read-only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), other types of Random Access Memory (RAM), Read-Only Memory (ROM), One-time Programmable Read-Only Memory (OTPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0125] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0126] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for this application. The above-mentioned multiple embodiments are not necessarily multiple independent embodiments. Dividing into multiple embodiments is only used to highlight the different technical features in different embodiments. Those skilled in the art should be aware that the above-mentioned multiple embodiments can also be combined in any way.

[0127] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0128] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0130] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above-mentioned methods of various embodiments of the present application.

[0131] The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.< / clientreply> < / clientreply>

Claims

1. A remote procedure call device, characterized in that: The remote procedure call device comprises: A universal interface, wherein the universal interface is used to provide a unified calling interface for each remote procedure call (RPC) protocol; A protocol adapter, wherein the protocol adapter is used to provide a specific protocol call for each RPC protocol; A dynamic proxy module, which is used to dynamically select a target RPC protocol to implement a remote service call; A data processing module, wherein the data processing module is used to process data through serialization technology and deserialization technology; Among them, the RPC protocols include: gRPC protocol, Netty protocol and Hadoop RPC protocol; the protocol adapters include: gRPC adapter for processing cross-language data communication based on HTTP / 2 protocol, and Netty adapter for processing asynchronous event-driven data communication based on TCP / UDP, and Hadoop adapter for processing data communication in Hadoop ecosystem.

2. The remote procedure call device according to claim 1, characterized in that: The dynamic proxy module is specifically used to receive a service call request sent by a client, and select the target RPC protocol according to the service call request.

3. The remote procedure call device according to claim 1, characterized in that: The data processing module is specifically used to determine the serialization and deserialization formats corresponding to the target RPC protocol according to the target RPC protocol, so as to encode and decode the data.

4. The remote procedure call device according to claim 1, characterized in that: The remote procedure call device also includes: A network communication unit, wherein the network communication unit is used to implement TCP communication, UDP communication and file transfer based on a network communication framework.

5. The remote procedure call device according to claim 1, characterized in that: The remote procedure call device also includes: The security unit is used to implement data encryption and identity authentication.

6. The remote procedure call device according to claim 1, characterized in that: The remote procedure call device also includes: An error handling and monitoring unit is used to provide an error handling mechanism, as well as monitoring and logging functions.

7. The remote procedure call device according to claim 1, characterized in that: The remote procedure call device also includes: The service discovery and load balancing unit is used to integrate the service discovery and load balancing mechanisms so as to allocate corresponding service instances to the client's requests.

8. A distributed system, characterized in that: The distributed system comprises: a remote procedure call device as described in any one of claims 1 to 7.

9. An electronic device, characterized in that: The electronic device is provided with the distributed system as claimed in claim 8.