C + +-based data processing micro-service framework

Abstract

The invention belongs to the technical field of C + + application architecture, and discloses a data processing micro-service framework based on C + +, which is characterized in that a service control center interacts with a communication layer to obtain a communication state of the communication layer; judging the communication efficiency of a communication layer based on the obtained communication state, and adjusting the communication layer; the service control center performs information interaction with the business layer through the communication layer, and obtains state information of business service in real time; sending an adjustment instruction to a service layer based on the acquired state information; the service control center interacts with a bottom layer OS through a communication layer to obtain the real-time use condition of system resources, and then the number and configuration of service instances are dynamically adjusted. And the communication layer is connected with the underlying OS and is used for processing system commands transmitted and issued by the service control center and the business layer. According to the invention, network resources can be effectively utilized, and unnecessary communication overhead is avoided, so that the communication efficiency of the whole system is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of C++ application architecture and relates to a data processing microservice framework based on C++. Background Art

[0002] With the continuous development of science and technology, the C++ programming language has been welcomed by more and more technical practitioners due to its high compatibility. However, since the current mainstream interaction mode is the development mode of front-end and back-end separation, the existing C++ technology cannot provide friendly support, and there is no ready-made framework technology for C++, let alone a microservice-type C++ framework. Therefore, due to technical and framework issues, most technicians will choose to bypass C++ in applications. If it involves the underlying business docking with hardware, etc., it cannot improve the stability and efficiency of the underlying system, and it cannot meet the current localization requirements, nor can it meet the problem of cross-platform deployment. Summary of the invention

[0003] The purpose of the present invention is to solve the problem that C++ in the prior art cannot improve the stability and efficiency of the underlying system and cannot meet the cross-platform deployment requirements, and to provide a data processing microservice framework based on C++.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] A data processing microservice framework based on C++, characterized by including: a service control center, a business layer, a communication layer and an underlying OS;

[0006] The service control center interacts with the communication layer to obtain the communication status of the communication layer; based on the obtained communication status, the communication efficiency of the communication layer is determined, and the communication layer is adjusted;

[0007] The service control center exchanges information with the business layer through the communication layer to obtain the status information of the business service in real time; based on the obtained status information, it sends adjustment instructions to the business layer;

[0008] The service control center interacts with the underlying OS through the communication layer to obtain the real-time usage of system resources and dynamically adjust the number and configuration of service instances;

[0009] The communication layer is connected to the underlying OS to process system commands transmitted and issued by the service control center and the business layer.

[0010] A further improvement of the present invention is:

[0011] Furthermore, the service control center includes a registration service module, an instruction distribution module and a service management module;

[0012] The registration service module ensures that all business services can be registered in the service control center before or when they are started, and each service registered in the service control center obtains a unique identity tag; through the registration service module, the service control center can track all available services in the system, thereby supporting communication and calling between services;

[0013] The instruction distribution module matches the data type required by the business service with the microservice node in the business layer, and broadcasts or distributes the data to the business layer accordingly when the data is published;

[0014] The service management module is responsible for monitoring and managing the network location and operation status of the business layer. If any abnormality is found in the service, the instruction distribution module sends an adjustment instruction to the business layer.

[0015] Furthermore, the business layer includes multiple business microservices, and the business microservices include microservice nodes; the microservice nodes match the data types required by the business services and distribute the instructions that need to be adjusted to the corresponding business microservices.

[0016] Furthermore, business microservices are bidirectionally connected to each other, and two communication links are used for upstream and downstream data; each business service independently defines upstream instructions and downstream data; the microservices of the business layer include data collection services and data processing services; the data collection service is responsible for the access of peripheral data of the entire system; the data processing service registers the data types it needs with one or more data collection services through the registration service module according to the processed business; it can also register the data types it needs with other data processing services through the registration service module to form multi-level data processing; the processed data is broadcast to other required services; if the data needs to be broadcast or distributed to the client, it is forwarded to the client through the data interface.

[0017] Furthermore, the service control center interacts with the communication layer to obtain the communication status of the communication layer; based on the obtained communication status, the communication efficiency of the communication layer is judged, and the communication layer is adjusted, specifically: the service control center and the communication layer obtain the real-time communication status of the communication layer through an interface or a communication protocol; the service control center dynamically adjusts the network topology of the communication layer or increases the bandwidth according to the channel utilization, data transmission rate, signal-to-noise ratio, bit error rate, throughput, system capacity and system reliability to improve the channel utilization and the overall performance of the system.

[0018] Furthermore, the service control center exchanges information with the business layer through the communication layer to obtain status information of the business service in real time; based on the obtained status information, the adjustment instructions sent to the business layer are specifically: after the business layer receives the status query request sent by the service control center, it generates status information according to its own operating status and returns it to the service control center through the communication layer; the service control center identifies the received status information and obtains potential abnormal behavior; after receiving the adjustment instruction, the business layer makes corresponding adjustments according to the specific content of the instruction, including: modifying the configuration file, reallocating CPU or memory resources and adjusting the business logic.

[0019] Furthermore, the service control center interacts with the underlying OS through the communication layer to obtain real-time usage of system resources, and then dynamically adjusts the number and configuration of service instances. Specifically, the service control center sends a query request to the underlying OS through the communication layer to obtain real-time usage of CPU usage, memory occupancy, disk I / O, and network bandwidth, and based on resource usage, determines whether it is necessary to dynamically adjust the number and configuration of service instances; if necessary, after receiving the adjustment instruction, the underlying OS starts or stops the service instance and adjusts the configuration parameters of the instance to optimize the underlying OS.

[0020] Furthermore, the communication layer includes an API interface, a communication mode and a communication protocol; the communication layer is connected to the service control center, the business layer and the underlying OS through the API interface; the communication layer also exchanges information with external clients through the API interface; the communication layer interacts with the service control center, the business layer and the underlying OS through one or more of the message mechanism of the simulated message queue, the synchronous communication mode, the asynchronous communication mode and the semi-synchronous communication mode; the communication protocol used is the RPC protocol based on the TCP and Http communication protocol.

[0021] Furthermore, the underlying OS includes Windows, Linux and Kylin systems.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] The present invention monitors the communication status of the communication layer in real time through the service control center, and adjusts the communication layer according to the judgment result of the communication efficiency, which can effectively utilize network resources and avoid unnecessary communication overhead, thereby improving the communication efficiency of the overall system. The present invention realizes comprehensive automated management from the communication layer to the business layer and then to the underlying operating system, reduces the need for manual intervention, reduces operation and maintenance costs, and improves the accuracy and efficiency of management. The present invention improves the synergy of C++ development, thereby improving development efficiency and reducing later debugging costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 This is a topology diagram of the C++-based data processing microservice framework of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0027] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0029] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0030] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0031] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0032] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0033] See also Figure 1 , the present invention discloses a C++-based data processing microservice framework, including: a service control center, a business layer, a communication layer and an underlying OS;

[0034] The service control center interacts with the communication layer to obtain the communication status of the communication layer; based on the obtained communication status, the communication efficiency of the communication layer is determined, and the communication layer is adjusted;

[0035] The service control center exchanges information with the business layer through the communication layer to obtain the status information of the business service in real time; based on the obtained status information, it sends adjustment instructions to the business layer;

[0036] The service control center interacts with the underlying OS through the communication layer to obtain the real-time usage of system resources and dynamically adjust the number and configuration of service instances;

[0037] The communication layer is connected to the underlying OS to process system commands transmitted and issued by the service control center and the business layer.

[0038] The service control center includes a registration service module, an instruction distribution module, and a service management module;

[0039] The registration service module ensures that all business services can be registered in the service control center before or when they are started, and each service registered in the service control center obtains a unique identity tag; through the registration service module, the service control center can track all available services in the system, thereby supporting communication and calling between services;

[0040] The instruction distribution module matches the data type required by the business service with the microservice node in the business layer, and broadcasts or distributes the data to the business layer accordingly when the data is published;

[0041] The service management module is responsible for monitoring and managing the network location and operation status of the business layer. If any abnormality is found in the service, the instruction distribution module sends an adjustment instruction to the business layer.

[0042] The business layer includes multiple business microservices, each of which includes a microservice node; the microservice node matches the data type required by the business service and distributes the instructions that need to be adjusted to the corresponding business microservice.

[0043] Business microservices are bidirectionally connected with each other, and uplink and downlink data use two communication links; each business service defines uplink instructions and downlink data separately;

[0044] The microservices of the business layer include data collection services, data processing services and other microservices; the data collection service is responsible for the access of peripheral data of the entire system; the data processing service registers the data types it needs with one or more data collection services through the registration service module according to the processed business; it can also register the data types it needs with other data processing services through the registration service module to form multi-level data processing; the processed data is broadcast to other required services; if the data needs to be broadcast or distributed to the client, it is forwarded to the client through the data interface.

[0045] The service control center interacts with the communication layer to obtain the communication status of the communication layer; based on the obtained communication status, the communication efficiency of the communication layer is judged, and the communication layer is adjusted, specifically: the service control center and the communication layer obtain the real-time communication status of the communication layer through an interface or a communication protocol; the service control center dynamically adjusts the network topology of the communication layer or increases the bandwidth according to the channel utilization, data transmission rate, signal-to-noise ratio, bit error rate, throughput, system capacity and system reliability to improve the channel utilization and the overall performance of the system.

[0046] The service control center exchanges information with the business layer through the communication layer to obtain the status information of the business service in real time; based on the status information obtained, the adjustment instructions sent to the business layer are as follows:

[0047] After receiving the status query request sent by the service control center, the business layer generates status information according to its own operating status and returns it to the service control center through the communication layer;

[0048] The service control center identifies the received status information and obtains potential abnormal behaviors; after receiving the adjustment instruction, the business layer makes corresponding adjustments according to the specific content of the instruction, including: modifying the configuration file, reallocating CPU or memory resources and adjusting the business logic.

[0049] The service control center interacts with the underlying OS through the communication layer to obtain the real-time usage of system resources, and then dynamically adjusts the number and configuration of service instances. Specifically, the service control center sends a query request to the underlying OS through the communication layer to obtain the real-time usage of CPU utilization, memory usage, disk I / O, and network bandwidth, and based on the resource usage, determines whether it is necessary to dynamically adjust the number and configuration of service instances; if necessary, after receiving the adjustment instruction, the underlying OS starts or stops the service instance and adjusts the configuration parameters of the instance to optimize the underlying OS.

[0050] The communication layer includes API interface, communication mode and communication protocol; the communication layer is connected with the service control center, business layer and underlying OS through API interface; the communication layer also exchanges information with external clients through API interface; the API interface mainly includes business data interface, instruction interface and communication interface, etc. The communication layer interacts with the service control center, business layer and underlying OS through one or more of the message mechanism of simulated message queue, synchronous communication mode, asynchronous communication mode and semi-synchronous communication mode; the communication protocol used is RPC protocol based on TCP and Http communication protocol.

[0051] The underlying OS is the operating system, including Windows, Linux and Kylin systems.

[0052] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data processing microservice framework based on C++, characterized by: include: Service control center, business layer, communication layer and underlying OS; The service control center interacts with the communication layer to obtain the communication status of the communication layer; based on the obtained communication status, the communication efficiency of the communication layer is determined, and the communication layer is adjusted; The service control center exchanges information with the business layer through the communication layer to obtain the status information of the business service in real time; based on the obtained status information, it sends adjustment instructions to the business layer; The service control center interacts with the underlying OS through the communication layer to obtain the real-time usage of system resources and dynamically adjust the number and configuration of service instances; The communication layer is connected to the underlying OS to process system commands transmitted and issued by the service control center and the business layer.

2. The C++-based data processing microservice framework according to claim 1, characterized in that: The service control center includes a registration service module, an instruction distribution module and a service management module; The registration service module ensures that all business services can be registered in the service control center before or when they are started, and each service registered in the service control center obtains a unique identity tag; through the registration service module, the service control center can track all available services in the system, thereby supporting communication and calling between services; The instruction distribution module matches the data type required by the business service with the microservice node in the business layer, and broadcasts or distributes the data to the business layer accordingly when the data is published; The service management module is responsible for monitoring and managing the network location and operation status of the business layer. If any abnormality is found in the service, the instruction distribution module sends an adjustment instruction to the business layer.

3. The C++-based data processing microservice framework according to claim 2, characterized in that: The business layer includes multiple business microservices, and the business microservices include microservice nodes; the microservice nodes match the data types required by the business services and distribute the instructions that need to be adjusted to the corresponding business microservices.

4. The C++-based data processing microservice framework according to claim 3, characterized in that: The business microservices are bidirectionally connected with each other, and the uplink and downlink data use two communication links; each business service defines the uplink instructions and downlink data separately; The microservices of the business layer include data collection services and data processing services; the data collection service is responsible for the access of peripheral data of the entire system; the data processing service is to register the data types it needs with one or more data collection services through the registration service module according to the processed business; You can also register the data types you need with other data processing services through the registration service module to form multi-level data processing; broadcast the processed data to other required services; if the data needs to be broadcast or distributed to the client, it will be forwarded to the client through the data interface.

5. The C++-based data processing microservice framework according to claim 4, characterized in that: The service control center interacts with the communication layer to obtain the communication status of the communication layer; based on the obtained communication status, the communication efficiency of the communication layer is judged, and the communication layer is adjusted, specifically: the service control center and the communication layer obtain the real-time communication status of the communication layer through an interface or a communication protocol; the service control center dynamically adjusts the network topology of the communication layer or increases the bandwidth according to the channel utilization, data transmission rate, signal-to-noise ratio, bit error rate, throughput, system capacity and system reliability to improve the channel utilization and the overall performance of the system.

6. The C++-based data processing microservice framework according to claim 5, characterized in that: The service control center exchanges information with the business layer through the communication layer to obtain the status information of the business service in real time; based on the obtained status information, the adjustment instructions sent to the business layer are specifically: After receiving the status query request sent by the service control center, the business layer generates status information according to its own operating status and returns it to the service control center through the communication layer; The service control center identifies the received status information and obtains potential abnormal behavior; After receiving the adjustment instruction, the business layer makes corresponding adjustments according to the specific content of the instruction, including: modifying the configuration file, reallocating CPU or memory resources, and adjusting the business logic.

7. The C++-based data processing microservice framework according to claim 6, characterized in that: The service control center interacts with the underlying OS through the communication layer to obtain the real-time usage of system resources, and then dynamically adjusts the number and configuration of service instances. Specifically, the service control center sends a query request to the underlying OS through the communication layer to obtain the real-time usage of CPU utilization, memory occupancy, disk I / O, and network bandwidth, and based on the resource usage, determines whether it is necessary to dynamically adjust the number and configuration of service instances; if necessary, after receiving the adjustment instruction, the underlying OS starts or stops the service instance and adjusts the configuration parameters of the instance to optimize the underlying OS.

8. The C++-based data processing microservice framework according to claim 7, characterized in that: The communication layer includes an API interface, a communication mode and a communication protocol; the communication layer is connected to the service control center, the business layer and the underlying OS through the API interface; the communication layer also exchanges information with external clients through the API interface; the communication layer interacts with the service control center, the business layer and the underlying OS through one or more of the message mechanism of the simulated message queue, the synchronous communication mode, the asynchronous communication mode and the semi-synchronous communication mode; the communication protocol used is the RPC protocol based on the TCP and Http communication protocols.

9. The C++-based data processing microservice framework according to claim 8, characterized in that: The underlying OS includes Windows, Linux and Kylin systems.

Images