Business processing system

Abstract

The invention discloses a business processing system, and relates to the technical field of computers, a monitoring end collects performance parameters detected by a data end, and the performance parameters of the data end are synchronized to a business end in time, so that the business end can acquire target business data according to the performance parameters of each data end provided by the monitoring end when the business end needs to acquire the target business data. And selecting a target data end from the plurality of data ends to send the target read request. It is ensured that the service end always obtains the service data from the data end with the read request processing performance larger than the target processing performance, and therefore the service data reading efficiency is improved. Meanwhile, the monitoring end monitors the performance parameters of the data end, abnormal conditions of the data end are found in time, the usability and stability of the service processing system are improved, the technical problem that the service processing efficiency is low in the related technology is solved, and the technical effect of improving the service processing efficiency is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a business processing system. BACKGROUND

[0002] In the related art, in a database system adopting a master-slave replication architecture, in order to ensure instant updating and consistency of business data, a business end usually obtains business data from a single data end, such as a master database. On the one hand, it is easy to cause the single data end to be overloaded and unable to provide business data in time. Especially in a high concurrency scenario, the processing capacity of the single data end is limited, and the response speed becomes slow, thereby causing the business end to be unable to obtain business data from the data end in time, and the business processing efficiency is low. With the increase of the number of businesses, this problem becomes more and more prominent, which seriously affects the performance of the business processing system and the user experience. On the other hand, since the reading capacity of other data ends is not fully utilized, the overall resources of the system are idle and wasted.

[0003] In the related art, an effective solution has not been proposed for the technical problem of low business processing efficiency. SUMMARY

[0004] The present application provides a business processing system to at least solve the technical problem in the related art that the business data is obtained from a single data end, causing the data end to be overloaded, or the data end to be abnormal, the data end being unable to provide business data in time, and the business processing efficiency being low.

[0005] The present application provides a business processing system, comprising: a plurality of business ends, a monitoring end and a plurality of data ends, the plurality of data ends are connected with the monitoring end, the monitoring end is further connected with each business end, and each data end stores business data required by each business end for processing a business;

[0006] The data end is configured to detect a current performance parameter and transmit the performance parameter to the monitoring end, wherein the performance parameter is used to indicate a processing performance of the data end on a read request at present;

[0007] The monitoring end is configured to receive the performance parameter transmitted by each data end, and transmit a data end identifier and the performance parameter having a corresponding relationship to each business end;

[0008] The target service end of the plurality of service ends is used for running a target service, and is used for matching a target data end identifier from a data end identifier and a performance parameter having a corresponding relationship, and the performance parameter satisfying a target parameter condition, sending a target read request to a target data end having the target data end identifier, and receiving target service data returned by the target data end in response to the target read request, wherein the target read request is used to request to obtain service data required for running the target service, and the performance parameter satisfying the target parameter condition indicates that the processing performance of the corresponding data end to the read request is greater than a target processing performance.

[0009] Through the present application, the plurality of data ends in the service processing system can detect the processing performance of each data end to the read request, and transmit the detected performance parameter to the monitoring end. After receiving the performance parameter transmitted by each data end, the monitoring end transmits the data end identifier of each data end and the corresponding performance parameter to each service end. When the service end needs to obtain target service data from the database during processing the service, the target data end identifier is matched according to the data end identifier and the performance parameter forwarded by the monitoring end, and then a target read request is sent to the target data end corresponding to the target data end identifier, and target service data returned by the target data end in response to the target read request is received. That is, the performance parameter detected by the data end is collected by the monitoring end, and the performance parameter of the data end is synchronized to the service end in time, so that the service end can select the target data end from the plurality of data ends to send the target read request according to the performance parameter of each data end provided by the monitoring end when the target service data is needed. It is ensured that the service end always obtains the service data from the data end whose processing performance to the read request is greater than the target processing performance, thereby improving the efficiency of reading the service data. At the same time, the abnormal situation of the data end can be found in time through the monitoring of the performance parameter of the data end by the monitoring end, thereby improving the availability and stability of the service processing system, and avoiding the situation that the data end is overloaded due to processing the read request by a single data end in a high concurrency situation. Therefore, the technical problem that the service data is obtained from a single data end in the related art, the data end is overloaded, or the data end is abnormal and cannot provide the service data in time, thereby reducing the efficiency of processing the service, can be solved, and the technical effect of improving the efficiency of processing the service is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating laborious work.

[0011] Figure 1 It is a structural block diagram of a service processing system in the related art;

[0012] Figure 2 A structural block diagram of a service processing system according to an embodiment of the present application;

[0013] Figure 3 A flowchart of detecting and sending performance parameters of a data end according to an embodiment of the present application;

[0014] Figure 4 A flowchart of a watch mechanism triggering process in a monitoring end according to an embodiment of the present application;

[0015] Figure 5 A flowchart of managing performance parameter registration information in a monitoring end according to an embodiment of the present application;

[0016] Figure 6 A flowchart of obtaining target service data in a service end according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0018] It should be noted that, in the description of the present application, the terms “include”, “contain” or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0019] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0020] The terms involved in the embodiments of the present application are explained as follows:

[0021] PostgreSQL: an open source relational database management system;

[0022] ETCD (Etcd Distributed Key-Value Store): a highly available distributed key-value storage system;

[0023] TTL (Time To Live, Time To Live): used to represent the validity period of database node information.

[0024] This application proposes a business data transmission system. Before describing the optional embodiments of this application, in order to better understand the inventive concept and the inventiveness of this solution, the relevant technologies are first described: Figure 1 This is a structural block diagram of a business processing system in related technologies, such as... Figure 1 As shown, a master-slave database architecture consists of one master database and multiple slave databases. The master database handles all write operations (such as data insertion, update, and deletion) sent by the slave clients, while the slave databases maintain data consistency by replicating data from the master database. The main purpose of the master-slave database architecture is to achieve redundant data backup, ensuring that in the event of a failure in the master database, the client can quickly switch to the slave database, thereby guaranteeing business continuity.

[0025] The business data transmission system in related technologies has the following technical problems:

[0026] 1) Read operations are concentrated on the main database, which leads to excessive load on the main database. Especially in high-concurrency scenarios, the response speed of the main database slows down, affecting the efficiency of business processing.

[0027] 2) The database is only used to copy data from the master database, which wastes resources in order to make full use of its read capabilities.

[0028] To address the aforementioned issues, embodiments of this application provide a business processing system, comprising: multiple business terminals, a monitoring terminal, and multiple data terminals. Each data terminal is connected to the monitoring terminal, and the monitoring terminal is also connected to each business terminal. Each data terminal stores the business data required for processing business operations by each business terminal. A data terminal is used to detect current performance parameters and transmit these parameters to the monitoring terminal, wherein the performance parameters indicate the current processing performance of the data terminal for read requests. The monitoring terminal is used to receive the performance parameters transmitted by each data terminal and transmit corresponding data terminal identifiers and performance parameters to each business terminal. A target business terminal among the multiple business terminals, used to run a target business, is used to match a target data terminal identifier whose performance parameters satisfy the target parameter conditions from the corresponding data terminal identifiers and performance parameters, send a target read request to the target data terminal with the target data terminal identifier, and receive the target business data returned by the target data terminal in response to the target read request. The target read request is used to request the business data required to run the target business, and the performance parameters satisfying the target parameter conditions indicate that the current processing performance of the corresponding data terminal for read requests is greater than the target processing performance.

[0029] Optionally, in this embodiment, Figure 2 This is a structural block diagram of a business processing system according to an embodiment of this application, such as... Figure 2As shown, the business processing system includes a plurality of business ends, a monitoring end, and a plurality of data ends. The business ends can interact with the plurality of data ends through the monitoring end. The business ends can be, but are not limited to, systems with data processing and business logic execution capabilities, including but not limited to: Web (Web Application) applications that can process user requests such as order processing, user queries, etc.; data analysis systems that can generate reports, perform data analysis and data mining; background service programs that can process background operations such as timed tasks, data synchronization, etc. During the running process of the business end, the business data needs to be obtained to support the running of the business logic of the business end. The business data can be, but is not limited to, various data generated, collected and processed by the enterprise in the process of daily operation and management, including user information, order data, payment data and inventory data, etc.

[0030] Optionally, in this embodiment, the data end can be, but is not limited to, a database storing business data, including but not limited to: master-slave databases (such as PostgreSQL), other relational databases (such as MySQL), and non-relational databases (such as MongoDB), etc. The plurality of databases all store the business data required by the business end to process the business, and can respond to the read request sent by the business end and return the business data required to be read by the read request.

[0031] Optionally, in this embodiment, as shown in Figure 2 The data end detects its performance parameters at regular intervals and reports the performance parameters to the monitoring end. The performance parameters can be, but are not limited to, parameters that can reflect the load status of the data end, and can be obtained through the current access amount, CPU (Central Processing Unit) usage, memory usage, bandwidth usage, etc.

[0032] Optionally, in this embodiment, the monitoring end can be, but is not limited to, a system with data storage capability and monitoring capability, including but not limited to a distributed key-value storage system ETCD. As shown in Figure 2 After receiving the performance parameters reported by each data end, the monitoring end stores the data end identifier (such as IP (Internet Protocol) address or node name, etc. unique identifier) corresponding to each data end (node) and the performance parameters. The data end identifier corresponding to the data end and the performance parameters can be stored in the monitoring end in the form of key-value pairs, such as taking the data end identifier of the data end as the key, and taking the performance parameters corresponding to the data end as the value, to obtain the performance parameter key-value pair.

[0033] Optionally, in the embodiment, the monitoring end also allows setting monitoring for all the key-value pair information stored by it, and when the key-value pair information stored by it changes, the monitoring end will immediately detect the changed key-value pair information and push the updated key-value pair information to all the business ends subscribing to the key-value pair information, so that the business end can obtain the latest information of the key-value pair information it subscribes to in time.

[0034] Optionally, in the embodiment, when the business end receives a user request or a system task and needs to obtain business data to support the running of its business logic, it can select the optimal data end from multiple data ends for read operation according to the performance parameters of the data end provided by the monitoring end, and receive the business data returned by the data end in response to the read operation. Through the interaction between the data end and the business end, it can be ensured that the business end can efficiently and accurately obtain the required business data, thereby improving the efficiency and accuracy of business processing.

[0035] Optionally, in the embodiment, the monitoring end can also delete the performance parameter information of a data end when the data end does not normally send performance parameters, or when the performance parameters of the data end exceed the normal performance parameter threshold, through monitoring all the key-value pair information stored by it, so as to ensure that the business end does not send a read request to the faulty data end when it needs to obtain business data, and at the same time, send warning information to the business end to remind the operation and maintenance personnel to detect the abnormal condition of the data end, thereby realizing real-time monitoring of the state of multiple data ends and improving the availability and stability of the business processing system.

[0036] Optionally, in the embodiment, when a data end needs to be added or reduced in the business processing system, the characteristics that the data end can detect its performance parameters and send them to the monitoring end are used, so that the business end can still normally obtain business data when the data end is added or reduced, avoiding the situation of business interruption, and realizing high scalability of the business processing system. Specifically, when the monitoring end detects the performance parameters of the added data end, it can send the identifier and the corresponding performance parameters of the added data end to the business end, and the business end can automatically select to obtain business data from the added data end; when the monitoring end detects that a data end does not send performance parameters, it does not allow sending the data end identifier and performance parameters corresponding to the data end to the business end, and the business end will not consider obtaining business data from the data end.

[0037] The service processing system provided in the embodiments of the present application collects the performance parameters detected by the data end through the monitoring end, and synchronizes the performance parameters of the data end to the service end in time, so that the service end can select a target data end for a target read request from multiple data ends according to the performance parameters of the data ends provided by the monitoring end when the service end needs to obtain target service data. It is ensured that the service end always obtains service data from a data end with a read request processing performance greater than a target processing performance, thereby improving the efficiency of service data reading. Meanwhile, the monitoring of the performance parameters of the data end by the monitoring end can also timely discover abnormal conditions of the data end, improve the availability and stability of the service processing system, and avoid the situation that a single data end processes read requests under high concurrency, which easily leads to excessive load of the data end or abnormal conditions of the data end, and the data end cannot provide service data in time, thereby leading to low service processing efficiency. Therefore, the technical problem that service data is obtained from a single data end in the related art, the data end cannot provide service data in time due to excessive load of the data end, and the service processing efficiency is low can be solved, and the technical effect of improving the service processing efficiency is achieved.

[0038] As an optional solution, the data end comprises a collector, a parameter detector and one or more execution components, the one or more execution components are respectively connected with the collector, and the collector is further connected with the parameter detector, wherein the one or more execution components are used to process a read request currently received by the data end; the collector is used to collect component running information and request execution information of the data end according to a preset collection period, and transmit the component running information and the request execution information to the parameter detector, wherein the component running information records component resource parameters of each execution component in the one or more execution components, the component resource parameters are used to indicate the usage of the corresponding execution component, and the request execution information is used to indicate the number of read requests currently processed by the data end; the parameter detector is used to receive the component running information and the request execution information from the collector, detect the performance parameters according to the component running information and the request execution information, and transmit the performance parameters to the monitoring end.

[0039] Optionally, in the embodiments, the data end comprises the collector, the parameter detector and the one or more execution components. The execution components can be but are not limited to components running on the data end, and can include but are not limited to CPU, memory, network card and the like. The execution components can process the read request received by the data end, and send service data indicated by the read request to the service end. Each execution component is connected with the collector, so that the collector can obtain running information thereof.

[0040] Optionally, in the embodiment, the collector collects the component running information and the request execution information of the data terminal at a preset collection period (e.g., every second), the component running information includes component resource parameters of each execution component, such as CPU usage, memory usage, bandwidth usage, etc., and the request execution information includes the number of read requests being processed by the data terminal. The collection of the component running information can include, but is not limited to, directly obtaining the component running information of each execution component from the execution component, or indirectly obtaining the component running information of each execution component through a system monitoring tool or a programming interface.

[0041] Optionally, in the embodiment, after the parameter detector receives the component running information and the request execution information from the collector, the parameter detector calculates the performance parameter of the data terminal according to the information including the CPU usage, the memory usage, the bandwidth usage and the service access amount, and then transmits the performance parameter to the monitoring terminal. The performance parameter is a comprehensive index that can reflect the processing performance of the data terminal on the read request.

[0042] As an optional solution, the parameter detector includes a weight generation tool and a parameter generation tool, and the weight generation tool is connected to the parameter generation tool. The weight generation tool is configured to detect the component weight of each execution component in one or more execution components, obtain the execution component and the component weight having a corresponding relationship, and transmit the execution component and the component weight having the corresponding relationship to the parameter generation tool. The component weight is used to indicate the degree of dependence of the data terminal on the corresponding execution component when responding to the read request. The parameter generation tool is configured to receive the execution component and the component weight having the corresponding relationship, and calculate the performance parameter of the data terminal according to the component running information, the request execution information and the execution component and the component weight having the corresponding relationship.

[0043] Optionally, in the embodiment, the parameter detector includes the weight generation tool and the parameter generation tool. The manner in which the parameter detector detects the performance parameter according to the component running information and the request execution information can include, but is not limited to, the following. First, considering that the data terminal has different degrees of resource dependence on different execution components when processing the read request, the weight generation tool is used to detect the degree of dependence of the data terminal on each execution component when responding to the read request, obtain a plurality of execution components and component weights having a corresponding relationship, and transmit the execution components and the component weights having the corresponding relationship to the parameter generation tool. Then, after the parameter generation tool receives the execution components and the component weights having the corresponding relationship transmitted by the weight generation tool, the parameter generation tool calculates the performance parameter of the data terminal according to the component running information, the request execution information and the execution components and the component weights having the corresponding relationship.

[0044] As an optional solution, the weight generation tool is configured to detect the component weight of each of the one or more execution components by detecting the running scenario in which the data end is currently located, and obtaining the execution component and the component weight with a corresponding relationship by the following steps: in the case that it is detected that the data end is currently located in a computation-intensive scenario, setting the computation component as a first weight, setting the storage component as a second weight, and setting the transmission component as a third weight, wherein the execution component comprises the computation component, the storage component and the transmission component, the computation component is configured to provide computation resources in the process of processing the read request, the storage component is configured to provide storage resources in the process of processing the read request, and the transmission component is configured to provide bandwidth resources in the process of processing the read request, the first weight is greater than the second weight, and the second weight is greater than the third weight; in the case that it is detected that the data end is currently located in a storage-intensive scenario, and setting the transmission component as a fourth weight, setting the storage component as a fifth weight, and setting the computation component as a sixth weight, wherein the fourth weight is greater than the fifth weight, and the fifth weight is greater than the sixth weight.

[0045] Optionally, in the embodiment, the running scenarios can include, but are not limited to, a computation-intensive scenario, a storage-intensive scenario, a transmission-intensive scenario, and the like. Different business scenarios correspond to different types of business processing tasks, and different business processing tasks have different degrees of dependence on resources of different execution components. For example, in the computation-intensive scenario, computation-intensive tasks such as big data analysis, scientific computing, machine learning, and the like need to be processed, a large amount of computation resources are needed to process complex computing logic, and a certain amount of storage resources are needed to store data to reduce the number of accesses to the disk, but the dependence on the computation resources is relatively small, and the bandwidth resources mainly affect the transmission speed of data and have the least impact on the computing task. Therefore, the dependence on the computation resources is the highest when processing the computation-intensive tasks, followed by the storage resources, and the utilization rate of the bandwidth resources is relatively low. In the storage-intensive scenario, storage-intensive tasks such as large-scale data storage, file system operation, log management, and the like need to be processed, mainly involving a large amount of data read and write operations, and efficient data transmission and storage capabilities are needed, the data read and write operations need to be transmitted through the network, and the bandwidth resources directly affect the read and write speed of the data. Therefore, the dependence on the bandwidth resources is the highest when processing the storage-intensive tasks, followed by the storage resources, and the utilization rate of the computation resources is relatively low.

[0046] Optionally, in the embodiment, the weight generation tool obtains the execution components and component weights with the corresponding relationship in the following manner, which can but not limited to include: first, detecting the running scenario in which the data terminal is currently located. When it is detected that the data terminal is in an operation-intensive scenario, it means that the higher the degree of dependence of the current business processing task on the operation resources of the operation component (such as CPU), the second is the storage resources of the storage component (such as memory, hard disk), and the bandwidth resources of the transmission component (such as network interface) have the lowest degree of dependence, therefore, the component weight of the operation component is set as the first weight, the component weight of the storage component is set as the second weight, and the component weight of the transmission component is set as the third weight, wherein the first weight is greater than the second weight, and the second weight is greater than the third weight. When it is detected that the data terminal is in a storage-intensive scenario, it means that the higher the degree of dependence of the current business processing task on the bandwidth resources of the transmission component, the second is the storage resources of the storage component, and the operation resources of the operation component have the lowest degree of dependence, therefore, the component weight of the transmission component is set as the fourth weight, the component weight of the storage component is set as the fifth weight, and the component weight of the operation component is set as the sixth weight, wherein the fourth weight is greater than the fifth weight, and the fifth weight is greater than the sixth weight.

[0047] As an optional solution, detecting the running scenario in which the data terminal is currently located includes: detecting the operation component utilization rate of the operation component in the data terminal, the storage component utilization rate of the storage component, and the transmission component utilization rate of the transmission component; in the case that the operation component utilization rate is greater than a first utilization threshold, it is determined that the data terminal is currently located in an operation-intensive scenario; in the case that the storage component utilization rate is greater than a second utilization threshold, and / or the transmission component utilization rate is greater than a third utilization threshold, it is determined that the data terminal is currently located in a storage-intensive scenario.

[0048] Optionally, in the embodiment, the weight generation tool detects the running scenario in which the data terminal is currently located in the following manner, which can but not limited to include: first, detecting the component resource parameters of all execution components in the data terminal, that is, the usage of each execution component, obtaining the operation component utilization rate of the operation component, including CPU usage rate, the storage component utilization rate of the storage component, including memory usage rate, and the transmission component utilization rate of the transmission component, including bandwidth usage rate. Then, according to the operation component utilization rate, the storage component utilization rate and the transmission component utilization rate, the running scenario in which the data terminal is currently located is judged, for example, in the case that the CPU usage rate is greater than a first utilization threshold, it is determined that the data terminal is currently located in an operation-intensive scenario; in the case that the memory usage rate is greater than a second utilization threshold, it is determined that the data terminal is currently located in a storage-intensive scenario; in the case that the bandwidth usage rate is greater than a third utilization threshold, it is determined that the data terminal is currently located in a storage-intensive scenario; in the case that the memory usage rate is greater than a second utilization threshold, and the bandwidth usage rate is greater than a third utilization threshold, it is determined that the data terminal is currently located in a storage-intensive scenario.

[0049] Optionally, in the embodiment, the first utilization threshold, the second utilization threshold and the third utilization threshold can be set according to actual needs, for example, when the first utilization threshold is set to 80%, it is considered that the current business processing system is in a computation-intensive scenario when the CPU usage exceeds 80%; when the second utilization threshold is set to 60%, it is considered that the current business processing system is in a storage-intensive scenario when the network bandwidth usage exceeds 60%. By detecting the component resource parameters of all execution components in the current data end, the running scenario of the current data end is judged, so that the component weight is dynamically adjusted according to the business processing needs, more accurate load balancing is realized, and the technical effect of improving the business processing effect is improved.

[0050] Optionally, in the embodiment, the parameter generation tool obtains the component running information (such as CPU usage C i , memory usage M i and bandwidth usage B i ) of the i-th data end, and the request execution information (such as business access volume T i ), which can but not limited to calculate the performance parameter of the i-th data end by the following formula:

[0051] Wherein, the performance parameter i of the i-th data end, the calculation method defaults that the data end is the same in the processing read request, the resource dependence degree of different execution components.

[0052] Optionally, in the embodiment, the parameter generation tool obtains the component running information (such as CPU usage C i , memory usage M i and bandwidth usage B i ) of the i-th data end, and the request execution information (such as business access volume T i ) and the component weight of each execution component, which can but not limited to calculate the performance parameter of the i-th data end by the following formula:

[0053] Wherein, the performance parameter i of the i-th data end, k1, k2 and k3 are the component weights of the computation component, the storage component and the transmission component, respectively, and the calculation method defaults that the data end is different in the processing read request, the resource dependence degree of different execution components.

[0054] As an optional scheme, the monitoring end comprises a parameter storage and a parameter synchronizer, and the parameter storage and the parameter synchronizer are connected, wherein the parameter storage is used to store the current performance parameter of each data end; the parameter synchronizer is used to synchronize the performance parameter stored in the parameter storage to each business end.

[0055] Optionally, in this embodiment, a preset collection period is set for the collector, so that the collector triggers a timed task according to the preset collection period to complete the performance parameter registration, that is, to transmit the current performance parameters to the monitoring terminal. The transmission format can be JSON format. Figure 3 This is a flowchart illustrating the data terminal detection and transmission performance parameters according to an embodiment of this application, such as... Figure 3 As shown, each data endpoint (such as a PostgreSQL database node) completes performance parameter registration through the following steps:

[0056] S301: Scheduled task trigger. Triggers the performance parameter registration process according to the preset data collection cycle;

[0057] S302: Check service status. Verify if the PostgreSQL service on the current data endpoint is running normally; if the service is not running, terminate the process and do not register performance parameters; if the service is running normally, execute S303.

[0058] S303: Collect performance data. Collect component operation information and request execution information such as CPU utilization, memory utilization, bandwidth utilization, and service access volume at this data terminal;

[0059] S304: Calculate load weight: Calculate the performance parameters (i.e., load weight) of the data terminal based on the component operation information, request execution information, and the corresponding execution components and component weights;

[0060] S305: Report performance parameters. Using the IP address of this data terminal as the key, transmit the performance parameters of the data terminal to the monitoring terminal (ETCD). If the transmission fails, log this error message; if the transmission is successful, confirm that the performance parameter registration of the data terminal is complete.

[0061] Optionally, in this embodiment, the monitoring terminal includes a parameter memory and a parameter synchronizer, which are connected. The parameter memory can store the performance parameters transmitted by each data terminal, stored in the form of corresponding data terminal identifiers and performance parameters, and ensures that the stored performance parameters are the latest performance parameters of the data terminal identified by the corresponding data terminal identifier. The parameter synchronizer can synchronize the corresponding data terminal identifiers and performance parameters stored in the parameter memory to each service terminal.

[0062] As an optional solution, the parameter storage stores data end identifier and performance parameter having a corresponding relationship, and the parameter synchronizer is further configured to synchronize the performance parameter stored in the parameter storage to each service end by the following steps: receiving a reference data end identifier corresponding to a reference service end, wherein the reference service end is any one of the plurality of service ends, and the reference data end identifier is used to identify a reference data end of the plurality of data ends that is allowed to process a read request sent by the reference service end; matching a reference performance parameter corresponding to the reference data end identifier from the data end identifier and the performance parameter having the corresponding relationship; and transmitting the reference performance parameter to the reference service end.

[0063] Optionally, in the embodiment, the manner in which the parameter synchronizer synchronizes the performance parameter stored in the parameter storage to each service end can include but is not limited to the following: first, a reference data end identifier corresponding to a reference service end (any one of the plurality of service ends) is received, and the reference data end identified by the reference data end identifier can process a read request sent by the reference service end. Then, a reference performance parameter corresponding to the reference data end identifier is matched from the data end identifier and the performance parameter having the corresponding relationship stored in the parameter storage, and the reference data end identifier and the reference performance parameter having the corresponding relationship are transmitted to the reference service end.

[0064] Optionally, in the embodiment, the manner in which the parameter synchronizer synchronizes the performance parameter stored in the parameter storage to each service end can include but is not limited to the following: when starting, the reference service end (any one of the plurality of service ends) subscribes to change information of a reference performance parameter corresponding to a reference data end identifier through a monitoring end (such as an ETCD client), wherein the reference data end identified by the reference data end identifier is a data end allowed to process a read request sent by the reference service end; when the reference performance parameter corresponding to the subscribed reference data end identifier is updated, the parameter synchronizer sends the updated reference performance parameter to the corresponding reference service end. Specifically, Figure 4 A flowchart of a watch mechanism triggering process in the monitoring end according to the embodiment of the application is shown in FIG. 1. Figure 4As shown, the monitoring end sets monitoring for all data end identifiers and performance parameter key-value pairs stored in the parameter storage by using the watch mechanism. When a reference data end performs performance parameter registration, the performance parameter registration information (i.e., the reference data end identifier and the reference performance parameter key-value pair having a corresponding relationship) stored in the parameter storage is updated, triggering the watch process. The monitoring end can monitor the update, and the watch mechanism can timely notify the business module (i.e., the reference business end) subscribed to the reference data end of the update, and transmit the updated reference data end identifier and the reference performance parameter having a corresponding relationship to the reference business end. After receiving the updated reference data end identifier and the reference performance parameter having a corresponding relationship, the reference business end processes the watch event by using a callback function, and updates the reference performance parameter corresponding to the reference data end identifier in the local cache, so that the reference business end can timely acquire the latest reference performance parameter of the reference data end identified by the reference data end identifier subscribed by the reference business end, and record the latest reference performance parameter. In this way, dynamic load adjustment is achieved, and the optimal data end is selected according to the current latest performance parameter each time a read request is sent.

[0065] As an optional solution, the monitoring end further includes a parameter deleter, wherein the parameter deleter is connected with the parameter storage, and is configured to acquire the parameter storage time length and the parameter valid time length of each performance parameter stored in the parameter storage, wherein the parameter storage time length is the time length for which the corresponding performance parameter has been stored in the parameter storage, and the parameter valid time length is used to indicate the time length for which the corresponding performance parameter is allowed to be stored in the parameter storage; filter out invalid performance parameters having a parameter storage time length greater than or equal to the parameter valid time length from the performance parameters stored in the parameter storage; and delete the invalid performance parameters.

[0066] Optionally, in the embodiment, the monitoring end further includes a parameter deleter, which can acquire the parameter storage time length (the time length for which the corresponding performance parameter has been stored in the parameter storage) and the parameter valid time length (the time length for which the corresponding performance parameter is allowed to be stored in the parameter storage) of each performance parameter stored in the parameter storage, and manage the performance parameters corresponding to each data end by comparing the parameter storage time length and the parameter valid time length. Figure 5 The flowchart for the monitoring end to manage the performance parameter registration information according to the embodiment of the application is shown in Figure 5 As shown, the steps for the monitoring end to manage the performance parameters of each data end can include but are not limited to the following steps:

[0067] S501, performance parameter registration and TTL setting. Each data end triggers the performance parameter registration process by a timing task, and the specific performance parameter registration process is shown in Figure 3The performance parameter registration information of each data terminal is set with a valid period TTL (i.e. the valid time length of the parameter), and the value of TTL is higher than the cycle time length of the timing task (triggered according to the preset collection cycle) ;

[0068] S502, detecting the performance parameter registration state. When the PostgreSQL service of the current data terminal is running normally and the performance data is collected normally, it indicates that the current data terminal has completed the performance parameter registration process according to the preset collection cycle, and the process goes to step S507; otherwise, it indicates that the current data terminal has not completed the performance parameter registration process according to the preset collection cycle, and the process goes to step S503;

[0069] S503, deleting the performance parameter registration information. When the current data terminal has not completed the performance parameter registration process according to the preset collection cycle, the corresponding performance parameter registration information of the data terminal has been stored in the parameter storage for a time length exceeding the valid period TTL (the allowed storage time length in the parameter storage), the data terminal is confirmed as a fault data terminal, and the corresponding performance parameter information of the data terminal is deleted from the parameter storage;

[0070] S504, notifying the business terminal by watch. After the monitoring terminal deletes the corresponding performance parameter information of the fault data terminal in the parameter storage, the watch process is triggered, and the watch mechanism will timely notify the business terminal subscribed to the data terminal identifier of the deletion;

[0071] S505, updating the cache by the business terminal. The business terminal will delete the corresponding performance parameter of the data terminal identifier in the local cache;

[0072] S506, not sending read request to the fault data terminal. When the business terminal needs to obtain business data, it will not send a read request to the fault data terminal, and at the same time, send a warning information to the business terminal to remind the operation and maintenance personnel to detect the abnormal condition of the data terminal, thereby improving the availability and stability of the business processing system; the process goes to step S511;

[0073] S507, maintaining the heartbeat and updating the data terminal information. When the current data terminal has completed the performance parameter registration process according to the preset collection cycle, the parameter storage also updates the performance parameter registration information (the data terminal identifier and the performance parameter having a corresponding relationship) of the data terminal, and each time the parameter storage is updated, the parameter storage time length (the time length for which the performance parameter has been stored in the parameter storage) is set to zero;

[0074] S508, notifying the business terminal by the watch mechanism. After the monitoring terminal updates the corresponding performance parameter information of the data terminal, the watch process is triggered, and the watch mechanism will timely notify the business terminal subscribed to the data terminal identifier of the update;

[0075] S509, the service end updates the cache. In the case that the performance parameter of the data end identifier is cached locally in the service end, the service end updates the performance parameter corresponding to the data end identifier cached locally; in the case that the performance parameter of the data end identifier is not cached locally in the service end, the performance parameter of the data end identifier is added newly;

[0076] S510, a read request is sent. When the service end needs to obtain service data, the service end selects a data end with the best read request processing performance from a plurality of data ends according to the performance parameter of the actual data end, and sends a read request to the data end with the best read request processing performance from the data end identifier and the performance parameter with the corresponding relationship cached locally;

[0077] S511, the process is ended.

[0078] Optionally, in the embodiment, in order to ensure the fault tolerance rate, the performance parameter registration information of the data end is set to have a validity period TTL (i.e. parameter validity duration) which is greater than or equal to twice the period length of the timing task (the performance parameter registration is triggered according to the preset collection period), i.e. the preset collection period, so as to ensure that if the data end fails to update its performance parameter registration information within a single preset collection period, the monitoring end does not immediately confirm the data end as a faulty data end and delete the performance parameter registration information of the data end, thereby increasing the fault tolerance rate and avoiding business interruption caused by transient abnormalities. At the same time, the value of the validity period TTL of the performance parameter registration information of the data end cannot be too large, otherwise when a real abnormality occurs, the monitoring end does not determine the data end as a faulty data end and does not delete the performance parameter registration information of the data end, so that the performance parameter of the faulty data end still exists in the local cache of the service end, and when a read request is sent to the faulty data end subsequently, the read request cannot be responded normally, resulting in business interruption.

[0079] As an optional solution, the target service end comprises a request sender and a data receiver, wherein the request sender is configured to detect whether there is a read request to be sent in the target service end currently; in the case that it is detected that there is a target read request to be sent in the target service end, a target data end identifier with a performance parameter satisfying a target parameter condition is matched from the data end identifier and the performance parameter with the corresponding relationship, and a target read request is sent to the target data end with the target data end identifier; and the data receiver is configured to receive target service data returned by the target data end in response to the target read request.

[0080] Optionally, in the embodiment, the target service end comprises a request transmitter and a data receiver, which work cooperatively to process the read request and obtain the service data returned by the data end. Specifically, the request transmitter obtains the performance parameter of the data end from the locally cached data end identifier and performance parameter having a corresponding relationship when detecting that the target service end has a read request to be sent, and matches the target data end identifier satisfying the target parameter condition (such as the minimum response time, the maximum availability, etc.) according to the preset target parameter condition, and selects the target data end identified by the target data end identifier to send the read request. The data receiver waits for the response of the target read request, receives the target service data returned by the target data end in response to the target read request, processes and verifies the received service data as necessary, and provides the target service data to the service end for use.

[0081] Optionally, in the embodiment, the target service end can intelligently select the optimal data end to send the read request and effectively receive and process the returned data, thereby not only improving the efficiency and accuracy of data reading, but also enhancing the flexibility and reliability of the system. By dynamically selecting the data end with the optimal performance, the system resources can be better utilized and the overall performance can be improved.

[0082] As an optional solution, the request transmitter is configured to match the target data end identifier satisfying the target parameter condition from the data end identifier and performance parameter having a corresponding relationship by the following steps: locating the maximum target performance parameter from the data end identifier and performance parameter having a corresponding relationship, wherein the greater the performance parameter, the greater the current processing performance of the data end to the read request; and determining the data end identifier corresponding to the target performance parameter in the data end identifier and performance parameter having a corresponding relationship as the target data end identifier.

[0083] Optionally, in the embodiment, the manner in which the request transmitter matches the target data end identifier satisfying the target parameter condition from the performance parameter of the plurality of data end identifiers can but not limited to include: first, calculating the performance parameter weight of the plurality of performance parameters corresponding to the plurality of data end identifiers locally cached by the service end, since the service end locally caches the data end identifier and performance parameter of all the data ends allowed to respond to the read request sent by the service end, the performance parameters of all the data ends are ensured to be within a comparable range by normalizing the plurality of performance parameters corresponding to the plurality of data end identifiers, and the specific normalization formula is:

[0084] wherein the performance parameter i is the performance parameter of the i th data end, the performance parameter weight i is the performance parameter weight of the i th data end, is the sum of the performance parameters of all n data ends.

[0085] Optionally, in the embodiment, the greater the performance parameter weight calculated by the above normalization formula, the more sufficient the component resources of the component in the data end, and the stronger the processing performance of the read request. When the service end needs to send a read request, the data end with the greatest performance parameter weight is selected as the target data end to send the read request, thereby realizing dynamic load balancing.

[0086] Optionally, in the embodiment, Figure 6 The flowchart for the service end to obtain target service data according to the embodiment of the application is shown in Figure 6 The service end locally caches the latest performance parameters (load conditions) of the latest data ends (pgsql nodes, i.e., PostgreSQL database nodes), and then performs normalization calculation on the plurality of performance parameters based on the plurality of performance parameters of the plurality of databases locally cached by the service end, to obtain the performance parameter weights of the databases. When the service end needs to send a new read request, the data end with the greatest performance parameter weight is selected from the plurality of data ends as the target data end to send the read request. The target data end is considered to be the data end with the least load, the most abundant system resources, and the strongest processing performance of the read request among the data ends that can currently process the read request. If the data end with the highest performance parameter weight fails to respond to the read request, the data ends are sorted from high to low according to the performance parameter weights, and the read request is sent in turn until the target service data is successfully read. Through the above steps, in a high-concurrency scenario, the service end can disperse the read request to a plurality of slave data ends according to the actual business requirements and the performance parameters of the data ends, significantly improve the read operation performance of the entire business processing system without reducing reliability, reduce the burden of the master database, and also ensure the stability of the write operation.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware, but in many cases the former is a better embodiment.

[0088] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0089] The above describes in detail the service processing system provided by the present application. The principles and implementation manners of the present application are described by using specific examples, and the above description of the embodiments is only used to help understand the method of the present application and the core idea thereof. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A business processing system, characterized in that, include: The system includes multiple business terminals, a monitoring terminal, and multiple data terminals. All of the data terminals are connected to the monitoring terminal, and the monitoring terminal is also connected to each of the business terminals. Each of the data terminals stores the business data required by each of the business terminals to process its business. The data terminal is used to detect the current performance parameters and transmit the performance parameters to the monitoring terminal, wherein the performance parameters are used to indicate the current processing performance of the data terminal for read requests; The monitoring terminal is used to receive the performance parameters transmitted by each of the data terminals, and transmit the corresponding data terminal identifiers and the performance parameters to each of the service terminals. Among the multiple service terminals, the target service terminal used to run the target service is used to match the target data terminal identifier whose performance parameters meet the target parameter conditions from the data terminal identifiers and performance parameters that have a corresponding relationship, send a target read request to the target data terminal with the target data terminal identifier, and receive the target service data returned by the target data terminal in response to the target read request. The target read request is used to request the acquisition of the service data required to run the target service, and the performance parameters meeting the target parameter conditions indicate that the current processing performance of the corresponding data terminal for the read request is greater than the target processing performance.

2. The system according to claim 1, characterized in that, The data terminal includes: a data collector, a parameter detector, and one or more execution components, wherein one or more execution components are respectively connected to the data collector, and the data collector is also connected to the parameter detector. One or more of the execution units are used to process the read request currently received by the data terminal; The collector is used to collect component operation information and request execution information of the data terminal according to a preset collection period, and transmit the component operation information and the request execution information to the parameter detector. The component operation information records the component resource parameters of each of the one or more execution components. The component resource parameters are used to indicate the usage status of a corresponding execution component. The request execution information is used to indicate the number of read requests that the data terminal is currently processing. The parameter detector is used to receive the component operation information and the request execution information from the collector; detect the performance parameters based on the component operation information and the request execution information; and transmit the performance parameters to the monitoring terminal.

3. The system according to claim 2, characterized in that, The parameter detector includes: a weight generation tool and a parameter generation tool, wherein the weight generation tool and the parameter generation tool are connected, and wherein... The weight generation tool is used to detect the component weight of each of the execution components in one or more execution components, obtain the execution components and component weights with corresponding relationships, and transmit the execution components and component weights with corresponding relationships to the parameter generation tool, wherein the component weight is used to indicate the degree of dependence of the data end on the corresponding execution component when responding to the read request; The parameter generation tool is used to receive the corresponding execution components and component weights; and to calculate the performance parameters of the data terminal based on the component operation information, the request execution information, and the corresponding execution components and component weights.

4. The system according to claim 3, characterized in that, The weight generation tool is used to detect the component weight of each of the one or more execution components through the following steps, so as to obtain the execution components and their corresponding weights: Detect the current operating scenario of the data terminal; When it is detected that the data terminal is currently in a computationally intensive scenario, the computing component is set as the first weight, the storage component as the second weight, and the transmission component as the third weight. The execution component includes the computing component, the storage component, and the transmission component. The computing component is used to provide computing resources during the processing of the read request, the storage component is used to provide storage resources during the processing of the read request, and the transmission component is used to provide bandwidth resources during the processing of the read request. The first weight is greater than the second weight, and the second weight is greater than the third weight. When it is detected that the data terminal is currently in a storage-intensive scenario, and the transmission component is set as the fourth weight, the storage component as the fifth weight, and the computing component as the sixth weight, wherein the fourth weight is greater than the fifth weight, and the fifth weight is greater than the sixth weight.

5. The system according to claim 4, characterized in that, The detection of the current operating scenario of the data terminal includes: The utilization rates of the computing unit, the storage unit, and the transmission unit in the data terminal are detected. If the utilization rate of the computing unit is greater than a first utilization threshold, it is determined that the data terminal is currently in the computing-intensive scenario. If the utilization rate of the storage component is greater than a second utilization threshold, and / or the utilization rate of the transmission component is greater than a third utilization threshold, it is determined that the data terminal is currently in the storage-intensive scenario.

6. The system according to claim 1, characterized in that, The monitoring terminal includes: a parameter storage device and a parameter synchronizer, wherein the parameter storage device and the parameter synchronizer are connected, wherein... The parameter memory is used to store the current performance parameters of each of the data terminals; The parameter synchronizer is used to synchronize the performance parameters stored in the parameter memory to each of the service terminals.

7. The system according to claim 6, characterized in that, The parameter memory stores the corresponding data terminal identifiers and performance parameters. The parameter synchronizer is further used to synchronize the performance parameters stored in the parameter memory to each of the service terminals through the following steps: Receive a reference data terminal identifier corresponding to a reference service terminal, wherein the reference service terminal is any one of the multiple service terminals, and the reference data terminal identifier is used to identify a reference data terminal among the multiple data terminals that is allowed to process the read request sent by the reference service terminal; The reference performance parameter corresponding to the reference data terminal identifier is matched from the data terminal identifier and the performance parameter that have a corresponding relationship; The reference performance parameters are transmitted to the reference service terminal.

8. The system according to claim 6, characterized in that, The monitoring terminal further includes: a parameter deleter, wherein the parameter deleter is connected to the parameter storage, wherein... The parameter deleter is used to obtain the parameter storage duration and parameter validity duration of each performance parameter stored in the parameter memory, wherein the parameter storage duration is the duration for which the corresponding performance parameter has been stored in the parameter memory, and the parameter validity duration is used to indicate the allowed storage duration of the corresponding performance parameter in the parameter memory; to filter out invalid performance parameters whose parameter storage duration is greater than or equal to the parameter validity duration from the performance parameters stored in the parameter memory; and to delete the invalid performance parameters.

9. The system according to claim 1, characterized in that, The target service terminal includes: a request sender and a data receiver, wherein, The request sender is used to detect whether there is a read request to be sent at the target service terminal; if the target service terminal detects that there is a target read request to be sent, it matches the target data terminal identifier whose performance parameters satisfy the target parameter conditions from the data terminal identifiers and performance parameters that have a corresponding relationship, and sends the target read request to the target data terminal with the target data terminal identifier; The data receiver is used to receive the target business data returned by the target data terminal in response to the target read request.

10. The system according to claim 9, characterized in that, The request sender is configured to match the target data terminal identifier whose performance parameters satisfy the target parameter condition from the data terminal identifiers and performance parameters that have a corresponding relationship through the following steps: The largest target performance parameter is located from the corresponding data terminal identifiers and performance parameters, wherein the larger the performance parameter, the greater the current processing performance of the data terminal for the read request; The data terminal identifier that has a corresponding relationship and the data terminal identifier corresponding to the target performance parameter in the performance parameters are determined as the target data terminal identifier.

Images