Business data reading methods and devices, electronic equipment, and storage media

Abstract

This invention discloses a business data reading method, apparatus, electronic device, and storage medium, relating to the field of financial technology. The method includes: acquiring a target message to be processed from a message queue and determining the target server cluster corresponding to the target message; generating a business data reading request when none of the servers in the target server cluster can retrieve the business data; sending the business data reading request to a second component service layer; and distributing the business data reading request to a cluster of servers waiting to connect to the internet, thereby reading the target business data from the cluster. This invention solves the technical problem in related technologies where distributed system server clusters cannot directly read and process the relevant data in a message after acquiring it.

Description

Technical Field

[0001] This invention relates to the field of financial technology and other related technical fields. Specifically, it relates to a business data reading method and apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of mobile technology, the internet has permeated all aspects of people's lives, and online business volume continues to grow. With the access of a large number of user terminals and services, a massive amount of business data is generated. Generally, this business data is processed by introducing business systems. When building a business system, it is often broken down into multiple application layers, each containing multiple server clusters.

[0003] In related technologies, for message contention scenarios in distributed systems, after the target server cluster obtains / consumes a message, it needs to access the business data of the business application form corresponding to the message in order to perform subsequent operations. However, the cluster server can only access the database server of its own cluster. The business data in the business application form corresponding to the obtained message may be in a different cluster from the consumer group. That is, the server to which the business data to be queried belongs does not belong to the cluster to which the cluster server belongs, resulting in the inability to directly read and process the relevant data (such as business data) after obtaining and consuming the message.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a business data reading method and apparatus, electronic device, and storage medium to at least solve the technical problem in related technologies where distributed system server clusters cannot directly read and process relevant data in messages after obtaining them.

[0006] According to one aspect of the present invention, a business data reading method is provided, applied to a consumer end of a distributed system, wherein the consumer end corresponds to a producer end, and the producer end generates multiple messages, including: obtaining a target message to be processed in a message queue and determining a target server cluster corresponding to the target message, wherein the target message carries a business application form, the business application form being used to determine multiple pieces of business data to be read, and the target server cluster being located in a first component service layer; generating a business data reading request when none of the servers in the target server cluster can query the business data; sending the business data reading request to a second component service layer, wherein the second component service layer is used to invoke a distributed routing strategy, the distributed routing strategy being used to determine a server cluster to be connected to store the business data; and distributing the business data reading request to the server cluster to be connected to read the target business data in the server cluster to be connected to.

[0007] Optionally, after sending the business data read request to the second component service layer, the method further includes: determining a routing factor based on the business data read request; inputting the routing factor into a preset hash function to calculate the hash value, wherein each hash value corresponds one-to-one with the cluster number of the server cluster; and matching the server cluster to be connected based on the hash value.

[0008] Optionally, the step of distributing the business data read request to the server cluster to be connected and reading the target business data in the server cluster to be connected includes: the second component service layer accessing the service registry to obtain a service cluster list and a distributed routing policy, wherein the service cluster list includes all server clusters in the first component service layer and multiple servers in each server cluster; filtering the multiple servers in the server cluster to be connected based on the distributed routing policy to obtain the target server; and forwarding the business data read request to the target server to read the target business data in the target server.

[0009] Optionally, the multiple servers in the server cluster include: a database server, an online application server, and a batch application server.

[0010] Optionally, the step of obtaining a target message to be processed in the message queue and determining the target server cluster corresponding to the target message includes: controlling the batch application server in each server cluster in the first component service layer to obtain the message to be processed in the message queue; when the message to be processed is obtained, increasing the message offset in the message queue by a preset value unit to isolate other server clusters besides the target server cluster from competing for the message to be processed; taking the message to be processed as the target message, and determining the target server cluster corresponding to the target message.

[0011] Optionally, the business data reading method further includes: splitting the business system into the first component service layer, the first component access layer, the second component service layer, the second component access layer, the common layer, and the routing layer based on preset application layer division requirements.

[0012] Optionally, the step of sending the business data read request to the second component service layer includes: calling an online application interface, wherein the first component service layer to which the multiple server clusters belong and the second component service layer pre-deploy the online application interface that interfaces with each other; and using the online application interface to send the business data read request to the second component service layer.

[0013] According to another aspect of the present invention, a business data reading device is also provided, applied to the consumer end of a distributed system, wherein the consumer end corresponds to the producer end, and the producer end generates multiple messages, including: an acquisition unit, configured to acquire a target message to be processed in a message queue and determine a target server cluster corresponding to the target message, wherein the target message carries a business application form, the business application form being used to determine multiple pieces of business data to be read, and the target server cluster being located in a first component service layer; a generation unit, configured to generate a business data reading request when the servers in the target server cluster cannot query the business data; a sending unit, configured to send the business data reading request to a second component service layer, wherein the second component service layer is used to invoke a distributed routing strategy, the distributed routing strategy being used to determine a server cluster to be connected to store the business data; and a distribution unit, configured to distribute the business data reading request to the server cluster to be connected to read the target business data in the server cluster to be connected to.

[0014] Optionally, the business data reading device further includes: a first determining module, used to determine a routing factor based on the business data reading request; a first calculating module, used to input the routing factor into a preset hash function to calculate the hashed hash value, wherein each hash value corresponds one-to-one with the cluster number of the server cluster; and a first matching module, used to match the server cluster to be connected based on the hashed hash value.

[0015] Optionally, the distribution unit includes: a first access module, used by the second component service layer to access the service registry center to obtain a service cluster list and a distributed routing policy, wherein the service cluster list includes all server clusters in the first component service layer and multiple servers in each server cluster; a first filtering module, used to filter the multiple servers in the server cluster to be connected based on the distributed routing policy to obtain a target server; and a first forwarding module, used to forward the business data reading request to the target server and read the target business data in the target server.

[0016] Optionally, the multiple servers in the server cluster include: a database server, an online application server, and a batch application server.

[0017] Optionally, the acquisition unit includes: a first control module, configured to control the batch application servers in each server cluster of the first component service layer to acquire the messages to be processed in the message queue; a first increment module, configured to increment the message offset in the message queue by a preset value unit when the messages to be processed are acquired, so as to isolate other server clusters besides the target server cluster from competing for the messages to be processed; and a first determination module, configured to take the messages to be processed as the target messages and determine the target server cluster corresponding to the target messages.

[0018] Optionally, the business data reading device further includes: a first splitting module, used to split the business system into a first component service layer, a first component access layer, a second component service layer, a second component access layer, a common layer, and a routing layer based on preset application layer division requirements.

[0019] Optionally, the sending unit includes: a first invocation module, used to invoke an online application programming interface (API), wherein the first component service layer to which the plurality of server clusters belong and the second component service layer pre-deploy mutually interfacing APIs; and a first sending module, used to send the business data reading request to the second component service layer using the online application programming interface.

[0020] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform any of the above-described business data reading methods.

[0021] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any of the above-described business data reading methods.

[0022] In this disclosure, the target message to be processed in the message queue is first obtained, and the target server cluster corresponding to the target message is determined. Then, if the servers in the target server cluster cannot query the business data, a business data read request is generated and sent to the second component service layer. Finally, the business data read request is distributed to the server cluster to be connected to read the target business data in the server cluster to be connected to. The target message carries a business application form, which is used to determine multiple pieces of business data to be read. The target server cluster is located in the first component service layer. The second component service layer is used to call the distributed routing strategy, which is used to determine the server cluster to be connected to store the business data.

[0023] In this disclosure, for situations where the server processing the message and the server containing the data carried in the business application are not in the same cluster, the data storage location is quickly and accurately located based on the business read request and distributed routing strategy, enabling precise reading and writing of distributed database data. It can handle messages and data under different server clusters, solving the technical problem in related technologies where distributed system server clusters cannot directly read and process the relevant data in the message after obtaining it.

[0024] This disclosure also enables the calculation of hash values ​​based on hash functions, thereby determining the server cluster where the target data is stored. It features low computational complexity, low collision rate, and uniform hashing, achieving uniform hashing of data and effectively solving the problem of data reading and writing in high-concurrency distributed systems. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0026] Figure 1 This is a flowchart of an optional business data reading method according to an embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of an optional business system hierarchy according to an embodiment of the present invention;

[0028] Figure 3 This is a flowchart of another optional business data reading method according to an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of another optional distributed service framework according to an embodiment of the present invention;

[0030] Figure 5 This is a schematic diagram of an optional business data reading device according to an embodiment of the present invention;

[0031] Figure 6 This is a hardware structure block diagram of an electronic device (or mobile device) for a business data reading method according to an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] To facilitate understanding of the present invention by those skilled in the art, some terms or nouns involved in the various embodiments of the present invention are explained below:

[0035] The distributed publish-subscribe messaging system, or KAFKA for short, enables seamless connection between producers and consumers and can handle all action stream data of consumers on the website.

[0036] ZooKeeper, or distributed application coordination service for short, encapsulates complex and error-prone critical services, providing users with easy-to-use interfaces and a high-performance, stable system.

[0037] Remote Procedure Call (RPC) is a type of remote procedure call.

[0038] DFS gateways enable network interconnection at the network layer and above, and are used for interconnection between two networks with different high-level protocols.

[0039] Dynamic Source Routing (DSR) is a routing protocol used in mobile ad hoc networks.

[0040] It should be noted that the business data reading method and apparatus in this disclosure can be used in the financial technology field for reading and processing data in a distributed system, and can also be used in any field other than the financial technology field for reading and processing data in a distributed system. The application field of the business data reading method and apparatus in this disclosure is not limited.

[0041] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent information from the aforementioned user or organization.

[0042] This invention can be applied to various distributed business systems / devices / appliances / products, as well as various distributed data reading devices / systems. Through a distributed routing strategy, the server cluster number for business data storage is determined based on the hash result, and the server receiving the online request is quickly located, thereby allowing the data carried by the online server to be read and processed.

[0043] The present invention will now be described in detail with reference to various embodiments.

[0044] Example 1

[0045] According to an embodiment of the present invention, a business data reading method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0046] The business data reading method is applied to the consumer end of a distributed system. The consumer end corresponds to the producer end, and the producer end generates multiple messages.

[0047] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of an optional business data reading method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0048] Step S101: Obtain the target message to be processed in the message queue and determine the target server cluster corresponding to the target message. The target message carries a business application form, which is used to determine multiple business data to be read. The target server cluster is located in the first component service layer.

[0049] Step S102: If the business data cannot be found on any of the servers in the target server cluster, a business data read request is generated.

[0050] Step S103: Send the business data read request to the second component service layer, wherein the second component service layer is used to call the distributed routing strategy, and the distributed routing strategy is used to determine the cluster of servers to be connected to store the business data.

[0051] Step S104: Distribute the business data read request to the cluster of servers to be connected, and read the target business data in the cluster of servers to be connected.

[0052] Through the above steps, the target message to be processed in the message queue is first obtained, and the target server cluster corresponding to the target message is determined. Then, if the servers in the target server cluster cannot find the business data, a business data read request is generated. The business data read request is sent to the second component service layer, where the second component service layer is used to call the distributed routing strategy, which is used to determine the server cluster to be connected to store the business data. Finally, the business data read request is distributed to the server cluster to be connected to read the target business data in the server cluster. In this embodiment, for the case where the server processing the message and the server to which the data carried by the business application belong are not in the same cluster, the data storage location is quickly and accurately located based on the business read request and the distributed routing strategy, realizing accurate reading and writing of distributed database data. This solves the technical problem in related technologies where the distributed system server cluster cannot directly read and process the relevant data in the message after obtaining it.

[0053] The following section will explain in detail each of the above implementation steps.

[0054] It should be noted that, based on the preset application layer division requirements, this embodiment divides the business system into a first component service layer, a first component access layer, a second component service layer, a second component access layer, a common layer, and a routing layer.

[0055] It should be noted that business systems can be divided into application service integration layer project groups, application business component layer project groups, and application public service project groups, with different project groups undertaking different business functions.

[0056] Optional, Figure 2 This is a schematic diagram of an optional business system hierarchy according to an embodiment of the present invention, such as... Figure 2As shown, the business system can be further divided into different business layers based on the application layer. Each business layer provides different services. The business layers include: a first component service layer, a first component access layer, a second component service layer, a second component access layer, a common layer, and a routing layer. Each business layer has one or more server clusters. Figure 2 The first component is represented by clusters (e.g., SET1, SET2). The second component access layer consists of two clusters, SET1 and SET2, used to provide unrouted services to channel applications. Business servers send call requests. SET1 in the second component access layer receives RPC requests distributed by the DFS gateway and can access the second component service layer and the common layer, thus responding to call requests from the receiving business system. The second component service layer also consists of two clusters, SET1 and SET2. Online application servers within these clusters can receive RPC requests from the second component access layer and access the clusters and routing layer of the first component service layer, while also interacting with the host through the DSR gateway. The first component access layer also consists of two clusters, SET1 and SET2, used to provide unrouted services to other distributed platform applications. The first component service layer contains multiple server clusters, each deploying multiple servers providing different services. The second component service layer is used to call one or more services of the first component layer and can also receive remote call requests from other business layers. The common layer and routing layer each contain a cluster SET, used to access common parameters and routing information, respectively.

[0057] Alternatively, each server cluster may include multiple servers, including: a database server, an online application server, and a batch application server.

[0058] like Figure 2 As shown, each server cluster has its own database server, online application server, and batch application server. Each layer of database server stores logs, parameters, and other data for its respective layer. The system's business data is stored in the first component service layer. The database server, online application server, and batch application server are deployed in multiple clusters for horizontal partitioning. The online application server and batch application server can access the database server in their respective clusters, thereby realizing distributed system database sharding and table partitioning.

[0059] Optionally, before reading and processing business data, the multiple server clusters of the first component service layer are numbered sequentially, and the service list and routing policy parameters of the containers corresponding to the online application servers in each cluster are registered with the service registry center.

[0060] Step S101: Obtain the target message to be processed in the message queue and determine the target server cluster corresponding to the target message.

[0061] This invention is based on the distributed publish-subscribe messaging system Kafka. It collects and stores data from sharded databases and tables, and distributes the data to consumers via message queues. The Kafka messaging system has two roles: producers (corresponding to the production end) and consumers (corresponding to the consumption end). Producers create messages and distribute them evenly across all partitions for consumers to read. Consumers read messages; they can subscribe to one or more message partitions and read messages in the order they were generated. Consumers distinguish read messages by their offsets. Within a given partition, each message offset is unique. Consumers store the offset of the last message read from each partition in ZooKeeper. This stored information is not lost if the consumer is shut down or restarted.

[0062] It should be noted that in the KAFKA distributed publish-subscribe messaging system, after a producer produces a message, it randomly pushes the message to the message queue of a certain partition. The message queue of the corresponding partition is incremented by a preset number of units, waiting for consumers to consume it. Since the first component service layer has multiple server clusters, there will be multiple batch application servers deployed in the clusters. Consumer groups are configured for the server clusters, that is, the first component service layer is configured with multiple consumer groups. Each consumer group randomly consumes data from the message queue, and the consumer group that wins the message is used as the target server cluster.

[0063] Optionally, step S101 includes: controlling the batch application servers in each server cluster in the first component service layer to obtain messages to be processed in the message queue; when a message to be processed is obtained, increasing the message offset in the message queue by a preset value unit to isolate other server clusters besides the target server cluster from competing for the message to be processed; taking the message to be processed as the target message, and determining the target server cluster corresponding to the target message.

[0064] It should be noted that when a consumer group retrieves a message in the message queue, it means that it has won the bid to process the message. At this time, the message offset can be increased by a preset value unit (for example, offset by one or two units) to lock the message, so that other consumer groups cannot retrieve the message.

[0065] Step S102: If the business data cannot be found on any of the servers in the target server cluster, a business data read request is generated.

[0066] It should be noted that after the target server cluster receives the target message, it needs to verify the message to confirm whether the business application data corresponding to the message is stored in the database server of the target server cluster. If it exists, the data carried in the message can be read directly and processed. If the business application data corresponding to the message does not exist in the database server of the target server cluster, since the batch application servers in the target server cluster can only access the database server of their own cluster, the consumer group cannot process the data corresponding to the message after consuming the message, which will cause the business application data to be blocked. For example, a message in the message queue is consumed by server cluster SET1 and the message offset is increased by a preset value, but the business application corresponding to the message is stored in the data of SET8, causing the message to be consumed but the business application to be blocked.

[0067] Optionally, if the database server in the target server cluster cannot find the target business data, a corresponding business data read request is generated.

[0068] Step S103: Send the business data read request to the second component service layer.

[0069] In this embodiment, step S103 includes: calling an online application programming interface (API), wherein the first component service layer and the second component service layer to which multiple server clusters belong pre-deployed an API that interfaces with each other; and using the API to send a business data read request to the second component service layer.

[0070] It should be noted that, in this embodiment of the invention, an online application programming interface (API) is pre-deployed in the first component service layer and the second component service layer. This API is used to store and read business data. Based on the API, the operation of the database server is controlled by the online application, thereby controlling the reading of the database. The business data reading request is sent to the second component service layer. Based on the request, the distributed routing strategy is invoked to locate the server cluster where the business data is located and read the business data from the database server in the cluster.

[0071] Optionally, after sending the business data read request to the second component service layer, the method further includes: determining a routing factor based on the business data read request; inputting the routing factor into a preset hash function to calculate the hash value, wherein each hash value corresponds one-to-one with the cluster number of the server cluster; and matching the server cluster to be connected based on the hash value.

[0072] It should be noted that the embodiments of the present invention use a hash function to calculate the hash value after hashing. Each hash value corresponds to a server cluster, and each hash value corresponds to a cluster number. The target business data is matched to the server cluster to be connected by the hash value.

[0073] Step S104: Distribute the business data read request to the cluster of servers to be connected, and read the target business data in the cluster of servers to be connected.

[0074] Optionally, step S104 includes: the second component service layer accessing the service registry to obtain a service cluster list and a distributed routing policy, wherein the service cluster list contains all server clusters in the first component service layer and multiple servers in each server cluster; filtering the multiple servers contained in the server cluster to be connected based on the distributed routing policy to obtain the target server; forwarding the business data reading request to the target server and reading the target business data in the target server.

[0075] It should be noted that after matching the server cluster corresponding to the target business data, the business data read request is sent to the server cluster to be connected. The second service component accesses the registration service center to obtain the list of service clusters and routing policies pre-registered for the server cluster to be connected. Based on the distributed routing policy, multiple servers in the multi-service list are filtered to determine the container where the target business data is located, and the business data read request is forwarded to the corresponding target server to read the business data.

[0076] Through the above embodiments, a distributed routing strategy is configured for the distributed business system. The hash value is calculated based on the hash function, and the server cluster number of the business data storage is determined by the hash result. The server receiving the online request is quickly located, and the data carried by the online server is read and processed.

[0077] The present invention will now be described with reference to a specific embodiment.

[0078] Figure 3 This is a flowchart of another optional business data reading method according to an embodiment of the present invention, such as... Figure 3 As shown, the method includes the following steps:

[0079] Step 1: The target batch application server competes for messages from the message queue of the KAFKA message service group;

[0080] The business system is divided into the first component service layer, the first component access layer, the second component service layer, the second component access layer, the common layer, and the routing layer.

[0081] The business system can be further divided into different business layers based on the application layer. Each business layer provides different services. The business layers include: the first component service layer, the first component access layer, the second component service layer, the second component access layer, the common layer, and the routing layer. Each business layer has one or more server clusters (represented as clusters in the diagram). The first component access layer is used to provide unrouted services to channel applications, such as receiving remote call requests from other software or systems. The first component service layer contains multiple server clusters, each with multiple servers providing different services. The second component service layer is used to call one or more services of the first component layer and can also receive remote call requests from other business layers. The second component access layer is used to provide unrouted services to other distributed platform applications. The common layer and the routing layer are used to access common parameters and routing information, respectively.

[0082] Each server cluster consists of multiple servers, including: database servers, online application servers, and batch application servers.

[0083] After the Kafka messaging system evenly distributes the messages created by producers to all partitions for consumers to read, multiple server clusters in the first component service layer compete for the target message in the message queue. The server cluster that wins the message is designated as the target server cluster. Figure 3 In this context, cluster SET1 is used as the target server cluster, and in the first component service layer, there are multiple clusters SET2, ..., SETn.

[0084] Step 2: In the target batch application, use RPC calls to invoke the encapsulated online application interface;

[0085] If the data corresponding to the message is not in the target service cluster, the batch applications in the target server cluster call the pre-configured first component service layer online application interface and send a business data read request to the second component service layer.

[0086] Step 3: After receiving the request, the second component service layer forwards the request to the public layer and invokes the distributed routing strategy;

[0087] Based on the online application programming interface (API), the operation of the database server is controlled through online application, thereby controlling the reading of the database. The business data reading request is sent to the second component service layer. Based on the request, the distributed routing strategy is invoked to locate the server cluster where the business data is located, and the business data is read from the database server in that cluster.

[0088] Step 4: Distribute the request to the first component service layer container corresponding to the SET hashed by the hash function according to the distributed routing strategy, and read the data.

[0089] After receiving a business data read request, the second component service layer sends a request to the service registry to obtain the list of service providers and routing policies, retrieves the routing factor in the request message, inputs a hash function to calculate the hash value after hashing, and matches it with the corresponding SET number to determine the server cluster to be connected.

[0090] After matching the server cluster corresponding to the target business data, the business data read request is sent to the server cluster to be connected. The second service component accesses the registration service center to obtain the list of service clusters and routing policies pre-registered for the server cluster to be connected. Based on the distributed routing policy, multiple servers in the multi-service list are filtered to determine the container where the target business data is located, and the business data read request is forwarded to the corresponding target server to read the business data.

[0091] The registration service center is used to connect the first component service layer and the second component service layer. Figure 4 This is a schematic diagram of another optional distributed service framework according to an embodiment of the present invention, such as... Figure 4 As shown, the service registry includes the ZooKeeper application server. The online application server in the first component service layer registers services with the ZooKeeper application server, registering the service list and routing policy parameters of the corresponding container with the service registry. The second component service layer can access the service registry, subscribe to services, and call the services provided by the first component composition layer based on the service route. The service route contains the routing parameters of the second component service layer. The corresponding hash value is calculated based on the routing field. Based on the hash value, the cluster SET to be connected can be obtained.

[0092] Through the above embodiments, a distributed routing strategy is configured for the distributed business system. The hash value is calculated based on the hash function, and the server cluster number of the business data storage is determined by the hash result. The server receiving the online request is quickly located, and the data carried by the online server is read and processed.

[0093] The invention will now be described in conjunction with another alternative embodiment.

[0094] Example 2

[0095] This embodiment provides a business data reading device, wherein each implementation unit included in the business data reading device corresponds to each implementation step in the above embodiment one.

[0096] The business data reading device is applied to the consumer end of the distributed system. The consumer end corresponds to the producer end, and the producer end generates multiple messages.

[0097] Figure 5This is a schematic diagram of an optional business data reading device according to an embodiment of the present invention, such as... Figure 5 As shown, the business data reading device includes: an acquisition unit 51, a generation unit 52, a sending unit 53, and a distribution unit 54, wherein...

[0098] The acquisition unit 51 is used to acquire the target message to be processed in the message queue and determine the target server cluster corresponding to the target message. The target message carries a business application form, which is used to determine multiple business data to be read. The target server cluster is located in the first component service layer.

[0099] The generation unit 52 is used to generate a business data read request when the business data cannot be found on any of the servers in the target server cluster.

[0100] Sending unit 53 is used to send a business data reading request to the second component service layer, wherein the second component service layer is used to call the distributed routing strategy, and the distributed routing strategy is used to determine the cluster of servers to be connected to store the business data.

[0101] The distribution unit 54 is used to distribute business data reading requests to the cluster of servers to be connected, and read the target business data in the cluster of servers to be connected.

[0102] The aforementioned business data reading device acquires the target message to be processed in the message queue through the acquisition unit 51 and determines the target server cluster corresponding to the target message; generates a business data reading request through the generation unit 52 when the business data cannot be found on each server in the target server cluster; sends the business data reading request to the second component service layer through the sending unit 53; and distributes the business data reading request to the server cluster to be connected through the distribution unit 54 to read the target business data in the server cluster to be connected.

[0103] In this embodiment, for cases where the server that receives the message and the server to which the data carried in the business application form belong are not in the same cluster, the data storage location is quickly and accurately located based on the business read request and the distributed routing strategy, thereby achieving precise reading and writing of distributed database data. This solves the technical problem in related technologies where the distributed system server cluster cannot directly read and process the relevant data in the message after receiving it.

[0104] Optionally, the business data reading device further includes: a first determining module, used to determine a routing factor based on the business data reading request; a first calculating module, used to input the routing factor into a preset hash function to calculate the hash value after hashing, wherein each hash value corresponds one-to-one with the cluster number of the server cluster; and a first matching module, used to match the server cluster to be connected based on the hash value after hashing.

[0105] Optionally, the distribution unit includes: a first access module, used by the second component service layer to access the service registry center to obtain a service cluster list and a distributed routing policy, wherein the service cluster list contains all server clusters in the first component service layer and multiple servers in each server cluster; a first filtering module, used to filter the multiple servers contained in the online server cluster based on the distributed routing policy to obtain the target server; and a first forwarding module, used to forward the business data reading request to the target server and read the target business data in the target server.

[0106] Optionally, the multiple servers in the server cluster include: database servers, online application servers, and batch application servers.

[0107] Optionally, the acquisition unit includes: a first control module, used to control the batch application servers in each server cluster in the first component service layer to acquire messages to be processed in the message queue; a first addition module, used to add a preset value unit to the message offset in the message queue when a message to be processed is acquired, so as to isolate other server clusters except the target server cluster from competing for the message to be processed; and a first determination module, used to take the message to be processed as the target message and determine the target server cluster corresponding to the target message.

[0108] Optionally, the business data reading device further includes: a first splitting module, used to split the business system into a first component service layer, a first component access layer, a second component service layer, a second component access layer, a common layer, and a routing layer based on preset application layer division requirements.

[0109] Optionally, the sending unit includes: a first calling module for calling an online application programming interface (API), wherein the first component service layer and the second component service layer to which multiple server clusters belong pre-deployed an API that interfaces with each other; and a first sending module for sending a business data reading request to the second component service layer using the API.

[0110] The aforementioned business data reading device may also include a processor and a memory. The aforementioned acquisition unit 51, generation unit 52, sending unit 53, distribution unit 54, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0111] The processors described above contain a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and their parameters can be adjusted to read and process data in the distributed system.

[0112] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0113] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute any of the above-described business data reading methods.

[0114] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors cause the one or more processors to implement any of the above-described business data reading methods.

[0115] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: obtaining a target message to be processed in a message queue and determining the target server cluster corresponding to the target message; generating a business data read request when the servers in the target server cluster cannot query the business data; sending the business data read request to the second component service layer; and distributing the business data read request to the server cluster to be connected, and reading the target business data in the server cluster to be connected.

[0116] Figure 6 This is a hardware structure block diagram of an electronic device (or mobile device) according to an embodiment of the present invention for a service data reading method. Figure 6 As shown, the electronic device may include one or more processors 602 (shown as 602a, 602b, ..., 602n in the figure) 602 (processor 602 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 604 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 6 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 6 The more or fewer components shown, or having the same Figure 6 The different configurations shown.

[0117] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0118] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0120] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0121] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0122] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0123] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A service data reading method characterized by comprising: The application is applied to a consumer of a distributed system, the consumer corresponds to a producer, the producer generates a plurality of messages, including: Obtaining a target message to be processed in a message queue and determining a target server cluster corresponding to the target message, wherein the target message carries a business application form, the business application form is used to determine a plurality of business data to be read, and the target server cluster is located in a first component service layer; In a case where each server in the target server cluster cannot query the business data, generating a business data reading request; Sending the business data reading request to a second component service layer, wherein the second component service layer is used to call a distributed routing strategy, and the distributed routing strategy is used to determine a to-be-online server cluster storing the business data; Distributing the business data reading request to the to-be-online server cluster and reading target business data in the to-be-online server cluster, including: the second component service layer accesses a service registration center to obtain a service cluster list and a distributed routing strategy, wherein the service cluster list contains all server clusters in the first component service layer and a plurality of servers in each server cluster, the plurality of servers in the server cluster include a database server, an online application server and a batch application server; based on the distributed routing strategy, the plurality of servers contained in the to-be-online server cluster are screened to obtain a target server; and the business data reading request is forwarded to the target server to read the target business data in the target server; The step of obtaining a target message to be processed in a message queue and determining a target server cluster corresponding to the target message includes: controlling a batch application server in each server cluster in the first component service layer to obtain a to-be-processed message in the message queue; in a case where the to-be-processed message is obtained, increasing a message offset in the message queue by a preset numerical unit to isolate other server clusters except the target server cluster from competing for the to-be-processed message; the to-be-processed message is taken as the target message, and the target server cluster corresponding to the target message is determined.

2. The method of claim 1, wherein, After the business data reading request is sent to the second component service layer, the method further includes: Determining a routing factor based on the business data reading request; Inputting the routing factor into a preset hash function to calculate a hash value after hashing, wherein each hash value corresponds to a cluster number of a server cluster in a one-to-one manner; Based on the hash value after hashing, the to-be-online server cluster is matched.

3. The method according to any one of claims 1 to 2, characterized in that, Further comprising: Based on a preset application level division requirement, the business system is divided into the first component service layer, the first component access layer, the second component service layer, the second component access layer, the public layer and the routing layer.

4. The method of claim 1, wherein, The step of sending the business data reading request to the second component service layer includes: Calling an online application program interface, wherein the first component service layer to which a plurality of server clusters belong and the second component service layer are pre-deployed to interface with each other through the online application program interface; The business data read request is sent to a second component service layer using the online application program interface.

5. A service data reading apparatus characterized by comprising: The consumer is applied to a distributed system, and the consumer corresponds to a producer. An obtaining unit is configured to obtain a target message to be processed in a message queue, and determine a target server cluster corresponding to the target message, wherein the target message carries a business application form, the business application form is used to determine multiple pieces of business data to be read, and the target server cluster is located in a first component service layer. A generating unit is configured to generate a business data read request in a case where each server in the target server cluster cannot query the business data. A sending unit is configured to send the business data read request to a second component service layer, wherein the second component service layer is configured to call a distributed routing strategy, and the distributed routing strategy is configured to determine a to-be-connected server cluster storing the business data. A distributing unit is configured to distribute the business data read request to the to-be-connected server cluster, and read target business data in the to-be-connected server cluster. The distributing unit includes a first access module configured to access a service registration center by the second component service layer to obtain a service cluster list and a distributed routing strategy, wherein the service cluster list includes all server clusters in the first component service layer and multiple servers in each server cluster, and the multiple servers in the server cluster include a database server, an online application server, and a batch application server; a first screening module configured to screen the multiple servers included in the to-be-connected server cluster based on the distributed routing strategy to obtain a target server; and a first forwarding module configured to forward the business data read request to the target server to read the target business data in the target server. The obtaining unit includes a first control module configured to control a batch application server in each server cluster in the first component service layer to obtain a to-be-processed message in the message queue; a first increasing module configured to increase a message offset in the message queue by a preset numerical unit to isolate other server clusters except the target server cluster from competing for the to-be-processed message in a case where the to-be-processed message is obtained; and a first determination module configured to take the to-be-processed message as the target message, and determine the target server cluster corresponding to the target message.

6. A computer-readable storage medium, characterized in that, A computer readable storage medium includes a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to perform the business data read method in any one of claims 1 to 4 when the computer program runs.

7. An electronic device, comprising: The device includes one or more processors and a memory, and the memory is configured to store one or more programs, wherein the one or more programs make the one or more processors implement the business data read method in any one of claims 1 to 4 when the one or more programs are executed by the one or more processors.

Images