Chain-of-responsibility-based data processing method and device, and computer device

Abstract

The embodiment of the application belongs to the field of cloud technology, and relates to a data processing method and device based on a chain of responsibility, computer equipment and a storage medium. The method comprises the following steps: obtaining to-be-processed data; based on the to-be-processed data, obtaining data processing evaluation information of each node in a node network; selecting a target number of candidate nodes from the node network; calculating a data processing evaluation value of each candidate node according to the data processing evaluation information of each candidate node; when the data processing evaluation value of each candidate node meets a preset evaluation value condition, determining each candidate node as a chain-of-responsibility node; and performing data processing on the to-be-processed data through a chain of responsibility formed by the chain-of-responsibility nodes. In addition, the application also relates to blockchain technology, and the to-be-processed data can be stored in a blockchain. The application improves the accuracy of data processing based on a chain of responsibility.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular to a data processing method, apparatus and computer equipment based on a chain of responsibility. Background Technology

[0002] With the development of computer technology, business data is often processed through cluster networks. For example, cluster networks are used to process business data in finance, securities, e-commerce, and other fields. A cluster network can contain multiple nodes, from which nodes can be selected to form a chain of responsibility. The chain of responsibility pattern is used to process and store business data.

[0003] The Chain of Responsibility pattern is a design pattern in which many nodes are connected by references to their successors, forming a chain along which requests can be passed. However, current Chain of Responsibility technologies typically pre-configured the chain, processing incoming business data directly according to this pre-defined chain. However, a pre-configured chain may not be suitable for processing certain types of business data, leading to lower accuracy in data processing. Summary of the Invention

[0004] The purpose of this application is to provide a data processing method, apparatus, computer equipment, and storage medium based on the chain of responsibility, so as to improve the accuracy of data processing based on the chain of responsibility.

[0005] To address the aforementioned technical problems, this application provides a data processing method based on a chain of responsibility, employing the following technical solution:

[0006] Obtain the data to be processed;

[0007] Based on the data to be processed, obtain the data processing evaluation information of each node in the node network;

[0008] Select a target number of candidate nodes from the node network;

[0009] Calculate the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node;

[0010] When the data processing evaluation value of each candidate node meets the preset evaluation value conditions, each candidate node is determined as a chain of responsibility node;

[0011] The data to be processed is processed through a chain of responsibility formed by the nodes of the chain of responsibility.

[0012] To address the aforementioned technical problems, this application also provides a data processing device based on a chain of responsibility, employing the following technical solution:

[0013] The data acquisition module is used to acquire the data to be processed.

[0014] The information acquisition module is used to acquire data processing evaluation information of each node in the node network based on the data to be processed;

[0015] A node selection module is used to select a target number of candidate nodes from the node network;

[0016] The evaluation value calculation module is used to calculate the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node.

[0017] The node determination module is used to determine each candidate node as a chain of responsibility node when the data processing evaluation value of each candidate node meets the preset evaluation value conditions.

[0018] The data processing module is used to process the data to be processed through the chain of responsibility formed by the nodes of the chain of responsibility.

[0019] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0020] Obtain the data to be processed;

[0021] Based on the data to be processed, obtain the data processing evaluation information of each node in the node network;

[0022] Select a target number of candidate nodes from the node network;

[0023] Calculate the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node;

[0024] When the data processing evaluation value of each candidate node meets the preset evaluation value conditions, each candidate node is determined as a chain of responsibility node;

[0025] The data to be processed is processed through a chain of responsibility formed by the nodes of the chain of responsibility.

[0026] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0027] Obtain the data to be processed;

[0028] Based on the data to be processed, obtain the data processing evaluation information of each node in the node network;

[0029] Select a target number of candidate nodes from the node network;

[0030] Calculate the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node;

[0031] When the data processing evaluation value of each candidate node meets the preset evaluation value conditions, each candidate node is determined as a chain of responsibility node;

[0032] The data to be processed is processed through a chain of responsibility formed by the nodes of the chain of responsibility.

[0033] Compared with the prior art, the embodiments of this application have the following main advantages: Acquiring data to be processed, and based on the data to be processed, acquiring data processing evaluation information of each node in the node network, the data processing evaluation information is used to record the historical processing status of nodes for a certain type of business data; selecting a target number of candidate nodes from the node network, and calculating the data processing evaluation value of each candidate node according to its data processing evaluation information, to determine the data processing performance and suitability of the candidate node for the data to be processed; when the data processing evaluation value of each candidate node meets the preset evaluation value conditions, it indicates that each candidate node meets the data processing requirements and can be used as a chain of responsibility node; processing the data to be processed through the chain of responsibility formed by each chain of responsibility node; this application improves the performance and accuracy of data processing based on the chain of responsibility by calculating the data processing evaluation value and selecting nodes suitable for processing the data to be processed to form a chain of responsibility. Attached Figure Description

[0034] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0036] Figure 2 This is a flowchart of an embodiment of the chain-of-responsibility-based data processing method according to this application;

[0037] Figure 3 This is a schematic diagram of a structure of an embodiment of the chain-of-responsibility-based data processing apparatus according to this application;

[0038] Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0040] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0041] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0042] like Figure 1 As shown, system architecture 100 may include terminal devices 101 and 102, server 103, data processing servers 104, 105, and 106, and network 107. Network 107 serves as the medium for providing communication links between terminal devices 101 and 102, server 103, and data processing servers 104, 105, and 106. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0043] Users can use terminal devices 101 and 102 to interact with server 103 via network 107 to receive or send messages, etc. For example, terminals 101 and 102 can generate business-related data to be processed and send the data to server 103. Various communication client applications can be installed on terminal devices 101 and 102, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0044] Terminal devices 101 and 102 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0045] Server 103 and data processing servers 104, 105, and 106 form a distributed node network, where servers 103 and 104, 105, and 106 can act as nodes in the network. Server 103 can play a control role, selecting data processing servers to form a chain of responsibility and instructing them to perform data processing, such as data cleaning. In one embodiment, server 103 can be a server selected from the data processing servers, serving not only a control role but also forming a chain of responsibility for data processing.

[0046] It should be noted that the data processing method based on the chain of responsibility provided in this application embodiment is generally executed by a server, and correspondingly, the data processing device based on the chain of responsibility is generally located in the server.

[0047] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0048] Continue to refer to Figure 2 The diagram illustrates a flowchart of an embodiment of the chain-of-responsibility-based data processing method according to this application. The chain-of-responsibility-based data processing method includes the following steps:

[0049] Step S201: Obtain the data to be processed.

[0050] In this embodiment, the chain-of-responsibility-based data processing method operates on electronic devices (e.g., Figure 1 The server shown can communicate with the terminal via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other currently known or future wireless connection methods.

[0051] Specifically, the server first obtains the data to be processed, which can be business data. The server can act as a gateway node, through which the data to be processed enters the node network.

[0052] Step S202: Based on the data to be processed, obtain the data processing evaluation information of each node in the node network.

[0053] Specifically, the data to be processed can be business data. Based on the business type involved in the data to be processed, data processing evaluation information of each node in the node network can be obtained. The data processing evaluation information includes evaluation information of multiple dimensions, which is used to record the historical processing status of nodes for a certain type of business data.

[0054] Step S203: Select the target number of candidate nodes from the node network.

[0055] Specifically, a target number of candidate nodes are randomly selected from the node network, where the target number can be pre-defined. In one embodiment, the target number is determined based on the data volume of the data to be processed; where data volume refers to the size of the data file, which can be measured in computer storage units such as KB (kilobits), MB (megabits), and GB (gigabits).

[0056] Step S204: Calculate the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node.

[0057] Specifically, the data processing evaluation information of each selected candidate node is calculated according to a preset evaluation value calculation algorithm to obtain the data processing evaluation value of each candidate node. The data processing evaluation value can be a numerical value, and its magnitude is used to measure the data processing performance and adaptability of the candidate node to the data to be processed. The larger the data processing evaluation value, the more suitable the candidate node is for processing the data to be processed.

[0058] Step S205: When the data processing evaluation value of each candidate node meets the preset evaluation value conditions, each candidate node is determined as a chain of responsibility node.

[0059] Specifically, evaluation criteria can be preset, which can include single-dimensional criteria and multi-dimensional criteria. A single-dimensional criterion could be that the data processing evaluation value of each candidate node must be greater than a preset evaluation value threshold. Multi-dimensional criteria could be that the data processing evaluation values ​​of each candidate node must meet certain conditions, such as the data processing evaluation values ​​of each candidate node not differing too much, and their variance not exceeding a preset variance threshold.

[0060] If the data processing evaluation values ​​of each candidate node meet the evaluation value conditions, then each candidate node meets the requirements and is determined as a chain of responsibility node.

[0061] Step S206: Data processing is performed on the data to be processed through the chain of responsibility formed by each chain of responsibility node.

[0062] Specifically, a responsibility chain is constructed based on each node of the responsibility chain for the data to be processed. The responsibility chain processes the data to be processed to obtain processed data. The responsibility chain can perform various processing methods on the data to be processed, such as data cleaning, data aggregation, and data analysis.

[0063] In one embodiment, the nodes in the chain of responsibility have an order. The data processing evaluation values ​​of the chain of responsibility nodes are arranged in descending or ascending order, and this order serves as the order of the chain of responsibility nodes corresponding to those evaluation values. The data to be processed is transferred and circulated within the chain of responsibility according to the order of the nodes.

[0064] It should be emphasized that, in order to further ensure the privacy and security of the aforementioned data to be processed and the data already processed, the aforementioned data to be processed and the data already processed can also be stored in a node of a blockchain.

[0065] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0066] In this embodiment, data to be processed is acquired, and based on the data to be processed, data processing evaluation information of each node in the node network is acquired. The data processing evaluation information is used to record the historical processing status of nodes for a certain type of business data. A target number of candidate nodes are selected from the node network, and the data processing evaluation value of each candidate node is calculated according to its data processing evaluation information to determine the data processing performance and suitability of the candidate nodes for the data to be processed. When the data processing evaluation value of each candidate node meets the preset evaluation value conditions, it indicates that each candidate node meets the data processing requirements and can be used as a chain of responsibility node. The data to be processed is processed through the chain of responsibility formed by the chain of responsibility nodes. This application improves the performance and accuracy of data processing based on the chain of responsibility by calculating the data processing evaluation value to select nodes suitable for processing the data to be processed to form a chain of responsibility.

[0067] Furthermore, before step S201, the method may further include: obtaining historical processing information of each node in the node network at different time periods, wherein the historical processing information includes request basic information and request service type; and generating data processing evaluation information of each node at different time periods based on the historical processing information of each node at different time periods, wherein the data processing evaluation information includes multiple evaluation factors, including time period identifier, service type identifier, number of requests received, request processing duration, and service type correlation.

[0068] Specifically, historical processing information of each node in the node network at different time periods is obtained. Nodes process data according to received data processing requests in the past, and historical processing information can be obtained by recording the data processing process. Historical processing information can be related to time periods, and data processing at different time periods will produce different historical processing information.

[0069] Historical processing information can include basic request information and request business type. Nodes can process data to be processed based on received data processing requests. The data processing request includes the request business type, indicating the business type involved in the data to be processed. Basic request information includes the number of requests received, request processing time, and request processing evaluation information. Specifically, the number of requests received can be the number of requests received by the node under specific business type and time period conditions; the request processing time can be the response time of the node to requests under specific business type and time period conditions, i.e., the processing time of the data to be processed; and the request processing evaluation information can be the evaluation information of the accuracy of the processing results of the data to be processed under specific business type and time period conditions.

[0070] Based on the historical processing information of each node at different time periods, data processing evaluation information for each node at different time periods can be generated. This data processing evaluation information includes multiple evaluation factors, including time period identifier, business type identifier, number of requests received, request processing duration, and business type relevance. Specifically, the time period identifier in the data processing evaluation information represents different time periods; the business type identifier, number of requests received, and request processing duration have the same meaning as those in the historical processing information; the business type relevance indicates the adaptability of a node to a certain business type. The more data a node processes for a certain business type (related to the number of requests received), the faster it processes (related to the request processing duration), and the higher its accuracy (related to the request processing evaluation information), the higher the business type relevance.

[0071] Data processing assessment information can be divided into multiple groups based on time period identifiers and business type identifiers. The time period identifiers can also differ under different business types, meaning the division of time periods is related to the business type. For example, for the catering industry, pre-defined dining and non-dining times are used, with the dining time period being shorter than the non-dining time period; for instance, the dining time period might be 1 hour long, while the non-dining time period is 4 hours long. For the clothing industry, the time period can be 4 hours long for all periods. For the hotel industry, pre-defined peak and non-peak occupancy periods are used, with the peak occupancy period being shorter than the non-peak occupancy period. Dividing the time periods according to business type yields different time period identifiers, ensuring that the data processing assessment information is linked to the actual business type and improving the accuracy of the data processing assessment information.

[0072] In this embodiment, data processing evaluation information for each node is generated at different time periods. The data processing evaluation information includes various evaluation factors such as time period identifier, business type identifier, number of requests received, request processing duration, and business type correlation. It can evaluate the processing status of a node on business data of a certain business type from different perspectives.

[0073] Furthermore, step S201 may include: obtaining a service type identifier from the data to be processed; obtaining the arrival time of the data to be processed to determine the time period identifier corresponding to the arrival time of the data; and obtaining data processing evaluation information of each node in the node network based on the service type identifier and the time period identifier.

[0074] Specifically, after receiving the data to be processed, the server extracts the service type identifier from the data and obtains the arrival time of the data to the server, determining the time period identifier corresponding to that arrival time. Using the service type identifier and time period identifier as search criteria, the server retrieves the data processing evaluation information of each node in the node network.

[0075] In this example, data processing evaluation information is obtained based on the business type identifier and time period identifier of the data to be processed, so that the data processing evaluation information matches the data to be processed that has arrived, thus ensuring the accuracy of subsequent data processing evaluation value calculation.

[0076] Furthermore, step S204 may include: determining the factor weights of each evaluation factor in the data processing evaluation information of each candidate node according to a preset weighting strategy; calculating each evaluation factor according to the obtained factor weights to obtain the data processing evaluation value of each candidate node.

[0077] Specifically, the data processing evaluation information includes multiple evaluation factors, which may include time period identifiers, business type identifiers, number of requests received, request processing time, and business type relevance. The factor weights of each evaluation factor in the data processing evaluation information are determined through a preset weighting strategy. The weighting strategy can be either pre-setting the factor weights or determining the factor weights through a preset weighting algorithm. The weighting algorithm can be the CRITIC algorithm, the analytic hierarchy process (AHP), the Relief algorithm, etc.

[0078] Time period identifiers and business type identifiers are typically used as search criteria to query data processing evaluation information related to these identifiers. However, these identifiers are not usually used when calculating data processing evaluation values. Therefore, whether factor weights are pre-set or determined through a weighting algorithm, no weight is added to the time period identifier or business type identifier; instead, weights are added to the number of requests received, request processing time, and business type relevance.

[0079] Based on the obtained factor weights, each evaluation factor is weighted and calculated. Specifically, a weighted summation can be performed to obtain the data processing evaluation value of each candidate node.

[0080] In this embodiment, factor weights are added to each evaluation factor to distinguish and differentiate them, thereby ensuring the rationality of the calculated data processing evaluation values.

[0081] Furthermore, step S205 may include: calculating the mean of the data processing evaluation value of each candidate node; calculating the fluctuation value of the data processing evaluation value of each candidate node based on the mean; when each fluctuation value is less than a preset fluctuation value threshold, determining that the data processing evaluation value of each candidate node meets the preset evaluation value condition, and identifying each candidate node as a chain of responsibility node.

[0082] Specifically, the mean of the data processing evaluation value of each candidate node is calculated, and then the fluctuation value of the data processing evaluation value of each candidate node is calculated based on the mean. The fluctuation value reflects the fluctuation of the data processing evaluation value of the candidate node relative to the mean. In one embodiment, the difference can be obtained by subtracting the data processing evaluation value from the mean, and the absolute value of the difference can be divided by the mean to obtain the fluctuation value.

[0083] A fluctuation threshold can be preset. If all fluctuation values ​​are less than the fluctuation threshold, it indicates that the processing capabilities of each candidate node are relatively similar. When each candidate node processes the data, there will be no significant performance gap, which can ensure stable processing of the data. If the data processing evaluation value of each candidate node meets the preset evaluation value conditions, each candidate node will be identified as a chain of responsibility node.

[0084] In this example, the mean of the data processing evaluation value of each candidate node is calculated, and then the fluctuation value is calculated separately. When each fluctuation value is less than the preset fluctuation value threshold, it indicates that the data processing capabilities of each node are relatively balanced and meet the evaluation value conditions, thus obtaining the responsibility chain node.

[0085] Furthermore, the above-mentioned data processing method based on the chain of responsibility may also include: when there are candidate nodes whose data processing evaluation values ​​do not meet the preset evaluation value conditions, selecting complement nodes to replace the candidate nodes, wherein the complement nodes are nodes other than the candidate nodes in the node network; after the node replacement is completed, calculating the data processing evaluation value of each candidate node according to the data processing evaluation information of each candidate node.

[0086] Specifically, after selecting the target number of candidate nodes from the node network, the remaining nodes in the node network are the complement nodes. When there are cases where the data processing evaluation value does not meet the preset evaluation value conditions, a corresponding number of complement nodes are randomly selected based on the number of candidate nodes that do not meet the evaluation value conditions to replace the candidate nodes that do not meet the evaluation value conditions.

[0087] After the node replacement is completed, iterative calculations are performed based on the data processing evaluation information of each candidate node until the data processing evaluation value of each candidate node meets the evaluation value conditions. Then the iteration stops, and the chain of responsibility node is obtained.

[0088] In this embodiment, when there are candidate nodes whose data processing evaluation values ​​do not meet the evaluation value conditions, the complement node is selected to replace the candidate node and iterative calculation is performed in order to find a candidate node that meets the requirements.

[0089] Furthermore, after step S206, the method may further include: obtaining the data to be stored and its corresponding data volume, wherein the data to be stored includes data to be processed and processed data; obtaining storage evaluation information for each database, wherein the storage evaluation information includes multiple storage evaluation factors; calculating storage configuration information based on the data volume and storage evaluation information according to a preset partitioning storage algorithm, wherein the storage configuration information includes the selected databases and their corresponding required storage volumes; and partitioning the data to be stored and storing it in the databases according to the storage configuration information to obtain inventory data.

[0090] Here, "data to be stored" refers to data that needs to be stored. This data can be unprocessed data (i.e., unprocessed data can be stored directly) or processed data. Processed data can be the result of the chain of responsibility processing the unprocessed data. It can be understood that intermediate data generated during data processing can also be stored as "data to be stored."

[0091] Specifically, the server obtains the data to be stored and its corresponding data volume, and then obtains the storage evaluation information of each database. The database is used to store the data; the database can be a database in each node of the above-mentioned node network, or it can be a database independent of the above-mentioned node network.

[0092] Storage evaluation information measures the data storage performance of a database using storage evaluation factors across multiple dimensions. Table 1 shows the storage evaluation factors for each database in one embodiment. Specifically, referring to Table 1, the storage evaluation factors include CPU factor, memory factor, and storage factor. The CPU factor is a factor related to the CPU of the server hosting the database, the memory factor is a factor related to the memory of the server hosting the database, and the storage factor is a factor related to the storage of the server hosting the database. The CPU factor, memory factor, and storage factor can have preset weights. In the example in Table 1, the weight of the CPU factor is 35, the weight of the memory factor is 25, and the weight of the storage factor is 40.

[0093]

[0094]

[0095] Table 1

[0096] When calculating data volume and storage evaluation information according to the preset partitioning storage algorithm, the database is first selected. If the database satisfies m i If the value is greater than x, then this database will be selected as...

[0097] n

[0098] The database that stores the data to be stored, where m i Let x represent the storage factor, n represent the volume of the data to be stored, and n represent the number of distributed databases. Assuming the data volume is x = 80G and n = 4, then databases 1 and 3 in Table 1 meet the requirements. The data to be stored will be divided into two parts and stored in databases 1 and 3 respectively.

[0099] For database 1, its CPU parameter is the product of the CPU factor and its weight, i.e., 0.3 × 35 = 10.5; its memory parameter is the product of the memory factor and its weight, i.e., 0.5 × 25 = 12.5. Similarly, for database 3, its CPU parameter is 0.9 × 35 = 31.5, and its memory parameter is 3 × 25 = 75; the total parameters are 10.5 + 12.5 + 31.5 + 75 = 129.5. The required data volume for the database is calculated based on the CPU parameter, memory parameter, and the data volume to be stored. Specifically, the required storage volume for database 1 is ((10.5 + 12.5) / 129.5) × 80G = 14G, and the required storage volume for database 3 is ((31.5 + 75) / 129.5) × 80G = 66G.

[0100] The selected databases and their required storage volume constitute the storage configuration information. Based on the storage configuration information, the data to be stored is partitioned and stored in the databases to obtain the inventory data. Continuing with the example in Table 1 above, database 1 selects 14G of data from the data to be stored for storage, and database 3 selects 66G of data from the data to be stored for storage.

[0101] In one embodiment, the number of databases selected is the number of shards. This application can also introduce a dynamic pre-sharding factor, which is a numerical value, typically N times the number of shards (N is a positive integer). According to the example in Table 1, with 2 shards and assuming N = 10, the dynamic pre-sharding factor is 20. Therefore, during data partitioning, the data to be stored is first divided into 20 parts based on the dynamic pre-sharding factor. When assigning each part of data, the ratio of the required storage volume of each database is used as a probability value, and the data is assigned to either database 1 or database 3 based on this probability value.

[0102] Alternatively, the partitioned storage algorithm can select only the databases, for example, database 1 and database 3. Then, based on the number of shards (2), a dynamic pre-sharding factor of 20 is obtained. The data to be stored is then divided into 20 parts according to this dynamic pre-sharding factor, with each part having a dynamic pre-sharding identifier between 0 and 19. Next, a modulo operation is performed between the dynamic pre-sharding identifier and the number of shards. The result is either 0 or 1. Data with a modulo result of 0 is assigned to one database, and data with a modulo result of 1 is assigned to another database. This achieves uniform partitioning and reduces the pressure on database storage.

[0103] In this example, a preset partitioning storage algorithm is used to calculate the data volume of the data to be stored and the storage evaluation information of each database, thereby selecting a suitable database for storage and obtaining the corresponding storage configuration information. Based on the storage configuration information, the data to be stored is partitioned and stored, reducing the storage pressure on the database.

[0104] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0105] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0106] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a data processing device based on a chain of responsibility, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0107] like Figure 3 As shown, the data processing device 300 based on the chain of responsibility described in this embodiment includes: a data acquisition module 301, an information acquisition module 302, a node selection module 303, an evaluation value calculation module 304, a node determination module 305, and a data processing module 306, wherein:

[0108] The data acquisition module 301 is used to acquire data to be processed.

[0109] The information acquisition module 302 is used to acquire data processing evaluation information of each node in the node network based on the data to be processed.

[0110] The node selection module 303 is used to select a target number of candidate nodes from the node network.

[0111] The evaluation value calculation module 304 is used to calculate the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node.

[0112] The node determination module 305 is used to determine each candidate node as a chain of responsibility node when the data processing evaluation value of each candidate node meets the preset evaluation value conditions.

[0113] The data processing module 306 is used to process the data to be processed through the chain of responsibility formed by the nodes of the chain of responsibility.

[0114] In this embodiment, data to be processed is acquired, and based on the data to be processed, data processing evaluation information of each node in the node network is acquired. The data processing evaluation information is used to record the historical processing status of nodes for a certain type of business data. A target number of candidate nodes are selected from the node network, and the data processing evaluation value of each candidate node is calculated according to its data processing evaluation information to determine the data processing performance and suitability of the candidate nodes for the data to be processed. When the data processing evaluation value of each candidate node meets the preset evaluation value conditions, it indicates that each candidate node meets the data processing requirements and can be used as a chain of responsibility node. The data to be processed is processed through the chain of responsibility formed by the chain of responsibility nodes. This application improves the performance and accuracy of data processing based on the chain of responsibility by calculating the data processing evaluation value to select nodes suitable for processing the data to be processed to form a chain of responsibility.

[0115] In some optional implementations of this embodiment, the data processing device 300 based on the chain of responsibility may further include: a history acquisition module and an evaluation generation module, wherein:

[0116] The history acquisition module is used to acquire historical processing information of each node in the node network at different time periods. The historical processing information includes request basic information and request service type.

[0117] The evaluation generation module is used to generate data processing evaluation information for each node in different time periods based on the historical processing information of each node in different time periods. The data processing evaluation information includes multiple evaluation factors, including time period identifier, business type identifier, number of requests received, request processing time, and business type correlation.

[0118] In this embodiment, data processing evaluation information for each node is generated at different time periods. The data processing evaluation information includes various evaluation factors such as time period identifier, business type identifier, number of requests received, request processing duration, and business type correlation. It can evaluate the processing status of a node on business data of a certain business type from different perspectives.

[0119] In some optional implementations of this embodiment, the information acquisition module 302 may include: a type acquisition submodule, a time acquisition submodule, and an evaluation acquisition submodule, wherein:

[0120] The type acquisition submodule is used to obtain the business type identifier from the data to be processed.

[0121] The Time Acquisition submodule is used to obtain the arrival time of the data to be processed in order to determine the time period identifier corresponding to the arrival time of the data.

[0122] The evaluation acquisition submodule is used to obtain data processing evaluation information for each node in the node network based on the service type identifier and time period identifier.

[0123] In this example, data processing evaluation information is obtained based on the business type identifier and time period identifier of the data to be processed, so that the data processing evaluation information matches the data to be processed that has arrived, thus ensuring the accuracy of subsequent data processing evaluation value calculation.

[0124] In some optional implementations of this embodiment, the evaluation value calculation module 304 may include: a weight determination submodule and an evaluation value calculation submodule, wherein:

[0125] The weight determination submodule is used to determine the factor weights of each evaluation factor in the data processing evaluation information of each candidate node according to a preset weight strategy.

[0126] The evaluation value calculation submodule is used to calculate the evaluation value of each candidate node based on the obtained factor weights.

[0127] In this embodiment, factor weights are added to each evaluation factor to distinguish and differentiate them, thereby ensuring the rationality of the calculated data processing evaluation values.

[0128] In some optional implementations of this embodiment, the node determination module 305 may include: a mean calculation submodule, a fluctuation value calculation submodule, and a node determination submodule, wherein:

[0129] The mean calculation submodule is used to calculate the mean of the data processing evaluation value of each candidate node.

[0130] The fluctuation value calculation submodule is used to calculate the fluctuation value of the data processing evaluation value of each candidate node based on the mean.

[0131] The node determination submodule is used to determine that the data processing evaluation value of each candidate node meets the preset evaluation value conditions when all fluctuation values ​​are less than the preset fluctuation value threshold, and to determine each candidate node as a responsibility chain node.

[0132] In this example, the mean of the data processing evaluation value of each candidate node is calculated, and then the fluctuation value is calculated separately. When each fluctuation value is less than the preset fluctuation value threshold, it indicates that the data processing capabilities of each node are relatively balanced and meet the evaluation value conditions, thus obtaining the responsibility chain node.

[0133] In some optional implementations of this embodiment, the chain-of-responsibility-based data processing device 300 may further include: a complement selection module, wherein:

[0134] The complement selection module is used to select a complement node to replace a candidate node when there is a candidate node whose data processing evaluation value does not meet the preset evaluation value conditions. The complement node is a node other than the candidate node in the node network.

[0135] The evaluation value calculation module 304 is also used to calculate the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node after the node replacement is completed.

[0136] In this embodiment, when there are candidate nodes whose data processing evaluation values ​​do not meet the evaluation value conditions, the complement node is selected to replace the candidate node and iterative calculation is performed in order to find a candidate node that meets the requirements.

[0137] In some optional implementations of this embodiment, the data processing device 300 based on the chain of responsibility may further include: a storage acquisition module, an evaluation acquisition module, a configuration generation module, and a segmented storage module, wherein:

[0138] The storage acquisition module is used to acquire the data to be stored and its corresponding data volume, wherein the data to be stored includes data to be processed and data that has already been processed.

[0139] The evaluation acquisition module is used to acquire storage evaluation information for each database, which includes various storage evaluation factors.

[0140] The configuration generation module is used to calculate the storage configuration information based on the data volume and storage evaluation information according to the preset partitioning storage algorithm. The storage configuration information includes the selected database and its corresponding required storage volume.

[0141] The partitioned storage module is used to partition the data to be stored and store it in the database according to the storage configuration information to obtain the inventory data.

[0142] In this example, a preset partitioning storage algorithm is used to calculate the data volume of the data to be stored and the storage evaluation information of each database, thereby selecting a suitable database for storage and obtaining the corresponding storage configuration information. Based on the storage configuration information, the data to be stored is partitioned and stored, reducing the storage pressure on the database.

[0143] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0144] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0145] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0146] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions based on the chain of responsibility data processing method. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0147] In some embodiments, the processor 42 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions of the chain-of-responsibility data processing method.

[0148] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0149] The computer device provided in this embodiment can execute the above-described chain-of-responsibility-based data processing method. Here, the chain-of-responsibility-based data processing method can be any of the chain-of-responsibility-based data processing methods described in the various embodiments above.

[0150] In this embodiment, data to be processed is acquired, and based on the data to be processed, data processing evaluation information of each node in the node network is acquired. The data processing evaluation information is used to record the historical processing status of nodes for a certain type of business data. A target number of candidate nodes are selected from the node network, and the data processing evaluation value of each candidate node is calculated according to its data processing evaluation information to determine the data processing performance and suitability of the candidate nodes for the data to be processed. When the data processing evaluation value of each candidate node meets the preset evaluation value conditions, it indicates that each candidate node meets the data processing requirements and can be used as a chain of responsibility node. The data to be processed is processed through the chain of responsibility formed by the chain of responsibility nodes. This application improves the performance and accuracy of data processing based on the chain of responsibility by calculating the data processing evaluation value to select nodes suitable for processing the data to be processed to form a chain of responsibility.

[0151] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the chain-of-responsibility-based data processing method as described above.

[0152] In this embodiment, data to be processed is acquired, and based on the data to be processed, data processing evaluation information of each node in the node network is acquired. The data processing evaluation information is used to record the historical processing status of nodes for a certain type of business data. A target number of candidate nodes are selected from the node network, and the data processing evaluation value of each candidate node is calculated according to its data processing evaluation information to determine the data processing performance and suitability of the candidate nodes for the data to be processed. When the data processing evaluation value of each candidate node meets the preset evaluation value conditions, it indicates that each candidate node meets the data processing requirements and can be used as a chain of responsibility node. The data to be processed is processed through the chain of responsibility formed by the chain of responsibility nodes. This application improves the performance and accuracy of data processing based on the chain of responsibility by calculating the data processing evaluation value to select nodes suitable for processing the data to be processed to form a chain of responsibility.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0154] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A data processing method based on the chain of responsibility, characterized in that, Includes the following steps: The historical processing information of each node in the node network at different time periods is obtained. The historical processing information includes basic request information and request service type. The basic request information includes the number of requests received, the request processing time, and the request processing evaluation result. The historical processing information is obtained by each node processing data according to the received data processing requests and recording the data processing process. The data processing requests include the request service type. Based on the historical processing information of each node in different time periods, data processing evaluation information of each node in different time periods is generated. The data processing evaluation information includes multiple evaluation factors, including time period identifier, business type identifier, number of requests received, request processing duration, and business type correlation. The data processing evaluation information is grouped according to time period identifier and business type identifier. Under different business types, the time period length corresponding to the time period identifier is different. The time period length is set according to the time scenario requirements of the business type. Obtain the data to be processed; Based on the data to be processed, obtain the data processing evaluation information of each node in the node network; Select a target number of candidate nodes from the node network; Calculate the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node; When the data processing evaluation value of each candidate node meets the preset evaluation value condition, the candidate node is determined as a responsibility chain node, including: calculating the mean of the data processing evaluation value of each candidate node, subtracting the data processing evaluation value from the mean to obtain the difference, dividing the absolute value of the difference by the mean to obtain the fluctuation value, and when each fluctuation value is less than the preset fluctuation value threshold, it is determined that the data processing evaluation value of each candidate node meets the preset evaluation value condition, and the candidate node is determined as a responsibility chain node; The data to be processed is processed through a chain of responsibility formed by the nodes of the chain of responsibility.

2. The data processing method based on the chain of responsibility according to claim 1, characterized in that, The step of obtaining data processing evaluation information for each node in the node network based on the data to be processed includes: Obtain the business type identifier from the data to be processed; Obtain the arrival time of the data to be processed to determine the time period identifier corresponding to the arrival time of the data; Based on the service type identifier and the time period identifier, obtain the data processing evaluation information of each node in the node network.

3. The data processing method based on the chain of responsibility according to claim 1, characterized in that, The step of calculating the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node includes: The factor weights of each evaluation factor in the data processing evaluation information of each candidate node are determined according to the preset weighting strategy. The data processing evaluation value of each candidate node is obtained by calculating the evaluation factors based on the obtained factor weights.

4. The data processing method based on the chain of responsibility according to claim 1, characterized in that, After the step of calculating the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node, the method further includes: When there is a candidate node whose data processing evaluation value does not meet the preset evaluation value conditions, a complement node is selected to replace the candidate node, wherein the complement node is a node other than the candidate node in the node network. After the node replacement is completed, the step of calculating the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node is executed.

5. The data processing method based on the chain of responsibility according to claim 1, characterized in that, The method further includes: Obtain the data to be stored and its corresponding data volume, wherein the data to be stored includes the data to be processed and the processed data; Obtain storage evaluation information for each database, wherein the storage evaluation information includes multiple storage evaluation factors; Storage configuration information is obtained by calculating the data volume and storage evaluation information according to a preset partitioning storage algorithm, wherein the storage configuration information includes the selected database and its corresponding required storage volume; Based on the storage configuration information, the data to be stored is segmented and stored in a database to obtain inventory data.

6. A data processing device based on a chain of responsibility, characterized in that, include: The historical acquisition module is used to acquire historical processing information of each node in the node network at different time periods. The historical processing information includes request basic information and request service type. The request basic information includes the number of requests received, the request processing time, and the request processing evaluation result. The historical processing information is obtained by each node processing data according to the received data processing requests and recording the data processing process. The data processing requests include the request service type. The evaluation generation module is used to generate data processing evaluation information for each node in different time periods based on the historical processing information of each node in different time periods. The data processing evaluation information includes multiple evaluation factors, including time period identifier, business type identifier, number of requests received, request processing duration, and business type correlation. The data processing evaluation information is grouped according to the time period identifier and the business type identifier. Under different business types, the time period length corresponding to the time period identifier is different. The time period length is set according to the time scenario requirements of the business type. The data acquisition module is used to acquire the data to be processed. The information acquisition module is used to acquire data processing evaluation information of each node in the node network based on the data to be processed; A node selection module is used to select a target number of candidate nodes from the node network; The evaluation value calculation module is used to calculate the data processing evaluation value of each candidate node based on the data processing evaluation information of each candidate node. A node determination module is used to determine each candidate node as a chain of responsibility node when the data processing evaluation value of each candidate node meets a preset evaluation value condition. The module includes: calculating the mean of the data processing evaluation values ​​of each candidate node; subtracting the data processing evaluation value from the mean to obtain a difference; dividing the absolute value of the difference by the mean to obtain a fluctuation value; and determining that the data processing evaluation value of each candidate node meets the preset evaluation value condition and determining each candidate node as a chain of responsibility node when all fluctuation values ​​are less than a preset fluctuation value threshold. The data processing module is used to process the data to be processed through the chain of responsibility formed by the nodes of the chain of responsibility.

7. A computer device comprising a memory and a processor, the memory storing computer-readable instructions, wherein the processor, when executing the computer-readable instructions, implements the steps of the chain-of-responsibility-based data processing method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the chain-of-responsibility-based data processing method as described in any one of claims 1 to 5.

Images