Data processing method, device, equipment, readable storage medium and product

Abstract

Embodiments of the present disclosure provide a data processing method, device, equipment, readable storage medium and product, the method comprising: obtaining at least one to be shunted data information and to be shunted calculation proportion information; generating at least one shunting rule tree diagram according to to be shunted calculation proportion information and preset configuration proportion; for each node in the shunting rule tree diagram, obtaining shunted data according to the storage location corresponding to the identification information of the shunting node corresponding to the node, and performing data shunting calculation on the shunted data according to the shunting proportion and the configuration proportion; generating a shunting result model according to the shunting result corresponding to each shunting node. Thus, data shunting calculation can be automatically performed on the nodes in the shunting rule tree diagram, without the need for encoding operation according to the calculation task. By generating a shunting result model according to the shunting result, based on the tree structure of the shunting rule tree diagram, the abnormal calculation node can be quickly located when a problem occurs in the shunting result subsequently.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of big data, and in particular to a data processing method, apparatus, device, readable storage medium, and product. Background Art

[0002] Profit or loss allocation refers to the distribution of a company's profits or losses across departments, or from departments to smaller organizations or individuals. Profit or loss allocation in the big data era is often characterized by massive amounts of data, complex organizational hierarchies, and diverse allocation rules.

[0003] The existing profit and loss diversion calculation method generally creates calculation tasks one-to-one based on the needs of the business party. A single calculation task must be manually executed in series. For example, the diversion from the overall to the first-level department can be completed first, and then the diversion from the first-level department to the second-level department can be executed after completion. The concurrency between calculation tasks depends on the creation of multiple calculation tasks on an external platform.

[0004] During the implementation of the present disclosure, the inventors discovered that the prior art suffers from at least the following issues: When using the aforementioned method to calculate profit and loss splits, abnormal split results require code tracking and location. Furthermore, the development of calculation tasks is performed on a one-to-one basis, requiring code adjustments when changes are made to split ratios, split rules, and other content. This often results in cumbersome operations and low computational efficiency. Summary of the Invention

[0005] The embodiments of the present disclosure provide a data processing method, apparatus, device, readable storage medium, and product to solve the technical problems of existing data processing methods that are complicated to operate and have low computational efficiency.

[0006] In a first aspect, an embodiment of the present disclosure provides a data processing method, including:

[0007] Obtain at least one piece of data to be diverted and information on a calculation ratio of the data to be diverted, wherein the data to be diverted includes identification information and a storage location corresponding to at least one diversion node, and the information on a calculation ratio of the data to be diverted includes a diversion ratio of each diversion node;

[0008] Generate at least one diversion rule tree diagram based on the calculated ratio information of the traffic to be diverted and the preset configuration ratio, wherein the nodes in the diversion rule tree diagram are identification information corresponding to the diversion nodes, and the edges in the diversion rule tree diagram are diversion ratios corresponding to the diversion nodes;

[0009] For each node in the shunting rule tree diagram, shunting data is obtained according to the storage location corresponding to the identification information of the shunting node corresponding to the node, data shunting calculation is performed on the shunting data according to the shunting proportion and the configuration proportion, and a shunting result corresponding to each shunting node is obtained.

[0010] A shunting result model is generated according to the shunting result corresponding to each shunting node.

[0011] In a second aspect, the embodiments of the present disclosure provide a data processing apparatus, including:

[0012] The obtaining module is configured to obtain at least one piece of to-be-shunted data information and to-be-shunted calculation proportion information, wherein the to-be-shunted data includes identification information and a storage location corresponding to at least one shunting node, and the to-be-shunted calculation proportion information includes a shunting proportion of each shunting node.

[0013] The generating module is configured to generate at least one shunting rule tree diagram according to the to-be-shunted calculation proportion information and a preset configuration proportion, wherein a node in the shunting rule tree diagram is identification information of a shunting node, and an edge in the shunting rule tree diagram is a shunting proportion of each shunting node.

[0014] The calculating module is configured to, for each node in the shunting rule tree diagram, obtain shunting data according to a storage location corresponding to identification information of a shunting node corresponding to the node, perform data shunting calculation on the shunting data according to the shunting proportion and the configuration proportion, and obtain a shunting result corresponding to each shunting node.

[0015] The processing module is configured to generate a shunting result model according to the shunting result corresponding to each shunting node.

[0016] In a third aspect, the embodiments of the present disclosure provide an electronic device, including a memory and a processor.

[0017] The memory is configured to store instructions executable by the processor.

[0018] The processor is configured to invoke program instructions in the memory to execute the data processing method in the first aspect.

[0019] In a fourth aspect, the embodiments of the present disclosure provide a computer-readable storage medium, which stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the data processing method in the first aspect.

[0020] In a fifth aspect, the embodiments of the present disclosure provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the data processing method in the first aspect.

[0021] The data processing method, device, equipment, readable storage medium and product provided by the embodiments of the present disclosure can generate at least one shunting rule tree diagram according to the to-be-shunted calculation proportion information and the preset configuration proportion, obtain shunting data according to the storage location corresponding to the identification information of the shunting node corresponding to each node in the shunting rule tree diagram, perform data shunting calculation on the shunting data according to the shunting proportion and the configuration proportion, and obtain the shunting result corresponding to each shunting node. A shunting result model is generated according to the shunting result corresponding to each shunting node. Thus, the shunting rule tree diagram can be automatically generated, and the nodes in the shunting rule tree diagram can be automatically subjected to data shunting calculation, without the need for encoding operation according to the calculation task. In addition, the shunting result model is generated according to the shunting result and based on the tree structure of the shunting rule tree diagram, so that the positioning of an abnormal calculation node can be quickly implemented when a problem occurs in the shunting result. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the present disclosure.

[0023] Figure 1 A flowchart of a data processing method provided by an embodiment of the present disclosure;

[0024] Figure 2 A schematic diagram of a shunting rule tree diagram provided by the present disclosure;

[0025] Figure 3 A flowchart of a data processing method provided by an embodiment of the present disclosure;

[0026] Figure 4 A flowchart of a data processing method provided by an embodiment of the present disclosure;

[0027] Figure 5 A modification schematic diagram of a shunting rule tree diagram provided by an embodiment of the present disclosure;

[0028] Figure 6 A structure schematic diagram of a data processing device provided by an embodiment of the present disclosure;

[0029] Figure 7 A structure schematic diagram of an electronic device provided by an embodiment of the present disclosure.

[0030] The above-described drawings have shown the specific embodiments of the present disclosure, and more detailed descriptions will be given hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present disclosure by any means, but to illustrate the concept of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0031] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, like reference numerals refer to like elements, unless the context clearly dictates otherwise. The following exemplary embodiments are described in enough detail to enable those having ordinary skill in the art to make and use the apparatus and methods in accordance with the present disclosure. The examples provided serve merely as illustration of the apparatus and methods consistent with the present disclosure.

[0032] Firstly, the terms related to the embodiments of the present disclosure are explained:

[0033] Allocation of Profit or Loss: a financial term, specifically referring to the distribution of the "profit / loss" of an enterprise to various departments (or subgroups), or from various departments (or subgroups) to smaller institutional units or individuals.

[0034] In view of the technical problems of the existing data processing method mentioned above, the operation is cumbersome, and the calculation efficiency is low. The present disclosure provides a data processing method, device, equipment, readable storage medium and product.

[0035] It should be noted that the data processing method, device, equipment, readable storage medium and product provided by the present disclosure can be applied in various data distribution scenarios.

[0036] Due to the actual demand, the business requirements are various and complex. At present, one-to-one mode is mainly used for the development of calculation tasks. When the business rules (distribution ratio, distribution rule) change, the code needs to be iterated or the data storage structure needs to be adjusted.

[0037] In the process of solving the above technical problems, the inventors found through research that a tree structure can be used to configure the distribution rule, and a key-value storage method can be used to store the distribution result and the data source. Specifically, at least one distribution rule tree diagram is generated according to the to-be-distributed calculation ratio information and the preset configuration ratio. For each node in the distribution rule tree diagram, the distribution data is obtained according to the storage location corresponding to the identification information of the distribution node corresponding to the node, and the distribution data is calculated according to the distribution ratio and the configuration ratio to obtain the distribution result corresponding to each distribution node. The distribution result model is generated according to the distribution result corresponding to each distribution node. Thus, the distribution rule tree diagram can be automatically generated, and the nodes in the distribution rule tree diagram can be automatically calculated for data distribution. There is no need to code according to the calculation task. In addition, by generating the distribution result model according to the distribution result and based on the tree structure of the distribution rule tree diagram, the positioning of the abnormal calculation node can be quickly realized when the distribution result is problematic.

[0038] The technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes can not be described again in some examples. The embodiments of the present disclosure will be described below with reference to the drawings.

[0039] Figure 1 A flowchart of a data processing method provided by Embodiment One of the present disclosure is shown in FIG. 1, which includes the following steps. Figure 1

[0040] In step 101, at least one data to be shunted and calculation proportion information to be shunted are obtained, wherein the data to be shunted includes identification information and storage location corresponding to at least one shunting node, and the calculation proportion information to be shunted includes shunting proportion of each shunting node.

[0041] The execution subject of the present embodiment is a data processing device, which can be coupled to a server capable of obtaining data from a database or a business system and performing data shunting calculation.

[0042] In the present embodiment, in order to realize the shunting operation of data, at least one data to be shunted and calculation proportion information to be shunted can be obtained first. The data to be shunted includes identification information and storage location corresponding to at least one shunting node, and the calculation proportion information to be shunted includes shunting proportion of each shunting node.

[0043] Optionally, the data to be shunted can be profit and loss data to be allocated, or any kind of task data that needs to be shunted to multiple nodes, etc., which is not limited by the present disclosure.

[0044] In step 102, at least one shunting rule tree diagram is generated according to the calculation proportion information to be shunted and a preset configuration proportion, wherein the nodes in the shunting rule tree diagram are identification information corresponding to the shunting nodes, and the edges in the shunting rule tree diagram are shunting proportions corresponding to the shunting nodes.

[0045] In the present embodiment, the configuration proportion can be set in advance, and the generation of the shunting rule tree diagram can be performed according to the configuration proportion subsequently. For example, the configuration proportion can be data allocation according to the number of people, or the configuration proportion can be data allocation according to the weight corresponding to the department. Taking an actual application as an example, a piece of data is shunted to two departments, department A has 40 people, and department B has 60 people. When the configuration proportion is data allocation according to the number of people, 40% of the data can be allocated to department A, and 60% of the data can be allocated to department B.

[0046] ​The tree structure has a natural data cascade dependency relationship, can be very convenient for storing shunting rules, and can visually display the whole process of shunting calculation. After obtaining the to-be-shunted data information and the to-be-shunted calculation proportion information, at least one shunting rule tree diagram can be generated according to the to-be-shunted calculation proportion information and the preset configuration proportion. In the shunting rule tree diagram, the root node is all to-be-shunted data, the node in the shunting rule tree diagram is the identification information corresponding to the shunting node, and the edge in the shunting rule tree diagram is the shunting proportion corresponding to each shunting node.

[0047] Since the number of to-be-shunted data information is at least one, at least one shunting rule tree diagram can be generated according to the at least one to-be-shunted data information, so as to realize the data shunting operation on the at least one to-be-shunted data information. By using the tree diagram to display and store the shunting rules, the user can visually track the calculation state of each shunting link through the tree diagram. When a problem occurs in a link calculation, the problem link can be quickly located.

[0048] In step 103, for each node in the shunting rule tree diagram, shunting data is obtained according to the storage location corresponding to the identification information of the shunting node corresponding to the node, and data shunting calculation is performed on the shunting data according to the shunting proportion and the configuration proportion, to obtain the shunting result corresponding to each shunting node.

[0049] In this embodiment, for each node in the shunting rule tree diagram, shunting data can be obtained in the storage location corresponding to the identification information of the shunting node corresponding to the node. For the shunting data, data shunting calculation is performed on the shunting data according to the shunting proportion and the configuration proportion, to obtain the shunting result corresponding to each shunting node.

[0050] Specifically, for each shunting node, the shunting data can be multiplied by the shunting proportion and the configuration proportion to obtain the shunting result corresponding to each shunting node.

[0051] In step 104, a shunting result model is generated according to the shunting result corresponding to each shunting node.

[0052] In this embodiment, after the shunting calculation is completed, a shunting result model can be generated according to the shunting result corresponding to each shunting node. Therefore, subsequent query operations of shunting calculation can be realized based on the shunting result model.

[0053] In actual applications, different businesses may have various data dimensions. In order to unify the storage structure, facilitate the expansion and shunting calculation of applications, a key-value (KV) structure can be used to generate a shunting result model according to the shunting result corresponding to each shunting node.

[0054] Figure 2 A schematic diagram of the shunting rule tree provided by the present disclosure is shown in FIG. 1. As shown in FIG. 1, a root node 21 can be for all data to be shunted, and the root node 21 can be provided with an identifier corresponding thereto. The data to be shunted can be shunted to a first-level department 22 according to a preset shunting ratio 1.0. Figure 2 The data shunted to the first-level department 22 can be shunted to a second-level department 23 according to a preset shunting ratio 1.0. The data shunted to the second-level department 23 can be shunted to a third-level department 24 according to a preset shunting ratio 1.0 and to a fourth-level department 25 according to a preset shunting ratio 1.0. The subsequent data can be calculated according to the shunting rule tree and the preset shunting ratio.

[0055] Further, based on the first embodiment, after step 101, the method further comprises:

[0056] According to the data to be shunted, corresponding shunted data is obtained from a storage location corresponding to each shunting node.

[0057] According to the shunted data, a shunted data source model is generated using a key-value storage structure.

[0058] After step 102, the method further comprises:

[0059] According to the data to be shunted, corresponding shunted data is obtained from a storage location corresponding to each shunting node.

[0060] In this embodiment, after obtaining the data to be shunted, corresponding shunted data can be obtained from a storage location corresponding to each shunting node according to the data to be shunted. According to the shunted data, a shunted data source model is generated using a key-value storage structure. By using a key-value storage structure to store data, when a shunted business requirement is added or changed, the table structure does not need to be changed, and only the addition and deletion of data in the corresponding position of the shunted data source model is required.

[0061] After the shunted data source model is established, when shunting calculation is performed, shunted data can be obtained from the shunted data source model. Specifically, according to the data to be shunted, corresponding shunted data is obtained from a storage location corresponding to each shunting node according to the data to be shunted.

[0062] ​The data processing method provided in this embodiment generates at least one shunting rule tree diagram according to the to-be-shunted calculation proportion information and the preset configuration proportion, acquires shunting data according to a storage location corresponding to identification information of a shunting node corresponding to each node in the shunting rule tree diagram, performs data shunting calculation on the shunting data according to the shunting proportion and the configuration proportion, and obtains a shunting result corresponding to each shunting node. A shunting result model is generated according to the shunting result corresponding to each shunting node. Thus, the shunting rule tree diagram can be automatically generated, and the nodes in the shunting rule tree diagram can be automatically subjected to data shunting calculation, without the need for coding operation according to a calculation task. In addition, the shunting result model is generated according to the shunting result and based on the tree structure of the shunting rule tree diagram, so that subsequent positioning of an abnormal calculation node can be quickly implemented when a problem occurs in the shunting result.

[0063] Figure 3 The flowchart of the data processing method provided in Embodiment Two of the present disclosure is shown in FIG. 3, which is based on Embodiment One. Figure 3 As shown in FIG. 3, step 103 specifically includes:

[0064] Step 301: A preset shunting node calculation rule is acquired, wherein the shunting node calculation rule includes a thread / process number for concurrent processing of a shunting node and a number of jobs processed by each thread / process.

[0065] Step 302: A product of the thread / process number and the number of jobs is determined as a target number of the shunting nodes for concurrent processing.

[0066] Step 303: Data shunting calculation is performed on the target number of shunting nodes concurrently, and a shunting result corresponding to each shunting node is obtained.

[0067] In this embodiment, the shunting rule tree diagram can include multiple nodes, and in order to improve the efficiency of data shunting calculation, multiple nodes can be processed simultaneously in a concurrent manner. Specifically, a preset shunting node calculation rule can be acquired, wherein the shunting node calculation rule includes a thread / process number for concurrent processing of a shunting node and a number of jobs processed by each thread / process. A product of the thread / process number and the number of jobs is determined as a target number of the shunting nodes for concurrent processing. For example, the number of threads can be 3, and the number of jobs processed by the threads can be 5, so the target number of the shunting nodes for concurrent processing can be 3 5=15. The first layer of concurrency is executed by threads / processes, and the second layer of concurrency is executed by jobs inside Hadoop.

[0068] After the target number of shunting nodes is determined, data shunting calculation can be performed on the target number of shunting nodes in parallel to obtain shunting results corresponding to the shunting nodes.

[0069] Further, before step 303, the method further comprises:

[0070] The target number of shunting nodes is determined according to the hierarchical order of the shunting nodes in the shunting rule tree diagram.

[0071] The target number of shunting nodes is sorted according to the hierarchical order of the shunting nodes in the shunting rule tree diagram.

[0072] In this embodiment, in order to avoid mutual dependence between multiple nodes, data processing can also be performed according to the hierarchical order of the shunting nodes in the shunting rule tree diagram when performing parallel calculation. Specifically, the target number of shunting nodes can be determined according to the hierarchical order of the shunting nodes in the shunting rule tree diagram. The target number of shunting nodes is sorted in parallel according to the hierarchical order.

[0073] Further, based on any of the above embodiments, step 303 comprises:

[0074] For each target shunting node corresponding to a job, it is detected whether the parent node corresponding to the target shunting node has completed data shunting calculation.

[0075] If yes, target shunting data is obtained according to the storage location corresponding to the identification information of the target shunting node, data shunting calculation is performed on the target shunting data according to the shunting ratio and the configuration ratio, and a shunting result corresponding to the target shunting node is obtained.

[0076] If no, data shunting calculation is not performed on the target shunting node.

[0077] In this embodiment, since data shunting operations are generally performed sequentially, for example, the data shunting result of a previous node is shunted to multiple nodes below. Therefore, when each node performs shunting calculation, it is necessary to ensure that the parent node corresponding to the node has been calculated to ensure the accuracy of the calculation result. Specifically, for a target shunting node corresponding to a job, it can be detected whether the parent node corresponding to the target shunting node has completed data shunting calculation. When the parent node corresponding to the target shunting node has completed data shunting calculation, target shunting data can be obtained according to the storage location corresponding to the identification information of the target shunting node, data shunting calculation can be performed on the target shunting data according to the shunting ratio and the configuration ratio, and a shunting result corresponding to the target shunting node is obtained. Otherwise, data shunting calculation can not be performed on the target shunting node, and the data shunting calculation can be performed after the parent node corresponding to the target shunting node has been calculated.

[0078] The data processing method provided in this embodiment can improve the efficiency of data shunting calculation by processing multiple nodes in parallel. In addition, by processing the nodes in the hierarchical order of the shunting nodes in the shunting rule tree diagram and performing shunting calculation after the parent nodes corresponding to each target shunting node are calculated, the accuracy of shunting calculation can be ensured.

[0079] Figure 4 The flowchart of the data processing method provided in Embodiment Three of the present disclosure is based on any of the above embodiments,

[0080] Step 401, in response to a modification instruction triggered by a user, wherein the modification instruction includes identification information of a node and a configuration ratio corresponding to the node.

[0081] Step 402, modifying the configuration ratio of the node in the shunting rule tree diagram according to the modification instruction.

[0082] In this embodiment, after the shunting rule tree diagram is established, when the business rules change, the configuration ratio in the shunting rule tree diagram can be modified. Specifically, in response to a modification instruction triggered by a user, wherein the modification instruction includes identification information of a node and a configuration ratio corresponding to the node. The configuration ratio of the node in the shunting rule tree diagram can be modified according to the modification instruction.

[0083] Figure 5 The modification diagram of the shunting rule tree diagram provided in the embodiment of the present disclosure is shown as follows, Figure 5 According to the modification instruction triggered by the user, 40% of the data 56 can be shunted to the child node 53 and 60% of the data 57 can be shunted to the child node 55 according to the shunting ratio.

[0084] Further, the method further includes:

[0085] obtaining an update instruction triggered by a user, wherein the update instruction includes a storage location and to-be-updated content.

[0086] updating the to-be-updated content to the storage location according to the update instruction.

[0087] In the embodiment, the data is stored by adopting the value-reducing storage structure, so that when a new shunting service demand or demand change occurs, the table structure does not need to be changed, and only the addition and deletion of data in the corresponding position of the shunting data source model need to be performed. Specifically, an update instruction triggered by a user can be acquired, wherein the update instruction includes a storage position and to-be-updated content. The to-be-updated content can be updated to the storage position according to the update instruction.

[0088] The data processing method provided in the embodiment modifies the configuration proportion of a node in a shunting rule tree diagram according to a modification instruction triggered by a user, and updates the to-be-updated content to the storage position according to an update instruction, so that when a service rule changes, the data structure does not need to be changed, and the code does not need to be rewritten, thereby improving the efficiency of data shunting calculation.

[0089] Figure 6 A structural schematic diagram of the data processing apparatus provided in Embodiment Four of the present disclosure is shown in FIG. 6. Figure 6 As shown in FIG. 6, the apparatus includes an acquisition module 61, a generation module 62, a calculation module 63, and a processing module 64. The acquisition module 61 is configured to acquire at least one to-be-shunted data information and to-be-shunted calculation proportion information, wherein the to-be-shunted data includes identification information and a storage position corresponding to each shunting node, and the to-be-shunted calculation proportion information includes a shunting proportion of each shunting node. The generation module 62 is configured to generate at least one shunting rule tree diagram according to the to-be-shunted calculation proportion information and a preset configuration proportion, wherein a node in the shunting rule tree diagram is the identification information corresponding to a shunting node, and an edge in the shunting rule tree diagram is the shunting proportion corresponding to each shunting node. The calculation module 63 is configured to, for each node in the shunting rule tree diagram, acquire shunting data according to a storage position corresponding to the identification information of a shunting node corresponding to the node, and perform data shunting calculation on the shunting data according to the shunting proportion and the configuration proportion, to obtain a shunting result corresponding to each shunting node. The processing module 64 is configured to generate a shunting result model according to the shunting result corresponding to each shunting node.

[0090] Further, on the basis of Embodiment Four, the calculation module is configured to, for each shunting node, multiply the shunting data by the shunting proportion and the configuration proportion, to obtain a shunting result corresponding to each shunting node.

[0091] Further, based on the embodiment four, the apparatus further comprises: an obtaining module, further configured to obtain the corresponding shunted data from the storage location corresponding to each shunting node according to the to-be-shunted data information; a generating module, further configured to generate a shunted data source model in a key-value storage structure according to the shunted data; the apparatus further comprises: a shunted data obtaining module, configured to obtain the shunted data corresponding to the shunting node from the shunted data source model according to the data identifier of the shunting node corresponding to each node in the shunting rule tree diagram.

[0092] Further, based on the embodiment four, the processing module is configured to generate a shunted result model in a key-value storage structure according to the shunted results of each shunting node

[0093] Further, based on the embodiment four, the computing module is configured to: obtain a preset shunting node computing rule, wherein the shunting node computing rule comprises the number of threads / processes for concurrent processing of the shunting node and the number of jobs processed by each thread / process; determine the product of the number of threads / processes and the number of jobs as a target number of shunting nodes for concurrent processing; and perform data shunting calculation on the target number of shunting nodes concurrently to obtain the shunted results corresponding to each shunting node.

[0094] Further, based on any of the above embodiments, the computing module is further configured to: determine the target number of shunting nodes in the hierarchical order of the shunting nodes in the shunting rule tree diagram; and sort the target number of shunting nodes according to the hierarchical order of the shunting nodes in the shunting rule tree diagram.

[0095] Further, based on any of the above embodiments, the computing module is configured to: for each target shunting node corresponding to a job, detect whether the parent node corresponding to the target shunting node has completed data shunting calculation; if yes, obtain target shunted data from the storage location corresponding to the identification information of the target shunting node, and perform data shunting calculation on the target shunted data according to the shunting proportion and the configuration proportion to obtain the shunted result corresponding to the target shunting node; and if no, do not perform data shunting calculation on the target shunting node.

[0096] Further, based on any of the above embodiments, the apparatus further comprises: an instruction obtaining module configured to obtain a modification instruction triggered by a user, wherein the modification instruction comprises the identification information of a node and the configuration proportion corresponding to the node; and a modifying module configured to modify the configuration proportion of the node in the shunting rule tree diagram according to the modification instruction.

[0097] Further, based on any of the above embodiments, the apparatus further includes an instruction obtaining module configured to obtain an update instruction triggered by the user, the update instruction including a storage location and to-be-updated content; and an updating module configured to update the to-be-updated content into the storage location according to the update instruction.

[0098] Yet another embodiment of the present disclosure further provides an electronic device, including: a memory, a processor;

[0099] a memory; a memory for storing instructions executable by the processor;

[0100] The processor is configured to invoke program instructions in the memory to execute the data processing method according to any of the above embodiments.

[0101] Yet another embodiment of the present disclosure further provides a computer readable storage medium, the computer readable storage medium has computer execution instructions stored therein, the computer execution instructions are executed by a processor to implement the data processing method according to any of the above embodiments.

[0102] Yet another embodiment of the present disclosure further provides a computer program product, including a computer program, the computer program is executed by a processor to implement the data processing method according to any of the above embodiments.

[0103] Figure 7 The electronic device provided in the fifth embodiment of the present disclosure is shown in the structural schematic diagram. The device can be a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0104] The device 700 can include one or more of the following components: a processing component 702, a memory 704, a power supply component 706, a multimedia component 708, an audio component 710, an input / output (I / O) interface 712, a sensor component 714, and a communication component 716.

[0105] The processing component 702 usually controls the overall operation of the device 700, such as operations associated with displaying, making phone calls, data communications, camera operations and recording operations. The processing component 702 can include one or more processors 720 to execute instructions to complete all or part of the steps of the methods described above. In addition, the processing component 702 can include one or more modules to facilitate interaction between the processing component 702 and other components. For example, the processing component 702 can include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0106] The memory 704 is configured to store various types of data to support operations of the device 700. Examples of such data include instructions for any application or methods operating on the device 700, contact data, phonebook data, messages, pictures, videos, and so on. The memory 704 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0107] The power supply component 706 supplies electrical power for various components of the device 700. The power supply component 706 can include a power supply management system, one or more power supplies, and other components associated with generating, managing, and distributing electrical power for the device 700.

[0108] The multimedia component 708 includes a screen providing an output interface between the device 700 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping, and gestures on the touch panel. The touch sensors can not only sense a boundary of a touch or swiping action, but also detect duration and pressure associated with the touch or swiping action. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. The front and / or rear camera can receive external multimedia data when the device 700 is in an operating mode, such as a shooting mode or a video mode. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0109] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC) that is configured to receive external audio signals when the device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 704 or transmitted via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0110] The I / O interface 712 provides an interface between the processing component 702 and peripheral interface modules, which can be a keyboard, a click wheel, buttons, and so on. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0111] The sensor component 714 includes one or more sensors for providing status assessments for various aspects of the device 700. For example, the sensor component 714 can detect an open / closed position of the device 700, relative positioning of components, such as a display and keypad of the device 700, changes in position of the device 700 or a component of the device 700, presence or absence of user contact with the device 700, orientation or acceleration / deceleration of the device 700, and temperature changes of the device 700. The sensor component 714 can include proximity sensor(s) configured to detect presence of nearby objects without any physical contact. The sensor component 714 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 714 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0112] The communication component 716 is configured to facilitate wired or wireless communication between the device 700 and another device. The device 700 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technology.

[0113] In an exemplary embodiment, the device 700 can be implemented using one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, or other electronic units to perform the above-described methods.

[0114] In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 704 including instructions, is also provided, which can be executed by the processor 720 of the device 700 to perform the above-described methods. For example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0115] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the disclosure be construed as including any paterns of this disclosure that can be derived from the description and illustrations presented herein without departing from the scope and spirit of the disclosure. The specification and examples are exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

[0116] It should be understood that the present disclosure is not limited to the precise structures as herein described and illustrated in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. A data processing method, characterized by, The method comprises the following steps: obtaining at least one piece of to-be-shunted data information and to-be-shunted calculation proportion information, wherein the to-be-shunted data comprises identification information and a storage location of each shunting node, and the to-be-shunted calculation proportion information comprises a shunting proportion of each shunting node; generating at least one shunting rule tree diagram according to the to-be-shunted calculation proportion information and a preset configuration proportion, wherein a node in the shunting rule tree diagram is the identification information of a shunting node, and an edge in the shunting rule tree diagram is the shunting proportion of each shunting node; for each node in the shunting rule tree diagram, obtaining shunting data from a storage location corresponding to the identification information of a shunting node corresponding to the node, and performing data shunting calculation on the shunting data according to the shunting proportion and the configuration proportion to obtain a shunting result corresponding to each shunting node; generating a shunting result model according to the shunting result corresponding to each shunting node; the data shunting calculation according to the shunting proportion and the configuration proportion to obtain the shunting result corresponding to each shunting node comprises: multiplying the shunting data by the shunting proportion and the configuration proportion for each shunting node to obtain the shunting result corresponding to each shunting node.

2. The method of claim 1, wherein, after the step of obtaining at least one piece of to-be-shunted data information and to-be-shunted calculation proportion information, the method further comprises the following steps: obtaining corresponding shunting data from a storage location corresponding to each shunting node according to the to-be-shunted data information; generating a shunting data source model in a key-value storage structure according to the shunting data; after the step of generating at least one shunting rule tree diagram according to the to-be-shunted calculation proportion information and a preset configuration proportion, the method further comprises the following step: for each node in the shunting rule tree diagram, obtaining shunting data corresponding to a shunting node from the shunting data source model according to a data identifier of the shunting node corresponding to the node.

3. The method of claim 1, wherein, the step of generating a shunting result model according to the shunting result corresponding to each shunting node comprises: generating a shunting result model in a key-value storage structure according to the shunting result corresponding to each shunting node.

4. The method according to any one of claims 1 to 3, characterized in that, the step of, for each node in the shunting rule tree diagram, obtaining shunting data from a storage location corresponding to the identification information of a shunting node corresponding to the node, and performing data shunting calculation on the shunting data according to the shunting proportion and the configuration proportion to obtain a shunting result corresponding to each shunting node, comprises the following steps: obtaining a preset shunting node calculation rule, wherein the shunting node calculation rule comprises a number of threads / processes for concurrent processing of a shunting node and a number of jobs processed by each thread / process; determining a product of the number of threads / processes and the number of jobs as a target number of shunting nodes to be concurrently processed; concurrently performing data shunting calculation on the target number of shunting nodes to obtain a shunting result corresponding to each shunting node.

5. The method of claim 4, wherein, before the step of concurrently performing data shunting calculation on the target number of shunting nodes, the method further comprises the following steps: determining the target number of shunting nodes according to a hierarchical order of shunting nodes in the shunting rule tree diagram; sorting the target number of shunting nodes according to the hierarchical order of shunting nodes in the shunting rule tree diagram.

6. The method of claim 4, wherein, The concurrent data shunting calculation on the target number of shunting nodes comprises: For each target shunting node corresponding to a job, detecting whether a parent node corresponding to the target shunting node completes data shunting calculation; If yes, obtaining target shunting data from a storage location corresponding to identification information of the target shunting node, and performing data shunting calculation on the target shunting data according to the shunting ratio and the configuration ratio to obtain a shunting result corresponding to the target shunting node; If no, not performing data shunting calculation on the target shunting node.

7. The method according to any one of claims 1 to 3, characterized in that, After the at least one shunting rule tree diagram is generated according to the to-be-shunted calculation ratio information and the preset configuration ratio, the method further comprises: In response to a modification instruction triggered by a user, wherein the modification instruction comprises identification information of a node and a configuration ratio corresponding to the node; According to the modification instruction, modifying the configuration ratio of the node in the shunting rule tree diagram.

8. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: Obtaining an update instruction triggered by a user, wherein the update instruction comprises a storage location and to-be-updated content; According to the update instruction, updating the to-be-updated content to the storage location.

9. A data processing apparatus, characterized by, The method comprises: An obtaining module, configured to obtain at least one to-be-shunted data information and to-be-shunted calculation ratio information, wherein the to-be-shunted data comprises identification information and a storage location of at least one shunting node, and the to-be-shunted calculation ratio information comprises a shunting ratio of each shunting node; A generating module, configured to generate at least one shunting rule tree diagram according to the to-be-shunted calculation ratio information and a preset configuration ratio, wherein a node in the shunting rule tree diagram is identification information of a shunting node, and an edge in the shunting rule tree diagram is a shunting ratio of each shunting node; A calculating module, configured to, for each node in the shunting rule tree diagram, obtain shunting data from a storage location corresponding to identification information of a shunting node corresponding to the node, and perform data shunting calculation on the shunting data according to the shunting ratio and the configuration ratio to obtain a shunting result corresponding to each shunting node; A processing module, configured to generate a shunting result model according to the shunting result corresponding to each shunting node; The calculating module is configured to, for each shunting node, multiply the shunting data by the shunting ratio and the configuration ratio to obtain the shunting result corresponding to each shunting node.

10. An electronic device, comprising: The method comprises: A memory and a processor; A memory; A memory for storing executable instructions of the processor; The processor is configured to invoke program instructions in the memory to execute the data processing method in any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer-readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the data processing method in any one of claims 1-8.

12. A computer program product, characterised in that, The computer program is executed by the processor to implement the data processing method in any one of claims 1-8.

Images