Data apportionment method and apparatus

Abstract

The application discloses a data apportionment method and device. The method comprises the following steps: obtaining original data to be apportioned and reference data as an apportionment basis; converting the original data and the reference data into an original data set and a reference data set in a key-value storage format respectively, wherein the original data set comprises a first key group set and a first value group set corresponding to the first key group set, and the reference data set comprises a second key group set and a second value group corresponding to the second key group set; fusing the first key group set and the second key group set to obtain a third key group set, determining an apportionment ratio based on the second value group, performing apportionment calculation on the first value group set based on the apportionment ratio, and obtaining a third value group set corresponding to the third key group set; and determining a data apportionment result based on the third key group set and the third value group set. The application solves the technical problems of poor generality and tail difference of a data apportionment method in the related art.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data sharing method and device. Background Art

[0002] During the design and development of IT systems, it is often necessary to adopt specific allocation rules based on business needs and to allocate data according to the specified allocation reference data ratio. The relevant data allocation technical solutions mainly design corresponding algorithms for specific scenarios and use relational data models to perform data splitting and calculation processing. However, with the refinement of original data and the diversification of allocation rules, the relevant data allocation technical solutions have become difficult to meet the needs of fine-grained allocation and have poor versatility.

[0003] At the same time, most of the rounding methods are used when allocating values. This method will cause the results to have tail differences, which will lead to the problem of discrepancy between the scattered and the total. The relevant technology only uses the method of increasing the number of digits retained after the decimal point of the value to reduce the tail differences. However, this method can only reduce the probability of tail differences and cannot completely avoid the situation where manual intervention is required to adjust the tail differences.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The embodiments of the present application provide a data sharing method and device to at least solve the technical problems in related technologies that data sharing methods have poor versatility and are prone to tail errors in results.

[0006] According to one aspect of an embodiment of the present application, a data allocation method is provided, including: obtaining original data to be allocated and reference data serving as a basis for allocation; converting the original data and the reference data into an original data set and a reference data set in a key-value storage format, respectively, wherein the original data set includes a first key group set and a first value group set corresponding to the first key group set, and the reference data set includes a second key group set and a second value group corresponding to the second key group set; fusing the first key group set and the second key group set to obtain a third key group set, and determining an allocation ratio based on the second value group, performing an allocation calculation on the first value group set based on the allocation ratio to obtain a third value group set corresponding to the third key group set; and determining a data allocation result based on the third key group set and the third value group set.

[0007] Optionally, the first key group set includes at least one column of first key group, wherein each column of first key group includes multiple rows of first keys of the same key attribute; the first value group set includes at least one column of first value group, wherein each column of first value group includes multiple rows of first values ​​corresponding to multiple rows of first keys of the same value attribute; the second key group set includes at least one column of second key group, wherein each column of second key group includes multiple rows of second keys of the same key attribute; the second value group is a column of second value group, and the second value group includes multiple rows of second values ​​corresponding to multiple rows of second keys of the same value attribute.

[0008] Optionally, the first key group set and the second key group set are merged to obtain a third key group set, including: determining a grouped key group set, a fixed key group set and an allocated key group set based on the first key group set and the second key group set, respectively, wherein the grouped key group set is the intersection of the first key group set and the second key group set, the fixed key group set is the set of unique first key groups in the first key group set, and the allocated key group set is the set of unique second key groups in the second key group set; when both the grouped key group set and the allocated key group set are not empty, the first key group set and the second key group set are merged based on the grouped key group set to obtain the third key group set.

[0009] Optionally, the first key group set and the second key group set are merged based on the grouped key group set to obtain a third key group set, including: grouping multiple rows of second keys in the second key group set based on the second key in the grouped key group set to obtain multiple groups of second key row sets, wherein each group of second key row sets includes at least one row of second keys, and each row of second keys corresponds to a second value; grouping multiple rows of first keys in the first key group set based on the first key in the grouped key group set to obtain multiple groups of first key row sets, wherein each group of first key row sets corresponds to a row of first values; for each group of first key row sets, the second key row sets corresponding to the second key that is the same as the first key corresponding to the first key row set are merged into the first key row set to obtain a third key row set, wherein the number of rows in the third key row set is the same as the number of rows in the second key row set, and each row of third keys in the third key row set corresponds to a row of third values; all third key row sets are used as the third key group set.

[0010] Optionally, the allocation ratio is determined based on the second value group, and the first value group set is allocated and calculated based on the allocation ratio to obtain a third value group set corresponding to the third key group set, including: for each group of third key row sets, determining that the third key of any row in the third key row set is the fourth key row, and the third key of each other row in the third key row set is the fifth key row; calculating the first sum value of the second values ​​corresponding to the third key of each row in the third key row set; for each fifth key row, determining the allocation ratio corresponding to the fifth key row based on the second value corresponding to the fifth key row and the first sum value, and calculating the third values ​​of each column corresponding to the fifth key row based on the first value of each column corresponding to the third key row set and the allocation ratio; summing the third values ​​of each column corresponding to all fifth key rows by column to obtain the second sum value of multiple columns; subtracting the first value of each column from the second sum value of multiple columns by column to obtain the third value of each column corresponding to the fourth key row; and taking all third values ​​as the third value group set.

[0011] Optionally, for any row of third keys in the third key row set, when there is a third value less than a preset threshold among the third values ​​of each column corresponding to the row of third keys, the row of third keys is deleted from the third key row set.

[0012] Optionally, when the grouping key set and / or the allocation key set is empty, data allocation is stopped, and the original data set is directly used as the data allocation result.

[0013] According to another aspect of an embodiment of the present application, a data sharing device is also provided, including: an acquisition module for acquiring original data to be shared and reference data serving as a basis for sharing; a conversion module for converting the original data and the reference data into an original data set and a reference data set in a key-value storage format, respectively, wherein the original data set includes a first key group set and a first value group set corresponding to the first key group set, and the reference data set includes a second key group set and a second value group corresponding to the second key group set; an sharing module for fusing the first key group set and the second key group set to obtain a third key group set, and determining a sharing ratio based on the second value group, performing sharing calculation on the first value group set based on the sharing ratio to obtain a third value group set corresponding to the third key group set; a determination module for determining a data sharing result based on the third key group set and the third value group set.

[0014] According to another aspect of an embodiment of the present application, a non-volatile storage medium is further provided, which includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above-mentioned data sharing method.

[0015] According to another aspect of an embodiment of the present application, an electronic device is further provided, which includes: a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the above-mentioned data sharing method through the computer program.

[0016] In an embodiment of the present application, the original data to be apportioned and the reference data used as the basis for apportionment are obtained; the original data and the reference data are respectively converted into an original data set and a reference data set in a key-value storage format, wherein the original data set includes a first key group set and a first value group set corresponding to the first key group set, and the reference data set includes a second key group set and a second value group corresponding to the second key group set; the first key group set and the second key group set are merged to obtain a third key group set, and the apportionment ratio is determined based on the second value group, and the first value group set is apportioned based on the apportionment ratio to obtain a third value group set corresponding to the third key group set; the data apportionment result is determined based on the third key group set and the third value group set. The above method can realize numerical apportionment according to a specific apportionment rule and a specified apportionment reference data ratio, and can solve the problem of tail difference caused by rounding off after numerical apportionment due to retaining decimal places, thereby ensuring that the numerical distribution before and after apportionment is consistent, thereby solving the technical problem that the data apportionment method in the related art is poor in versatility and the results are prone to tail difference. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 This is a flow chart of an optional data sharing method according to an embodiment of the present application;

[0019] Figure 2 This is a flow chart of an optional data sharing method according to an embodiment of the present application.

[0020] Figure 3 It is a structural diagram of an optional data sharing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0022] Example 1

[0023] During the design and development of IT systems, such as in professional fields like billing and finance, it is often necessary to allocate data based on business needs using specific allocation rules and designated allocation reference data ratios. The relevant data allocation technology solutions primarily design algorithms for specific scenarios and utilize relational data models for data splitting and calculation. This solution has poor versatility and cannot be applied to multiple scenarios. Furthermore, most numerical allocations use rounding, which can result in discrepancies in the results and lead to discrepancies between the total and the total. The relevant solution simply reduces the discrepancy by increasing the number of decimal places. However, this method can only reduce the probability of discrepancies and cannot completely avoid the need for manual intervention to adjust the discrepancies.

[0024] In order to solve the above problems, an embodiment of the present application provides a data sharing method, which not only completely eliminates the tail difference, but also ensures that the data sharing method can be applied to various existing scenarios involving numerical sharing, thereby greatly improving the efficiency of IT system design and development.

[0025] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a list of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] Figure 1 FIG. 1 is a flow chart of an optional data sharing method according to an embodiment of the present application. Figure 1 As shown, the method includes at least steps S102-S108, wherein:

[0027] Step S102: obtaining original data to be allocated and reference data serving as a basis for allocation.

[0028] Step S104, converting the original data and the reference data into an original data set and a reference data set in a key-value storage format, respectively, wherein the original data set includes a first key group set and a first value group set corresponding to the first key group set, and the reference data set includes a second key group set and a second value group corresponding to the second key group set.

[0029] As an optional implementation, the embodiment of the present application comprehensively considers various application scenarios and combines the respective advantages of the structured data model and the key-value storage model to provide an optional data sharing storage model. The basic structure of the model is defined as follows:

[0030] Original data set S o , which can be expressed as S o (K o:V o ), where K o ={k o1 , k o2 , k o3 , ...}, is the first key group set in the original data, which can be regarded as the attribute field set in the data table. In actual scenarios, it corresponds to text or enumeration fields such as amount, region, date and time, income index, cost type, and business type; V o ={v o1 , v o2 , v o3 , …}, is the first value group set in the original data, which can be regarded as the numeric field set in the data table. In actual application scenarios, it corresponds to monetary or numeric fields such as accounting income, taxable income, price, tax, and cost, and is a numeric object that needs to be split and calculated.

[0031] Reference data set S r , which can be expressed as S r (K r :v), where K r ={k r1 , k r2 , k r3 , …}, is the second key group set in the reference data used as the basis for allocation, which can be regarded as the attribute field set in the data table. In actual scenarios, it corresponds to text or enumeration fields such as amount, region, date and time, income indicator, cost type, and business type; the second value group v is the only numeric column in the reference data set and is used to calculate the allocation ratio.

[0032] Fixed key set K f =K o -K r , is the original data set S o The set of unique keys in the metric that do not directly participate in the amortization calculation.

[0033] Grouping key set K g =K o ∩K r , is the original data set S o and the reference data set S r The set of common keys is used as a column group for grouping and aggregation to participate in the allocation calculation. When , it represents the original data set S o and the reference data set S r There is no common key, so no amortization calculation is required.

[0034] The set of allocated key groups K s =K r -K o , is the reference data set S rThe set of unique keys in the original data set S is used to split o The key columns added later participate in the allocation calculation. , it means that no new key is added during the allocation, and no allocation operation is required.

[0035] Amortized minimum v min , is a pre-set non-negative value used to control the degree of refinement in the apportionment calculation, avoiding unnecessary system performance overhead and result dataset storage overhead caused by over-refinement. That is, if the absolute value of a value after the apportionment calculation is less than the minimum value, the value will be discarded.

[0036] Result data set S d , which can be expressed as S d (K d :V d ), where K d is the third key row set in the result data set, which is the original data set S o The first key group set K o and the reference data set S r The union of the second key group set K, that is, K d =K o ∪K r =K f +K g +K s ; V d It is the third value group set in the result data set, which is the numerical column set obtained by the apportionment calculation. The third value group set V d The set of columns and first value groups V o The columns are one-to-one corresponding, and the sum of the values ​​in the corresponding columns is equal.

[0037] Based on the defined data allocation storage model, the acquired original data and reference data can be converted into an original data set and a reference data set in corresponding key-value storage formats, respectively.

[0038] Specifically, the first key group set includes at least one column of first key group, wherein each column of first key group includes multiple rows of first keys of the same key attribute; the first value group set includes at least one column of first value group, wherein each column of first value group includes multiple rows of first values ​​of the same value attribute corresponding to multiple rows of first keys; the second key group set includes at least one column of second key group, wherein each column of second key group includes multiple rows of second keys of the same key attribute; the second value group is a column of second value group, and the second value group includes multiple rows of second values ​​of the same value attribute corresponding to multiple rows of second keys.

[0039] Step S106: merge the first key group set and the second key group set to obtain a third key group set, determine the allocation ratio based on the second value group, and perform allocation calculation on the first value group set based on the allocation ratio to obtain a third value group set corresponding to the third key group set.

[0040] As an optional implementation, the third key group set can be obtained in the following manner: based on the first key group set and the second key group set, respectively determine the grouped key group set, the fixed key group set and the allocated key group set, wherein the grouped key group set is the intersection of the first key group set and the second key group set, the fixed key group set is the set of unique first key groups in the first key group set, and the allocated key group set is the set of unique second key groups in the second key group set; when both the grouped key group set and the allocated key group set are not empty, the first key group set and the second key group set are merged based on the grouped key group set to obtain the third key group set.

[0041] In which, during the merging process, multiple rows of second keys in the second key group set can be grouped based on the second key in the grouping key group set to obtain multiple groups of second key row sets, wherein each group of second key row sets includes at least one row of second key, and each row of second key corresponds to a second value; multiple rows of first keys in the first key group set are grouped based on the first key in the grouping key group set to obtain multiple groups of first key row sets, wherein each group of first key row sets corresponds to a row of first value; for each group of first key row sets, the second key row sets corresponding to the second key that is the same as the first key corresponding to the first key row set are merged into the first key row set to obtain a third key row set, wherein the number of rows in the third key row set is the same as the number of rows in the second key row set, and each row of third key in the third key row set corresponds to a row of third value; all third key row sets are used as third key group sets.

[0042] Optionally, when the grouping key set and / or the allocation key set is empty, data allocation is stopped, and the original data set is directly used as the data allocation result.

[0043] After obtaining the third key group set, the third value group set corresponding to the third key group set can be obtained in the following manner: for each group of third key row sets, determine that the third key of any row in the third key row set is the fourth key row, and the third keys of all other rows in the third key row set are the fifth key row; calculate the first sum value of the second values ​​corresponding to the third key of each row in the third key row set; for each fifth key row, determine the apportionment ratio corresponding to the fifth key row based on the second value corresponding to the fifth key row and the first sum value, and calculate the third values ​​of each column corresponding to the fifth key row based on the first values ​​of each column corresponding to the third key row set and the apportionment ratio; sum the third values ​​of each column corresponding to all fifth key rows by column to obtain the second sum value of multiple columns; find the difference between the first value of each column and the second sum value of multiple columns by column to obtain the third value of each column corresponding to the fourth key row; and take all third values ​​as the third value group set.

[0044] In the above process of determining the third value group set, for any row of third keys in the third key row set, when there is a third value less than a preset threshold in the third values ​​of each column corresponding to the third key in the row, the third key in the row is deleted from the third key row set.

[0045] Step S108: Determine a data allocation result based on the third key group set and the third value group set.

[0046] Specifically, the result data set corresponding to the third key group set and the third value group set is converted into corresponding relational data to obtain the final data allocation result.

[0047] Figure 2 The schematic diagram of a complete data sharing process is shown, wherein the original data and reference data are first converted into the original data set S in the key-value storage format. o and the reference data set S r , and set the minimum allocation value v min ; Then define the numerical amortization calculation algorithm P, which can be expressed as S d =P(S o , S r , v min ), the specific process is as follows:

[0048] Based on the original data set S o The first key group set K in o and the reference data set S r The second key set K in r Determine the group key set K respectively g , fixed key set K f and the amortized key set K s ;

[0049] Determine the group key set K gand the amortized key set K s Is it empty? If the group key set K g and / or the allocated key set K s When it is empty, there is no need to perform data allocation, and the original data is directly combined as the data allocation result;

[0050] If the grouping key set K g and / or the allocated key set K s When it is not empty, the reference data set S r According to the group key group set K g The second key pair in the second key group set K r The multiple rows of second keys in the , are grouped to obtain multiple sets of second key rows, where each set of second key rows can be recorded as K r [k i ], k i K g A tuple in the original data set S o According to the group key group set K g The first key pair first key group set K in o The first keys of multiple rows in the , are grouped to obtain multiple sets of first key rows, where each set of first key rows can be recorded as K o [k i ], k i The same is K g A tuple in ;

[0051] For each set of first key rows K r [k i ], the second key row set K corresponding to the second key that is the same as the first key corresponding to the first key row set o [k i ]Merge to the first key row set K r [k i ], get the third key row set K d [k i ]=(K f [k i ], K g [k i ], K s [k i ]); traverse each set of first key rows K r [k i ], get the third key group set K d ;

[0052] For each set of third key rows K d [k i ], determine the third key in any row is the fourth key row k n, the third key of each other row is the fifth key row k m , calculate the third key row set K d [k i The first sum value ∑v[k i ]; For each row of the fifth key row k m , based on the fifth key row k m The corresponding second value v[k m ] and the first sum value ∑v[k i ] Determine the allocation ratio corresponding to the fifth key row And based on the third key row set K d [k i ]The first value v of each column corresponding to o [k i ] and the apportionment ratio s m Calculate the fifth key row k m The corresponding third value of each column: v d [k m ]=v o [k i ]*s m .

[0053] If all the fifth key rows k calculated m The corresponding third value v in each column d [k m ] are greater than the minimum allocation value v min Condition, for all fifth key rows k m The corresponding third value v in each column d [k m ]Sum by column to get the second sum value of multiple columns ∑v d [k m ], for each column first value v o [k i ] and the second sum value of multiple columns ∑v d [k m ] Calculate the difference by column and get the fourth key row k n The corresponding third value v in each column d [k n ]=v o [k i ]-∑v d [k m ]; traverse each set of third key rows K d [k i ], get the third value group set v d .

[0054] Based on the above data allocation method, the data allocation method in Example 1 is described in detail by taking the scenario of allocating business revenue from cities to counties as an example.

[0055] Original data set S o As shown in Table 1.

[0056] Table 1

[0057]

[0058] Reference data set S r As shown in Table 2.

[0059] Table 2

[0060]

[0061] Set the minimum amortization value v min Set it to 1.00 yuan and start the numerical allocation calculation:

[0062] Determine the group key set K g ={"region","amount"}, fixed key set K f = {"category","tax rate"}, and the allocation key set K s = {"county", "business hall"}, rearrange the order of the original data set and the reference data set to obtain the new format S' o and S′ r , S′ o ={{"category","tax rate"},{"region","amount"}:{"price","tax"}},S′ r ={{"Region","Amount"},{"County","Business Hall"}:"Amount"}.

[0063] Next, the reference data set S r According to K g = {"region","amount"} to obtain multiple sets of second key row sets K r [k i ], such as k i =[Province A, one-time income] is one of the groups; for the original data set S o According to K g = {"region","amount"} to group, and obtain multiple sets of first key row sets K o [k i ], such as k i =[Province A, one-time income] is one of the groups. For each group of the first key row set K o [k i ], will be combined with the first key row set K o [k i ] The second key row set K corresponding to the second key with the same first key r [ki ]Merge to the first key row set K o [k i ], get the third key row set K d [k i ], a set of third key rows K d [k i ]As shown in Table 3.

[0064] Let {"sale of goods", "11%"}, {"Province A", "one-time income"}, {"County A", "Business Hall A"} be the fourth key row, then {"sale of goods", "11%"}, {"Province A", "one-time income"}, {"County B", "Business Hall B"} and {"sale of goods", "11%"}, {"Province A", "one-time income"}, {"County B", "Business Hall C"} be the fifth key row, and calculate the third value corresponding to the fifth key row {"sale of goods", "11%%"}, {"Province A", "one-time income"}, {"County B", "Business Hall B"} respectively. The third value corresponding to the fifth key row {"Selling Goods", "11%"}, {"Province A", "One-time Income"}, {"County B", "Business Hall C"} is The third value corresponding to the fourth key row {"sales of goods", "11%"}, {"Province A", "one-time income"}, {"County A", "Business Hall A"} is [890-84.34-519.46, 110-10.42-64.20]=[286.20, 35.38]. The final data allocation results are shown in Table 3.

[0065] Table 3

[0066]

[0067] In an embodiment of the present application, the original data to be apportioned and the reference data used as the basis for apportionment are obtained; the original data and the reference data are respectively converted into an original data set and a reference data set in a key-value storage format, wherein the original data set includes a first key group set and a first value group set corresponding to the first key group set, and the reference data set includes a second key group set and a second value group corresponding to the second key group set; the first key group set and the second key group set are merged to obtain a third key group set, and the apportionment ratio is determined based on the second value group, and the first value group set is apportioned based on the apportionment ratio to obtain a third value group set corresponding to the third key group set; the data apportionment result is determined based on the third key group set and the third value group set. The above method can realize numerical apportionment according to a specific apportionment rule and a specified apportionment reference data ratio, and can solve the problem of tail difference caused by rounding off after numerical apportionment due to retaining decimal places, thereby ensuring that the numerical distribution before and after apportionment is consistent, thereby solving the technical problem that the data apportionment method in the related art is poor in versatility and the results are prone to tail difference.

[0068] Example 2

[0069] According to an embodiment of the present application, a data sharing device for implementing the above data sharing method is also provided. Figure 3 As shown, the device at least includes an acquisition module 31, a conversion module 32, an allocation module 33 and a determination module 34, wherein:

[0070] The acquisition module 31 is used to acquire the original data to be allocated and the reference data used as the basis for allocation.

[0071] The conversion module 32 is used to convert the original data and the reference data into an original data set and a reference data set in a key-value storage format, respectively, wherein the original data set includes a first key group set and a first value group set corresponding to the first key group set, and the reference data set includes a second key group set and a second value group corresponding to the second key group set.

[0072] As an optional implementation, the embodiment of the present application comprehensively considers various application scenarios and combines the respective advantages of the structured data model and the key-value storage model to provide an optional data sharing storage model. The basic structure of the model is defined as follows:

[0073] Original data set S o , which can be expressed as S o (K o :V o ), where K o ={k o1 , k o2 , k o3, ...}, is the first key group set in the original data, which can be regarded as the attribute field set in the data table. In actual scenarios, it corresponds to text or enumeration fields such as amount, region, date and time, income index, cost type, and business type; V o ={v o1 , v o2 , v o3 , …}, is the first value group set in the original data, which can be regarded as the numeric field set in the data table. In actual application scenarios, it corresponds to monetary or numeric fields such as accounting income, taxable income, price, tax, and cost, and is a numeric object that needs to be split and calculated.

[0074] Reference data set S r , which can be expressed as S r (K r :v), where K r ={k r1 , k r2 , k r3 , …}, is the second key group set in the reference data used as the basis for allocation, which can be regarded as the attribute field set in the data table. In actual scenarios, it corresponds to text or enumeration fields such as amount, region, date and time, income indicator, cost type, and business type; the second value group v is the only numeric column in the reference data set and is used to calculate the allocation ratio.

[0075] Fixed key set K f =K o -K r , is the original data set S o The set of unique keys in the metric that do not directly participate in the amortization calculation.

[0076] Grouping key set K g =K o ∩K r , is the original data set S o and the reference data set S r The set of common keys is used as a column group for grouping and aggregation to participate in the allocation calculation. When , it represents the original data set S o and the reference data set S r There is no common key, so no amortization calculation is required.

[0077] The set of allocated key groups K s =K r -K o , is the reference data set S r The set of unique keys in the original data set S is used to split o The key columns added later participate in the allocation calculation. , it means that no new key is added during the allocation, and no allocation operation is required.

[0078] The amortized minimum value, vmin, is a pre-set non-negative value used to control the degree of refinement in the amortized calculation, avoiding unnecessary system performance overhead and result dataset storage overhead caused by over-refinement. Specifically, if the absolute value of a value after amortization calculation is less than the amortized minimum value, the value is discarded.

[0079] Result data set S d , which can be expressed as S d (K d :V d ), where K d is the third key row set in the result data set, which is the original data set S o The first key group set K o and the reference data set S r The second key group set K r The union of K d =K o ∪K r =K f +K g +K s ; V d It is the third value group set in the result data set, which is the numerical column set obtained by the apportionment calculation. The third value group set V d The set of columns and first value groups V o The columns are one-to-one corresponding, and the sum of the values ​​in the corresponding columns is equal.

[0080] Based on the defined data distribution storage model, the conversion module can convert the acquired original data and reference data into an original data set and a reference data set in corresponding key-value storage formats, respectively.

[0081] Specifically, the first key group set includes at least one column of first key group, wherein each column of first key group includes multiple rows of first keys of the same key attribute; the first value group set includes at least one column of first value group, wherein each column of first value group includes multiple rows of first values ​​of the same value attribute corresponding to multiple rows of first keys; the second key group set includes at least one column of second key group, wherein each column of second key group includes multiple rows of second keys of the same key attribute; the second value group is a column of second value group, and the second value group includes multiple rows of second values ​​of the same value attribute corresponding to multiple rows of second keys.

[0082] The allocation module 33 is used to merge the first key group set and the second key group set to obtain a third key group set, determine the allocation ratio based on the second value group, and perform allocation calculation on the first value group set based on the allocation ratio to obtain a third value group set corresponding to the third key group set.

[0083] As an optional implementation, the allocation module can obtain the third key group set in the following manner: based on the first key group set and the second key group set, respectively determine the grouped key group set, the fixed key group set and the allocated key group set, wherein the grouped key group set is the intersection of the first key group set and the second key group set, the fixed key group set is the set of unique first key groups in the first key group set, and the allocated key group set is the set of unique second key groups in the second key group set; when both the grouped key group set and the allocated key group set are not empty, the first key group set and the second key group set are merged based on the grouped key group set to obtain the third key group set.

[0084] In which, during the merging process, multiple rows of second keys in the second key group set can be grouped based on the second key in the grouping key group set to obtain multiple groups of second key row sets, wherein each group of second key row sets includes at least one row of second key, and each row of second key corresponds to a second value; multiple rows of first keys in the first key group set are grouped based on the first key in the grouping key group set to obtain multiple groups of first key row sets, wherein each group of first key row sets corresponds to a row of first value; for each group of first key row sets, the second key row sets corresponding to the second key that is the same as the first key corresponding to the first key row set are merged into the first key row set to obtain a third key row set, wherein the number of rows in the third key row set is the same as the number of rows in the second key row set, and each row of third key in the third key row set corresponds to a row of third value; all third key row sets are used as third key group sets.

[0085] Optionally, when the grouping key set and / or the allocation key set is empty, data allocation is stopped, and the original data set is directly used as the data allocation result.

[0086] After obtaining the third key group set, the allocation module can obtain the third value group set corresponding to the third key group set in the following manner: for each group of third key row sets, determine that the third key of any row in the third key row set is the fourth key row, and the third keys of all other rows in the third key row set are the fifth key row; calculate the first sum value of the second values ​​corresponding to the third keys of each row in the third key row set; for each fifth key row, determine the allocation ratio corresponding to the fifth key row based on the second value corresponding to the fifth key row and the first sum value, and calculate the third values ​​of each column corresponding to the fifth key row based on the first values ​​of each column corresponding to the third key row set and the allocation ratio; sum the third values ​​of each column corresponding to all fifth key rows by column to obtain the second sum value of multiple columns; find the difference between the first value of each column and the second sum value of multiple columns by column to obtain the third value of each column corresponding to the fourth key row; and take all third values ​​as the third value group set.

[0087] In the above process of determining the third value group set, for any row of third keys in the third key row set, when there is a third value less than a preset threshold in the third values ​​of each column corresponding to the third key in the row, the third key in the row is deleted from the third key row set.

[0088] The determination module 34 is configured to determine a data allocation result based on the third key group set and the third value group set.

[0089] Specifically, the determination module may convert the result data set corresponding to the third key group set and the third value group set into corresponding relational data, that is, obtain the final data allocation result.

[0090] It should be noted that each module in the data sharing device in the embodiment of the present application corresponds one-to-one to each implementation step of the data sharing method in Example 1. Since a detailed description has been given in Example 1, some details not reflected in this embodiment can be referred to Example 1 and will not be elaborated here.

[0091] Example 3

[0092] According to an embodiment of the present application, a non-volatile storage medium is also provided, which includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the data sharing method in Example 1.

[0093] According to an embodiment of the present application, a processor is also provided, which is used to run a program, wherein the data sharing method in Example 1 is executed when the program is running.

[0094] According to an embodiment of the present application, an electronic device is also provided, which includes: a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the data sharing method in Example 1 through the computer program.

[0095] Optionally, when the program is running, the following steps are executed: obtaining original data to be apportioned and reference data serving as a basis for apportionment; converting the original data and the reference data into an original data set and a reference data set in a key-value storage format, respectively, wherein the original data set includes a first key group set and a first value group set corresponding to the first key group set, and the reference data set includes a second key group set and a second value group corresponding to the second key group set; fusing the first key group set and the second key group set to obtain a third key group set, and determining an apportionment ratio based on the second value group, performing an apportionment calculation on the first value group set based on the apportionment ratio to obtain a third value group set corresponding to the third key group set; and determining a data apportionment result based on the third key group set and the third value group set.

[0096] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0097] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0098] The terms "first", "second", etc. in the specification, claims, and drawings of the present application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, and can be electrical or other forms.

[0100] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0101] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0102] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code.

[0103] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data sharing method, characterized in that: include: Obtain the original data to be allocated and the reference data used as the basis for allocation; Convert the original data and the reference data into an original data set and a reference data set in a key-value storage format, respectively, wherein the original data set includes a first key group set and a first value group set corresponding to the first key group set, and the reference data set includes a second key group set and a second value group corresponding to the second key group set, wherein the first key group set includes at least one column of first key group, and each column of the first key group includes multiple rows of first keys with the same key attribute; the first value group set includes at least one column of first value group, and each column of the first value group includes multiple rows of first values ​​with the same value attribute corresponding to the multiple rows of first keys; the second key group set includes at least one column of second key group, and each column of the second key group includes multiple rows of second keys with the same key attribute; the second value group is a column of second value group, and the second value group includes multiple rows of second values ​​with the same value attribute corresponding to the multiple rows of second keys; The first key group set and the second key group set are merged to obtain a third key group set, and an allocation ratio is determined based on the second value group. The first value group set is allocated based on the allocation ratio to obtain a third value group set corresponding to the third key group set, including: determining the intersection of the first key group set and the second key group set as a grouped key group set; grouping multiple rows of second keys in the second key group set based on the second key in the grouped key group set to obtain multiple groups of second key row sets, each group of the second key row set includes at least one row of second keys, and each row of second keys corresponds to a second value; grouping multiple rows of first keys in the first key group set based on the first key in the grouped key group set to obtain multiple groups of first key row sets, each group of the first key row set corresponds to a row of first value; for each group of the first key row sets, the second key row sets corresponding to the second key that is the same as the first key corresponding to the first key row set are merged into the first key row set to obtain a third key row set, the third key row set The number of rows in the combination is the same as the number of rows in the second key row set, and each row of the third key in the third key row set corresponds to a row of third values; all the third key row sets are used as the third key group set; for each group of the third key row sets, determine that any row of the third key in the third key row set is the fourth key row, and each other row of the third key in the third key row set is the fifth key row; calculate the first sum value of the second values ​​corresponding to the third key of each row in the third key row set; for each row of the fifth key row, determine the allocation ratio corresponding to the fifth key row based on the second value corresponding to the fifth key row and the first sum value, and calculate the third values ​​of each column corresponding to the fifth key row based on the first value of each column corresponding to the third key row set and the allocation ratio; sum the third values ​​of each column corresponding to all the fifth key rows by column to obtain the second sum value of multiple columns; subtract the first value of each column from the second sum value of multiple columns by column to obtain the third value of each column corresponding to the fourth key row; use all the third values ​​as the third value group set; A data allocation result is determined based on the third key group set and the third value group set.

2. The method according to claim 1, characterized in that Merging the first key group set and the second key group set to obtain a third key group set includes: Determining the grouped key group set, the fixed key group set, and the allocated key group set based on the first key group set and the second key group set, respectively, wherein the fixed key group set is a set of unique first key groups in the first key group set, and the allocated key group set is a set of unique second key groups in the second key group set; When both the grouping key group set and the apportionment key group set are not empty, the first key group set and the second key group set are merged based on the grouping key group set to obtain the third key group set.

3. The method according to claim 1, characterized in that include: For any row of third keys in the third key row set, when there is a third value less than a preset threshold in the third values ​​of each column corresponding to the row of third keys, the row of third keys is deleted from the third key row set.

4. The method according to claim 2, characterized in that The method further comprises: When the grouping key set and / or the apportionment key set is empty, data apportionment is stopped, and the original data set is directly used as the data apportionment result.

5. A data sharing device, characterized in that: include: The acquisition module is used to obtain the original data to be allocated and the reference data used as the basis for allocation; a conversion module, configured to convert the original data and the reference data into an original data set and a reference data set in a key-value storage format, respectively, wherein the original data set includes a first key group set and a first value group set corresponding to the first key group set, and the reference data set includes a second key group set and a second value group corresponding to the second key group set, wherein the first key group set includes at least one column of first key groups, each column of the first key group includes multiple rows of first keys with the same key attribute; the first value group set includes at least one column of first value groups, each column of the first value group includes multiple rows of first values ​​with the same value attribute corresponding to the multiple rows of first keys; the second key group set includes at least one column of second key groups, each column of the second key group includes multiple rows of second keys with the same key attribute; the second value group is one column of second value groups, each column of the second key group includes multiple rows of second values ​​with the same value attribute corresponding to the multiple rows of second keys; An allocation module is used to merge the first key group set and the second key group set to obtain a third key group set, determine an allocation ratio based on the second value group, and perform allocation calculation on the first value group set based on the allocation ratio to obtain a third value group set corresponding to the third key group set, including: determining the intersection of the first key group set and the second key group set as a grouped key group set; grouping multiple rows of second keys in the second key group set based on the second key in the grouped key group set to obtain multiple groups of second key row sets, each group of the second key row set includes at least one row of second keys, and each row of second keys corresponds to a second value; grouping multiple rows of first keys in the first key group set based on the first key in the grouped key group set to obtain multiple groups of first key row sets, each group of the first key row set corresponds to a row of first value; for each group of the first key row sets, merging the second key row sets corresponding to the second key that is the same as the first key corresponding to the first key row set into the first key row set to obtain a third key row set, the third The number of rows in the key row set is the same as the number of rows in the second key row set, and each row of the third key in the third key row set corresponds to a row of the third value; all the third key row sets are used as the third key group set; for each group of the third key row sets, determine that any row of the third key in the third key row set is the fourth key row, and each other row of the third key in the third key row set is the fifth key row; calculate the first sum value of the second values ​​corresponding to the third key in each row of the third key row set; for each row of the fifth key row, determine the allocation ratio corresponding to the fifth key row based on the second value corresponding to the fifth key row and the first sum value, and calculate the third values ​​of each column corresponding to the fifth key row based on the first values ​​of each column corresponding to the third key row set and the allocation ratio; sum the third values ​​of each column corresponding to all the fifth key rows by column to obtain the second sum value of multiple columns; subtract the first value of each column from the second sum value of multiple columns by column to obtain the third value of each column corresponding to the fourth key row; use all the third values ​​as the third value group set; A determination module is used to determine a data allocation result based on the third key group set and the third value group set.

6. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the data sharing method according to any one of claims 1 to 4.

7. An electronic device, characterized in that: include: A memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the data sharing method according to any one of claims 1 to 4 through the computer program.

Images