Method and device for acquiring frequent binomial set, equipment and storage medium

Abstract

The application discloses an acquisition method and device for frequent binomial sets, terminal equipment and a storage medium. The number of occurrences of an element of a data set is continuously detected and counted. If the count value of the number of occurrences of the element reaches a preset threshold value, the element is counted into a partial frequent one-item set. The preset threshold value is the minimum support degree set by a user according to actual needs. Item pairs formed by the data set according to the partial frequent one-item set are screened. Hash operation is performed on the screened item pairs. The frequent binomial set is acquired according to the result of the hash operation and the preset threshold value. Thus, the acquisition method for frequent binomial sets provided by the application improves the PCY-Multistage algorithm. Hash operation is only performed on item pairs that meet the partial frequent one-item set, thereby reducing the device memory consumption when calculating the frequent binomial set and improving the user experience.

Description

Technical Field

[0001] This invention relates to the field of big data analytics, and more particularly to methods, apparatus, devices, and computer storage media for acquiring frequent binomial sets. Background Technology

[0002] In the financial lending industry, it's common practice to analyze historical customer borrowing behavior data, such as loan duration, loan product term and interest rate, repayment method, and whether interest-free coupons are used, to identify the most frequently used (most popular) loan product combinations. This allows for the optimization and design of loan product combinations that meet the needs of the general public. This data analysis process typically involves the PCY-Multistage algorithm, used in big data analytics to compute frequent itemsets.

[0003] However, the existing PCY-Multistage algorithm generates a large number of frequent hash buckets during the operation, which consumes a lot of device memory when calculating frequent binary itemsets, thus causing inconvenience to the data analysis process.

[0004] Therefore, how to improve the PCY-Multistage algorithm to reduce device memory consumption when frequently calculating binary itemsets during data analysis is a problem that urgently needs to be solved in the field of big data analysis. Summary of the Invention

[0005] The main objective of this invention is to provide a method, apparatus, device, and computer storage medium for obtaining frequent binomial sets, aiming to improve the PCY-Multistage algorithm and thereby reduce device memory consumption when calculating frequent binomial sets during data analysis.

[0006] To achieve the above objectives, the present invention provides a method for obtaining frequent binary itemsets. This method is applied to the PCY-Multistage data analysis algorithm and includes the following steps:

[0007] The frequency of occurrence of elements in the dataset is continuously detected and counted. If the count of the frequency of occurrence of an element reaches a preset threshold, the element is included in a partial frequent item set. The preset threshold is the minimum support level set by the user according to actual needs.

[0008] Filter the item pairs formed from the data set based on the aforementioned partial frequent item set;

[0009] Perform a hash operation on the filtered item pairs;

[0010] Frequent binomial sets are obtained based on the result of the hash operation and the preset threshold.

[0011] Furthermore, following the step of including a subset of frequent itemsets, the method further includes:

[0012] Stop counting the occurrences of elements in the aforementioned frequent itemset.

[0013] Further, the step of filtering item pairs formed from the data set based on the partially frequent item set may include:

[0014] Obtain the corresponding item pair elements based on the item pairs formed from the data set;

[0015] The item pair elements are compared with the partially frequent item set to obtain the comparison results;

[0016] The item pairs are then filtered based on the comparison results.

[0017] Furthermore, prior to the step of performing a hash operation on the filtered item pairs, the method further includes:

[0018] Determine the hash formula used for the hash operation;

[0019] The step of performing a hash operation on the filtered item pairs may include:

[0020] Based on the hash formula, a hash value is obtained by performing a calculation on the filtered item pairs, and the hash bucket corresponding to the filtered item pairs is determined based on the hash value.

[0021] The set of hash buckets corresponding to the filtered item pairs is determined based on the hash buckets.

[0022] Furthermore, there are multiple hash formulas, and the step of calculating the hash value based on the hash formula for the filtered item pairs, and determining the hash bucket corresponding to the filtered item pairs based on the hash value, may include:

[0023] Multiple hash formulas are used to calculate the selected item pairs to obtain multiple hash values ​​corresponding to the selected item pairs.

[0024] Based on the multiple hash values, determine the multiple hash buckets corresponding to the filtered item pairs;

[0025] The step of determining the hash bucket set corresponding to the filtered item pairs based on the hash buckets may include:

[0026] Based on the multiple hash buckets, determine the multiple hash bucket sets corresponding to the filtered item pairs.

[0027] Furthermore, the step of obtaining frequent binomial sets based on the result of the hash operation and the preset threshold may include:

[0028] The number of times each hash bucket appears in the hash bucket set is counted, and the hash buckets corresponding to the number of times they reach the preset threshold are identified as frequent hash buckets.

[0029] If the hash value corresponding to the filtered item pair belongs to the frequent hash bucket, then the filtered item pair is included in the candidate binary itemset.

[0030] The frequent binomial set is obtained based on the candidate binomial set.

[0031] Furthermore, the step of obtaining the frequent binomial set based on the candidate binomial set may include:

[0032] The occurrence count of each item pair in the candidate binomial set is compared with the preset threshold.

[0033] If the number of occurrences of item pairs in the candidate binomial set reaches the preset threshold, then the item pairs in the candidate binomial set are determined to be frequent item pairs.

[0034] Multiple frequent item pairs are grouped into a set, and the set is labeled as a frequent binomial set.

[0035] Furthermore, to achieve the above objectives, the present invention also provides an apparatus for obtaining frequent binary itemsets, the apparatus comprising:

[0036] The detection and counting module is used to continuously detect and count the number of times an element appears in a dataset. If the count value of the number of times an element appears reaches a preset threshold, the element is included in a partial frequent item set. The preset threshold is the minimum support set by the user according to actual needs.

[0037] The filtering module is used to filter item pairs formed from the data set based on the partial frequent item set;

[0038] The hash operation module is used to perform hash operations on the filtered item pairs;

[0039] The acquisition module is used to acquire frequent binomials based on the result of the hash operation and the preset threshold.

[0040] The present invention implements the steps of the method for obtaining frequent binomial sets as described above in the operation of each functional module of the device for obtaining frequent binomial sets.

[0041] In addition, to achieve the above objectives, the present invention also provides a terminal device, the terminal device comprising: a memory, a processor, and a program for obtaining frequent binary itemsets stored in the memory and executable on the processor, wherein when the program for obtaining frequent binary itemsets is executed by the processor, it implements the steps of the method for obtaining frequent binary itemsets as described above.

[0042] In addition, to achieve the above objectives, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the method for obtaining frequent binomial sets as described above.

[0043] Furthermore, this embodiment of the invention also proposes a computer program product that includes a program for obtaining frequent binary itemsets. When the program for obtaining frequent binary itemsets is executed by a processor, it implements the steps of the method for obtaining frequent binary itemsets as described above.

[0044] The steps implemented when the frequent binomial set acquisition program running on the processor is executed can be referred to in various embodiments of the frequent binomial set acquisition method of the present invention, and will not be repeated here.

[0045] The present invention proposes a method, apparatus, device, and computer storage medium for obtaining frequent binomial sets. This involves continuously detecting and counting the frequency of occurrences of elements in a data set. If the count of an element's occurrences reaches a preset threshold, the element is included in a partial frequent binomial set. The preset threshold is a minimum support level set by the user based on actual needs. The method then filters item pairs formed from the data set based on the partial frequent binomial set. A hash operation is performed on the filtered item pairs. Finally, the frequent binomial set is obtained based on the hash operation result and the preset threshold.

[0046] This invention continuously detects and counts elements in a dataset representing user behavior data. If the frequency of an element's occurrence reaches a preset threshold, the element is included in a partially frequent itemset. The preset threshold is the minimum support set by the user based on actual needs, representing the minimum statistical importance of an element / itemet. Then, based on the partially frequent itemsets, the dataset is pruned and filtered. Only when the elements in an item pair match the partially frequent itemsets is a hash operation performed on the item pair. Based on the hash operation result and the preset threshold, the most frequent binomial itemsets are finally determined.

[0047] Thus, the method for obtaining frequent binary itemsets provided by this invention improves upon the PCY-Multistage algorithm. It changes the calculation process in the existing PCY-Multistage algorithm, which involves traversing the data set to generate a large number of hash buckets and then filtering binary itemsets based on frequent item sets and hash buckets to obtain frequent binary itemsets. Instead, it only performs hash operations when item pairs match some frequent item sets, thereby reducing device memory consumption and improving the user experience when calculating frequent binary itemsets. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the hardware operating environment of the terminal device involved in the embodiments of the present invention;

[0049] Figure 2 This is a flowchart illustrating an embodiment of a method for obtaining frequent binomial sets according to the present invention.

[0050] Figure 3 A schematic diagram of the module structure of a device for acquiring frequent binomial sets according to the present invention.

[0051] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0052] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0053] like Figure 1 As shown, Figure 1 This is a schematic diagram of the hardware operating environment of the terminal device involved in the embodiment of the present invention.

[0054] It should be noted that, Figure 1 This can be a schematic diagram of the hardware operating environment of the terminal device. In this embodiment of the invention, the terminal device can be an electronic terminal, PC, portable computer, or other terminal device used for performing data analysis.

[0055] like Figure 1As shown, the terminal device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0056] Those skilled in the art will understand that Figure 1 The terminal device structure shown does not constitute a limitation on the terminal device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0057] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a distributed task processing program. The operating system is a program that manages and controls the hardware and software resources of the sample terminal device, and supports the distributed task processing program and the operation of other software or programs.

[0058] exist Figure 1 In the terminal device shown, the user interface 1003 is mainly used for data communication with various terminals; the network interface 1004 is mainly used for connecting to the backend server and communicating with the backend server; and the processor 1001 can be used to call the acquisition program for frequent binary itemsets stored in the memory 1005 and perform the following operations:

[0059] The frequency of occurrence of elements in the dataset is continuously detected and counted. If the count of the frequency of occurrence of an element reaches a preset threshold, the element is included in a partial frequent item set. The preset threshold is the minimum support level set by the user according to actual needs.

[0060] Filter the item pairs formed from the data set based on the aforementioned partial frequent item set;

[0061] Perform a hash operation on the filtered item pairs;

[0062] Frequent binomial sets are obtained based on the result of the hash operation and the preset threshold.

[0063] Furthermore, after the step of including some frequent binomial sets, the processor 1001 may call the acquisition program for frequent binomial sets stored in the memory 1005 and perform the following operations:

[0064] Stop counting the occurrences of elements included in the aforementioned frequent itemset.

[0065] Furthermore, the processor 1001 can call the acquisition program for frequent binomial sets stored in the memory 1005, and also perform the following operations:

[0066] Obtain the corresponding item pair elements based on the item pairs formed from the data set;

[0067] The item pair elements are compared with the partially frequent item set to obtain the comparison results;

[0068] The item pairs are then filtered based on the comparison results.

[0069] Furthermore, before the step of performing a hash operation on the filtered item pairs, the processor 1001 may invoke the acquisition program for frequent binary itemsets stored in the memory 1005 and perform the following operations:

[0070] Determine the hash formula used for the hash operation;

[0071] Based on the hash formula, a hash value is obtained by performing a calculation on the filtered item pairs, and the hash bucket corresponding to the filtered item pairs is determined based on the hash value.

[0072] The set of hash buckets corresponding to the filtered item pairs is determined based on the hash buckets.

[0073] Furthermore, there are multiple hash formulas, and the processor 1001 can call the acquisition program for frequent binomial sets stored in the memory 1005, and also perform the following operations:

[0074] Multiple hash formulas are used to calculate the selected item pairs to obtain multiple hash values ​​corresponding to the selected item pairs.

[0075] Based on the multiple hash values, determine the multiple hash buckets corresponding to the filtered item pairs;

[0076] Based on the multiple hash buckets, determine the multiple hash bucket sets corresponding to the filtered item pairs.

[0077] Furthermore, the processor 1001 can call the acquisition program for frequent binomial sets stored in the memory 1005, and also perform the following operations:

[0078] The number of times each hash bucket appears in the hash bucket set is counted, and the hash buckets corresponding to the number of times they reach the preset threshold are identified as frequent hash buckets.

[0079] If the hash value corresponding to the filtered item pair belongs to the frequent hash bucket, then the filtered item pair is included in the candidate binary itemset.

[0080] The frequent binomial set is obtained based on the candidate binomial set.

[0081] Furthermore, the processor 1001 can call the acquisition program for frequent binomial sets stored in the memory 1005, and also perform the following operations:

[0082] The occurrence count of each item pair in the candidate binomial set is compared with the preset threshold.

[0083] If the number of occurrences of item pairs in the candidate binomial set reaches the preset threshold, then the item pairs in the candidate binomial set are determined to be frequent item pairs.

[0084] Multiple frequent item pairs are grouped into a set, and the set is labeled as a frequent binomial set.

[0085] Based on the above structure, various embodiments of the present invention for the method of obtaining frequent binomial sets are proposed.

[0086] It should be noted that the existing PCY-Multistage algorithm generates a large number of frequent hash buckets during the operation, which consumes a lot of device memory when calculating frequent binary itemsets, thus causing inconvenience to the data analysis process.

[0087] Therefore, how to improve the PCY-Multistage algorithm to reduce device memory consumption when frequently calculating binary itemsets during data analysis is a problem that urgently needs to be solved in the field of big data analysis.

[0088] Based on the above phenomena, various embodiments of the present invention for obtaining frequent binary itemsets are proposed. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0089] First embodiment: Please refer to Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the method for obtaining frequent binary itemsets according to the present invention. The method for obtaining frequent binary itemsets provided by the present invention includes the following steps:

[0090] Step S100: Continuously detect and count the number of times an element appears in the data set. If the count of the number of times an element appears reaches a preset threshold, the element is included in a partially frequent item set. The preset threshold is the minimum support level set by the user according to actual needs.

[0091] It should be noted that in this embodiment, the preset threshold is the minimum support set by the user, which represents the lowest statistical importance of an element / item set / item pair.

[0092] The terminal detects and counts the number of times an element appears in the data set representing the product combination. That is, it records the number of times an element appears in the data set. When the number of times an element appears reaches a preset threshold, the element is included in a partial frequent item set.

[0093] Specifically, for example, suppose the data set is T1 = {1,2,3}, T2 = {2,3,4}, T3 = {3,4,5}, T4 = {4,5,6}, T5 = {1,3,5}, T6 = {2,4,6}, T7 = {1,3,4}, T8 = {2,4,5}, T9 = {3,4,6}, T10 = {1,2,4}, T11 = {2,3,5}, T12 = {3,4,6}, and the preset threshold is 4. The terminal detects and counts the elements "1,2,3,4,5,6" in the data set. When the terminal detects the data set up to T8, the number of occurrences of the elements "2,3,4,5" has reached 4, that is, the preset threshold has been reached. Then, the elements "2,3,4,5" are all included in the partially frequent itemset. At this time, the partially frequent itemset is {2,3,4,5}.

[0094] Step S200: Filter the item pairs formed from the data set based on the partial frequent item set.

[0095] The terminal forms multiple item pairs based on the data set representing the product portfolio, and filters the item pairs based on a set of some frequent items.

[0096] Specifically, for example, the corresponding item pairs formed by the data set T1 = {1,2,3} are {1,2}, {1,3}, and {2,3}, and the corresponding item pairs formed by T2 = {2,3,4} are {2,3}, {2,4}, and {3,4}. Then, the terminal further detects that the elements "1 and 2" in the item pair {1,2} do not completely belong to the partially frequent item set {2,3,4,5}, while the elements "2 and 3" in the item pair {2,3} completely belong to the partially frequent item set {2,3,4,5}, and so on.

[0097] Step S300: Perform a hash operation on the filtered item pairs.

[0098] The terminal compares the partial frequent item sets and item pairs. If all elements in the item pair belong to the partial frequent item set, the item pair is determined to have been filtered, and a hash operation is performed on the item pair.

[0099] Specifically, for example, the terminal compares the partially frequent item set {2,3,4,5} with the item pair {1,2}. Since only "2" in the elements "1 and 2" of the item pair {1,2} belongs to the partially frequent item set, the item pair {1,2} fails the screening. Similarly, the terminal compares the partially frequent item set {2,3,4,5} with the item pair {2,3}. Since both "2" and "3" in the item pair {2,3} belong to the partially frequent item set, the item pair {2,3} passes the screening, and the terminal performs a hash operation on the item pair {2,3}.

[0100] Step S400: Obtain frequent binomial sets based on the result of the hash operation and the preset threshold.

[0101] After performing a hash operation on the filtered item pairs, the terminal further filters the item pairs based on the hash operation result and a preset threshold, and obtains frequent binary itemsets based on the final filtering result.

[0102] In this embodiment, the frequency of occurrence of elements in the data set representing user behavior data is continuously detected and counted. If the frequency of occurrence of an element reaches a preset threshold, the element is included in a partially frequent itemset. The preset threshold is the minimum support set by the user according to actual needs, which is used to represent the minimum importance of the element / item set in a statistical sense. Then, the pairs formed by the partially frequent itemsets are pruned and filtered based on the data set. Only when the elements in the item pair match the partially frequent itemset is the item pair hashed. The frequent binomial set is finally determined based on the result of the hash operation and the preset threshold.

[0103] Thus, this embodiment improves the PCY-Multistage algorithm by changing the calculation process in the existing PCY-Multistage algorithm, which involves traversing the data set to generate a large number of hash buckets and then filtering the binary itemsets based on the frequent item set pointers and hash buckets to obtain frequent binary itemsets. Instead, hashing is only performed when the item pairs match some frequent item sets, thereby reducing device memory consumption and improving the user experience when calculating frequent binary itemsets.

[0104] Furthermore, based on the first embodiment of the method for obtaining frequent binary itemsets described above, a second embodiment of the method for obtaining frequent binary itemsets of the present invention is proposed.

[0105] In a second embodiment of the method for obtaining frequent binomial sets according to the present invention, in step S100 above, after the step of including a portion of frequent binomial sets, the method further includes:

[0106] Step S101: Stop counting the occurrences of elements included in the partial frequent itemset.

[0107] After the terminal includes elements in the dataset whose counts of occurrences have reached a preset threshold into a partial frequent itemset, it continues to detect and count the occurrences of elements in the dataset. If an element that has already been included in a partial frequent itemset is detected, the corresponding count will not be increased to reduce computational costs.

[0108] Specifically, for example, suppose the data set is T1 = {1, 2, 3}, T2 = {2, 3, 4}, T3 = {3, 4, 5}, T4 = {4, 5, 6}, T5 = {1, 3, 5}, T6 = {2, 4, 6}, T7 = {1, 3, 4}, T8 = {2, 4, 5}, T9 = {3, 4, 6}, T10 = {1, 2, 4}, T11 = {2, 3, 5}, T12 = {3, 4, 6}, and the preset threshold is 4. The terminal targets the elements in the data set... The elements "1, 2, 3, 4, 5, 6" are detected and counted. When the terminal detects the data set T8, the number of occurrences of elements "2, 3, 4, 5" has reached 4, which means they have all reached the preset threshold. At this time, elements "2, 3, 4, 5" are included in the partial frequent itemset. When the terminal continues to detect the data set T8{2, 4, 5}, since elements "2, 4, 5" have all been included in the partial frequent itemset, the count value corresponding to each element "2, 4, 5" will still be 4, and will not increase accordingly.

[0109] Thus, this embodiment limits the terminal from continuing to count elements of the frequently counted itemsets during the detection and counting process of the data set, thereby reducing computational costs and improving the practicality of the present invention.

[0110] Furthermore, based on the first embodiment of the method for obtaining frequent binary itemsets described above, a third embodiment of the method for obtaining frequent binary itemsets of the present invention is proposed.

[0111] In the third embodiment of the method for obtaining frequent binomial sets of the present invention, step S200 above may include:

[0112] Step S201: Obtain the corresponding item pair elements based on the item pairs formed by the data set.

[0113] After forming multiple item pairs based on the data set, the terminal detects the item pairs to obtain the corresponding item pair elements.

[0114] Specifically, for example, after the terminal forms item pairs {1,2}, {1,3} and {2,3} based on the data set T1{1,2,3}, it then detects item pair {1,2} to obtain the corresponding item pair elements "1 and 2", detects item pair {1,3} to obtain the corresponding relative elements "1 and 3", and detects item pair {2,3} to obtain the corresponding relative elements "2 and 3".

[0115] Step S202: The item pair elements are compared with the partial frequent item set to obtain the comparison result.

[0116] The terminal compares the obtained item pairs with elements from a set of frequent itemsets.

[0117] Specifically, for example, the terminal detects the item pair {1,2} and obtains the corresponding item pair elements "1 and 2", which are then compared with the partially frequent item set {2,3,4,5}.

[0118] Step S203: Filter the item pairs according to the comparison results.

[0119] After comparing the item pair elements with some frequent itemsets, if the item pair elements match the elements of some frequent itemsets, the terminal determines that the item pair corresponding to the item pair elements has passed the filtering.

[0120] Specifically, for example, if the term pair "2 and 3" corresponding to the terminal-determined term pair {2,3} matches the partially frequent term set {2,3,4,5}, then the terminal-determined term pair {2,3} passes the filtering.

[0121] In this embodiment, after the terminal forms multiple item pairs based on the data set, it detects the item pairs to obtain the corresponding item pair elements; the terminal compares the item pair elements obtained by detecting the item pairs with the elements in a partial frequent item set; if the item pair elements match the elements in the partial frequent item set, the terminal determines that the item pair corresponding to the item pair element has passed the filtering.

[0122] Thus, this embodiment defines the specific steps for filtering item pairs based on partially frequent itemsets, thereby improving the practicality of the present invention's method for obtaining frequent binomial sets.

[0123] Furthermore, based on the first embodiment of the method for obtaining frequent binary itemsets described above, a fourth embodiment of the method for obtaining frequent binary itemsets of the present invention is proposed.

[0124] In the fourth embodiment of the method for obtaining frequent binomial sets of the present invention, before step S300 above, the following may be included:

[0125] Step S400: Determine the hash formula used for the hash operation.

[0126] In this embodiment, using a hash formula to perform hash operations is a way to filter item pairs based on minimum support. Different hash formulas will achieve different filtering effects.

[0127] Specifically, for example, the terminal determines the hash formula used for hash operation as i*j%10, that is, when performing operation on the item {2,3} according to the above hash formula, i=2 and j=3 are substituted into the above hash formula for operation.

[0128] In this embodiment, step S300 may include:

[0129] Step S302: Calculate the hash value for the filtered item pairs based on the hash formula, and determine the hash bucket corresponding to the filtered item pairs based on the hash value.

[0130] After determining the hash formula, the terminal performs calculations on the filtered item pairs according to the hash formula to obtain multiple hash values ​​corresponding to the item pairs, and determines multiple hash buckets corresponding to the item pairs based on the hash values.

[0131] Specifically, for example, according to the hash formula i*j%10, the operation is performed on the item {2,3}, that is, 2*3%10=6, and the corresponding hash bucket is 6.

[0132] Step S303: Determine the set of hash buckets corresponding to the filtered item pairs based on the hash buckets.

[0133] After the terminal performs a hash operation on the item pair to obtain the hash bucket, it determines the hash bucket set corresponding to the above item pair based on the number of hash buckets.

[0134] Specifically, for example, after filtering and hashing the pairs of items formed by the data sets T1={1,2,3}, T2={2,3,4}, T3={3,4,5}, T4={4,5,6}, T5={1,3,5}, T6={2,4,6}, T7={1,3,4}, T8={2,4,5}, T9={3,4,6}, T10={1,2,4}, T11={2,3,5}, and T12={3,4,6}, the hash bucket set is determined to be {0:1,2:4,3:1,4:2,5:1,6:1,8:4} based on the formed hash buckets and the number of times the hash buckets appear. That is, "0:1" means that the hash bucket 0 appears 1 time, "2:4" means that the hash bucket 2 appears 4 times, and so on.

[0135] Thus, this embodiment provides specific steps for performing hash operations on filtered item pairs, improving the practicality of the present invention for obtaining frequent binary itemsets.

[0136] Furthermore, based on the fourth embodiment of the method for obtaining frequent binary itemsets described above, where there are multiple hash formulas, a fifth embodiment of the method for obtaining frequent binary itemsets of the present invention is proposed.

[0137] In the fifth embodiment of the method for obtaining frequent binomial sets of the present invention, step S302 may include:

[0138] Step A1: Perform calculations on the filtered item pairs based on multiple hash formulas to obtain multiple hash values ​​corresponding to the filtered item pairs;

[0139] It should be noted that since hashing is a method of filtering item pairs based on minimum support, more than one hash formula is usually used to perform the operation on the item pairs in order to achieve better filtering results.

[0140] The terminal performs hash operations on the item pairs filtered from the partially frequent item set according to multiple hash formulas, and obtains the hash values ​​corresponding to the above hash formulas respectively.

[0141] Specifically, for example, suppose the terminal performs a hash operation on the item pair {2,3} using the first hash formula i*j%10 and the second hash formula i*j*5%10 to obtain the first hash value 6 and the second hash value 0.

[0142] Step A2: Determine the multiple hash buckets corresponding to the filtered item pairs based on the multiple hash values;

[0143] After the terminal performs a hash operation on the item pair to obtain multiple hash values, it determines the hash bucket corresponding to the item pair based on the hash values.

[0144] Specifically, for example, assuming the terminal performs a hash operation on the item pair {2,3} and obtains a first hash value of 6 and a second hash value of 0, then the first hash bucket is determined to be 6 and the second hash bucket is determined to be 0; the hash operation for other item pairs is deduced in the same way.

[0145] In this embodiment, step S303 may include:

[0146] Step A3: Determine the set of multiple hash buckets corresponding to the filtered item pairs based on the multiple hash buckets.

[0147] After determining multiple hash buckets, the terminal determines the set of multiple hash buckets corresponding to the filtered item pairs based on the frequency of occurrence of each of the multiple hash buckets.

[0148] Specifically, for example, after filtering and hashing the pairs of items formed by the data sets T1={1,2,3}, T2={2,3,4}, T3={3,4,5}, T4={4,5,6}, T5={1,3,5}, T6={2,4,6}, T7={1,3,4}, T8={2,4,5}, T9={3,4,6}, T10={1,2,4}, T11={2,3,5}, and T12={3,4,6}, the corresponding first hash bucket set is determined as {0:1,2:4,3:1,4:2,5:1,6:1,8:4} and the second hash bucket set is {0:8}, based on the formed hash buckets and the number of hash buckets.

[0149] Thus, this embodiment provides specific steps for filtering item pairs using multiple hash formulas, thereby further improving the filtering effect and enhancing the practicality of the present invention.

[0150] Furthermore, based on the fourth embodiment of the method for obtaining frequent binary itemsets described above, a sixth embodiment of the method for obtaining frequent binary itemsets of the present invention is proposed.

[0151] In the sixth embodiment of the method for obtaining frequent binomial sets of the present invention, step S302 above may include:

[0152] Step B1: Count the number of times each hash bucket appears in the hash bucket set, and determine the hash buckets corresponding to the number of times they reach the preset threshold as frequent hash buckets.

[0153] The terminal compares the number of times a hash bucket appears in the hash bucket set with a preset threshold representing the minimum support. If the number of times a hash bucket appears reaches the preset threshold, then the hash bucket is determined to be a frequent hash bucket.

[0154] Specifically, for example, if the terminal determines the hash bucket set as {0:1,2:4,3:1,4:2,5:1,6:1,8:4} after performing a hash operation on the item pair according to the hash formula i*j%10, then hash bucket 2 and hash bucket 8 are determined as frequent hash buckets according to the preset threshold of 4.

[0155] Step B2: If the hash value corresponding to the filtered item pair belongs to the frequent hash bucket, then the filtered item pair is included in the candidate binary itemset.

[0156] After determining the frequent hash buckets, the terminal identifies the corresponding item pairs as candidate item pairs based on the hash values ​​in the frequent hash buckets and adds them to the candidate binary itemsets.

[0157] Specifically, for example, if the hash value of the item pair {2,4} is 8 according to the hash formula i*j%10, and it belongs to the frequent hash bucket 8, then the terminal determines that the item pair {2,4} is a candidate item pair and includes it in the candidate binary itemset.

[0158] Step B3: Obtain the frequent binomial set based on the candidate binomial set.

[0159] After determining the candidate two-items set, the terminal further obtains frequent two-items sets for the candidate two-items set according to a preset threshold representing the minimum support.

[0160] Furthermore, in one feasible embodiment, step B3 above may include:

[0161] Step B31: Compare the occurrence count of each item pair in the candidate binomial set with the preset threshold.

[0162] Step B32: If the number of occurrences of item pairs in the candidate binomial set reaches the preset threshold, then the item pairs in the candidate binomial set are determined to be frequent item pairs.

[0163] Step B33: Combine the multiple frequent item pairs into a set, and label the set as a frequent binomial set.

[0164] After determining the candidate binomial sets, the terminal compares the frequency of each item pair in the candidate binomial sets with a preset threshold representing the minimum support. If the frequency of each item pair reaches the preset threshold, the item pair is determined to be a frequent item pair. After determining all frequent item pairs, the terminal assembles the frequent item pairs into a set and marks the set as a frequent binomial set.

[0165] Specifically, for example, the terminal determines the candidate two-item sets as [{1,2},{2,4},{3,4},{3,4},{2,4},{2,6},{3,4},{2,4},{3,4},{3,6},{3,4},{1,2},{2,4}], where the candidate pair {2,4} appears 4 times and the candidate pair {3,4} appears 5 times. The preset threshold representing the minimum support is 4. Therefore, the terminal determines the candidate pairs {2,4} and {3,4} as frequent item pairs, and then combines the frequent item pairs {2,4} and {3,4} to obtain the frequent two-item sets [{2,4}, {3,4}].

[0166] Thus, this embodiment provides specific implementation steps for obtaining frequent binomial sets based on the result of hash operation and a preset threshold representing minimum support, thereby improving the practicality of the present invention.

[0167] In addition, please refer to Figure 3 The present invention also proposes a device for obtaining frequent binary itemsets. The device for obtaining frequent binary itemsets includes:

[0168] The detection and counting module is used to continuously detect and count the number of times an element appears in a dataset. If the count value of the number of times an element appears reaches a preset threshold, the element is included in a partial frequent item set. The preset threshold is the minimum support set by the user according to actual needs.

[0169] The filtering module is used to filter item pairs formed from the data set based on the partial frequent item set;

[0170] The hash operation module is used to perform hash operations on the filtered item pairs;

[0171] The acquisition module is used to acquire frequent binomials based on the result of the hash operation and the preset threshold.

[0172] Preferably, the detection counting module includes:

[0173] A stopping unit is used to stop counting elements included in the partial frequent itemset.

[0174] Preferably, the filtering module includes:

[0175] The first acquisition unit is used to acquire the corresponding item pair element based on the item pairs formed by the data set; and to compare the item pair element with the partial frequent item set.

[0176] Preferably, the apparatus for acquiring frequent binomial sets further includes:

[0177] A determining module is used to determine the hash formula used for the hash operation.

[0178] Preferably, the hash operation module includes:

[0179] The determining unit is configured to perform a hash operation on the filtered item pairs based on the hash formula to obtain a hash value, determine the hash bucket corresponding to the filtered item pairs based on the hash value, and determine the hash bucket set corresponding to the filtered item pairs based on the hash bucket.

[0180] Preferably, the determining unit includes:

[0181] A subunit is defined for performing calculations on the filtered item pairs based on multiple hash formulas to obtain multiple hash values ​​corresponding to the filtered item pairs; determining multiple hash buckets corresponding to the filtered item pairs based on the multiple hash values; and determining multiple hash bucket sets corresponding to the filtered item pairs based on the multiple hash buckets.

[0182] Preferably, the acquisition module includes:

[0183] The second acquisition unit is used to count the number of times each hash bucket appears in the hash bucket set, and determine the hash bucket corresponding to the number of times it reaches the preset threshold as a frequent hash bucket; if the hash value corresponding to the item pair obtained after filtering belongs to the frequent hash bucket, then the item pair obtained after filtering is included in the candidate two-item set; and the frequent two-item set is obtained based on the candidate two-item set.

[0184] Preferably, the second acquisition unit includes:

[0185] The generation unit is used to compare the occurrence count of each item pair in the candidate binomial set with the preset threshold; if the occurrence count of the item pair in the candidate binomial set reaches the preset threshold, the item pair in the candidate binomial set is determined to be a frequent item pair; multiple frequent item pairs are combined to form a set, and the set is marked as a frequent binomial set.

[0186] Furthermore, this embodiment of the invention also proposes a terminal device, which includes: a memory, a processor, and a program for obtaining frequent binary itemsets stored in the memory and executable on the processor. When the program for obtaining frequent binary itemsets is executed by the processor, it implements the steps of the method for obtaining frequent binary itemsets as described above.

[0187] The steps implemented when the frequent binomial set acquisition program running on the processor is executed can be referred to in various embodiments of the frequent binomial set acquisition method of the present invention, and will not be repeated here.

[0188] Furthermore, this embodiment of the invention also proposes a storage medium for use in a computer. The storage medium can be a non-volatile computer-readable storage medium, on which a program for obtaining frequent binary itemsets is stored. When the program for obtaining frequent binary itemsets is executed by a processor, it implements the steps of the method for obtaining frequent binary itemsets as described above.

[0189] The steps implemented when the frequent binomial set acquisition program running on the processor is executed can be referred to in various embodiments of the frequent binomial set acquisition method of the present invention, and will not be repeated here.

[0190] Furthermore, this embodiment of the invention also proposes a computer program product that includes a program for obtaining frequent binary itemsets. When the program for obtaining frequent binary itemsets is executed by a processor, it implements the steps of the method for obtaining frequent binary itemsets as described above.

[0191] The steps implemented when the frequent binomial set acquisition program running on the processor is executed can be referred to in various embodiments of the frequent binomial set acquisition method of the present invention, and will not be repeated here.

[0192] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0193] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0194] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause an electronic terminal for performing data analysis to execute the methods described in the various embodiments of the present invention.

[0195] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for obtaining frequent binomial sets, characterized in that, The method for obtaining frequent binary itemsets is applied to the PCY-Multistage data analysis algorithm, and the method for obtaining frequent binary itemsets includes the following steps: The frequency of occurrence of elements in the dataset is continuously detected and counted. If the count of the frequency of occurrence of an element reaches a preset threshold, the element is included in a partial frequent item set. The preset threshold is the minimum support level set by the user according to actual needs. Filter the item pairs formed from the data set based on the aforementioned partial frequent item set; Determine the hash formula to be used for hash operations; Perform a hash operation on the filtered item pairs; Frequent binomials are obtained based on the result of the hash operation and the preset threshold. Wherein, there is one hash formula, and the hash operation on the filtered item pairs specifically includes: operating on the filtered item pairs based on the hash formula to obtain a hash value, determining the hash bucket corresponding to the filtered item pairs based on the hash value, and determining the hash bucket set corresponding to the filtered item pairs based on the hash bucket; or, there are multiple hash formulas, and the hash operation on the filtered item pairs specifically includes: operating on the filtered item pairs based on multiple hash formulas respectively to obtain multiple hash values ​​corresponding to the filtered item pairs, determining multiple hash buckets corresponding to the filtered item pairs based on the multiple hash values, and determining the multiple hash bucket set corresponding to the filtered item pairs based on the multiple hash buckets.

2. The method for obtaining frequent binomial sets as described in claim 1, characterized in that, Following the step of including a subset of frequent itemsets, the method further includes: Stop counting the occurrences of elements included in the aforementioned frequent itemset.

3. The method for obtaining frequent binomial sets as described in claim 1, characterized in that, The step of filtering item pairs formed from the data set based on the partial frequent item set includes: Obtain the corresponding item pair elements based on the item pairs formed from the data set; The item pair elements are compared with the partially frequent item set to obtain the comparison results; The item pairs are then filtered based on the comparison results.

4. The method for obtaining frequent binomial sets as described in claim 1, characterized in that, The step of obtaining frequent binary itemsets based on the result of the hash operation and the preset threshold includes: The number of times each hash bucket appears in the hash bucket set is counted, and the hash buckets corresponding to the number of times they reach the preset threshold are identified as frequent hash buckets. If the hash value corresponding to the filtered item pair belongs to the frequent hash bucket, then the filtered item pair is included in the candidate binary itemset. The frequent binomial set is obtained based on the candidate binomial set.

5. The method for obtaining frequent binomial sets as described in claim 4, characterized in that, The step of obtaining the frequent binary itemsets based on the candidate binary itemsets includes: The occurrence count of each item pair in the candidate binomial set is compared with the preset threshold. If the number of occurrences of item pairs in the candidate binomial set reaches the preset threshold, then the item pairs in the candidate binomial set are determined to be frequent item pairs. Multiple frequent item pairs are grouped into a set, and the set is labeled as a frequent binomial set.

6. A device for acquiring frequent binomial sets, characterized in that, The device for acquiring frequent binary itemsets includes: The detection and counting module is used to continuously detect and count the number of times an element appears in a dataset. If the count value of the number of times an element appears reaches a preset threshold, the element is included in a partial frequent item set. The preset threshold is the minimum support set by the user according to actual needs. The filtering module is used to filter item pairs formed from the data set based on the partial frequent item set; The determination module is used to determine the hash formula used for hash operations; A hash operation module is used to perform hash operations on filtered item pairs. The hash formula is singular, and the hash operation on the filtered item pairs specifically includes: performing an operation on the filtered item pairs based on the hash formula to obtain a hash value; determining the hash bucket corresponding to the filtered item pairs based on the hash value; and determining the hash bucket set corresponding to the filtered item pairs based on the hash buckets. Alternatively, the hash formula is multiplied, and the hash operation on the filtered item pairs specifically includes: performing operations on the filtered item pairs based on multiple hash formulas respectively to obtain multiple hash values ​​corresponding to the filtered item pairs; determining multiple hash buckets corresponding to the filtered item pairs based on the multiple hash values; and determining the multiple hash bucket set corresponding to the filtered item pairs based on the multiple hash buckets. The acquisition module is used to acquire frequent binomials based on the result of the hash operation and the preset threshold.

7. A terminal device, characterized in that, The terminal device includes: a memory, a processor, and a program for obtaining frequent binary itemsets stored in the memory and executable on the processor, wherein when the program for obtaining frequent binary itemsets is executed by the processor, it implements the steps of the method for obtaining frequent binary itemsets as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for obtaining frequent binomial sets as described in any one of claims 1 to 5.

Images