Efficient item set mining method and device

Abstract

The invention provides a high-utility item set mining method and equipment, and the method comprises the steps: determining an initial zero-placeholder utility matrix, a first transaction utility vector and an initial transaction weighted utility hash table of a 1-item set according to an obtained transaction database; determining an item set position hash table according to the initial utility matrix; updating the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold to obtain a second transaction utility vector and a target transaction weighted utility hash table; according to the second transaction utility vector, an item set position hash table, a target transaction weighted utility hash table and an initial utility matrix of the 1-item set, determining a zero-placeholder utility matrix containing residual utility of the 1-item set; and based on a depth-first strategy, according to a target transaction weighted utility hash table and the zero-occupancy utility matrix containing the residual utility, performing item set expansion to obtain a to-be-judged item set and a corresponding zero-occupancy utility matrix, and determining a high-utility item set from the to-be-judged item set and the corresponding zero-occupancy utility matrix, thereby realizing high-efficiency mining.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a high-utility item set mining method and device. Background Art

[0002] With the advent of the era of information explosion, how to mine valuable information data from massive and unstructured data has become increasingly important in Internet technology. Data mining is a computational process that extracts data that users are interested in and can understand and use from real, massive, noisy data through specific algorithms. Among them, the high-utility item set mining algorithm stands out with its unique advantages. It not only inherits the basic concept of data mining, but also optimizes it according to the characteristics of the data, while paying attention to the frequency and utility calculation of the data. The mined utility combination not only represents the high-frequency mode of user behavior, but also reflects its high-profit contribution to merchants or platforms in actual situations, providing strong data support for precision marketing, inventory management, and user experience optimization. In summary, in the context of the era of information explosion, the high-utility item set mining algorithm has become an indispensable data analysis tool in Internet technology and various industries with its efficient processing ability for massive data, opening up a new path for deep data mining and value discovery.

[0003] At present, the research on high-utility item set mining algorithms mainly includes two methods: utility list based and database mapping based. Based on the vertical structure of the utility list, the utility list is merged to complete the item set evaluation process. Based on database mapping, the horizontal data structure is adopted to continuously compress the original database size through projection.

[0004] However, with the continuous expansion of data scale, the problems of computational complexity and memory consumption of high-utility item set mining algorithms have become increasingly prominent, becoming a bottleneck restricting their widespread application and in-depth development. First, the frequent scanning of the original database by the high-utility item set mining algorithm is the main reason for its low computational efficiency. In the face of massive data, this traditional scanning mode is particularly cumbersome, not only time-consuming and labor-intensive, but also easily leads to delays and blockages in data access, which in turn affects the real-time and accuracy of data mining. Secondly, the number of candidate item sets generated by the algorithm during the mining process is astonishing, and each candidate item set needs to go through item set expansion and utility list construction to complete a complex utility evaluation process. This process is not only computationally intensive, but also involves comprehensive consideration of multiple parameters and variables, which increases the complexity and uncertainty of the algorithm. This complexity not only prolongs the running time of the algorithm, but may also affect the accuracy and comprehensiveness of the mining results. The high-utility item set mining algorithm also has disadvantages in terms of memory consumption. During the mining process, the algorithm will generate a large number of intermediate results, including candidate item sets, utility lists, residual utility lists, etc. These intermediate results need to be stored in memory for subsequent processing. However, as the size of the data set increases, the number of intermediate results increases dramatically, which puts tremendous pressure on memory resources. When memory resources are insufficient, the algorithm's running speed will drop significantly, and it may even crash due to memory overflow. This not only wastes computing resources, but may also lead to the failure of data mining tasks and the loss of data. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a high-utility item set mining method and device, which specifically include:

[0006] In a first aspect, the present invention provides a high-utility item set mining method, comprising:

[0007] According to the acquired transaction database, the initial utility matrix, the first transaction utility vector and the initial transaction weighted utility hash table corresponding to the 1-item set are determined. The utility matrix is ​​a zero-occupied matrix. The first transaction utility vector is used to record the utility of each transaction. The initial transaction weighted utility hash table is used to record the transaction weighted utility of each 1-item set, and the transaction weighted utility of each 1-item set is stored in ascending order.

[0008] According to the initial utility matrix corresponding to the 1-item set, determine the item set position hash table, which records the position of each item set in the utility matrix;

[0009] Update the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold to obtain a second transaction utility vector and a target transaction weighted utility hash table;

[0010] Determine a utility matrix including residual utility corresponding to the 1-itemset according to the second transaction utility vector, the item set position hash table, the target transaction weighted utility hash table and the initial utility matrix corresponding to the 1-itemset;

[0011] Based on the depth-first strategy, the item set is expanded according to the weighted utility hash table of the target transaction and the utility matrix containing the residual utility corresponding to the 1-item set to obtain the item set to be judged, and determine whether the item set to be judged is a high-utility item set.

[0012] In a second aspect, the present invention further provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0013] Memory, used to store computer programs;

[0014] The processor is used to implement any method provided in the first aspect when executing a program stored in the memory.

[0015] In a third aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any one of the methods provided in the first aspect is implemented.

[0016] Beneficial effects of the present invention:

[0017] The high-utility item set mining method and device provided by the present invention determine the initial utility matrix, the first transaction utility vector and the initial transaction weighted utility hash table corresponding to the 1-item set according to the acquired transaction database, the utility matrix is ​​a zero-occupied matrix, the first transaction utility vector is used to record the utility of each transaction, the initial transaction weighted utility hash table is used to record the transaction weighted utility of each 1-item set, and the transaction weighted utility of each 1-item set is stored in ascending order; according to the initial utility matrix corresponding to the 1-item set, determine the item set position hash table, the item set position hash table records the position of each item set in the utility matrix; update the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold, and obtain to the second transaction utility vector and the target transaction weighted utility hash table; according to the second transaction utility vector, the item set position hash table, the target transaction weighted utility hash table and the initial utility matrix corresponding to the 1-item set, determine the utility matrix containing the residual utility corresponding to the 1-item set; based on the depth-first strategy, according to the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-item set, expand the item set to be judged, and determine whether the item set to be judged is a high-utility item set. Its two-dimensional utility matrix structure based on the zero-occupancy matrix can simplify the construction and merging process of the utility list in the high-utility item set mining algorithm, reduce the computational complexity, improve the mining efficiency, and reduce resource occupation.

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a flow chart of a high-utility itemset mining method provided by the present invention;

[0020] Figure 2 A schematic diagram of a process for determining a set of items to be judged provided by the present invention;

[0021] Figure 3 A schematic diagram of a process for determining a high-utility item set from an item set to be determined provided by the present invention;

[0022] Figure 4 A schematic diagram of an experimental process provided by the present invention;

[0023] Figure 5 Another schematic diagram of an experimental process provided by the present invention;

[0024] Figure 6 A schematic diagram of experimental data provided by the present invention;

[0025] Figure 7 The present invention provides another schematic diagram of an experimental process. DETAILED DESCRIPTION

[0026] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0027] The problems in the prior art are not solved. The present invention proposes a high-utility item set mining method. The method adopts a two-dimensional utility matrix structure based on a zero-occupancy matrix, which can simplify the construction and merging process of the utility list in the high-utility item set mining algorithm. In addition, the method uses a pointer-maintained prefix-free item set expansion method to reduce the cost of executing each connection. At the same time, the method proposes a novel two-stage local pruning strategy, which can compress the search space.

[0028] Figure 1 A flowchart of a high-utility item set mining method provided by the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0029] S101. Determine an initial utility matrix, a first transaction utility vector, and an initial transaction weighted utility hash table corresponding to a 1-item set according to an acquired transaction database.

[0030] Among them, the utility matrix is ​​a zero-occupied matrix.

[0031] The first transaction utility vector is used to record the utility of each transaction.

[0032] The initial transaction weighted utility hash table is used to record the transaction weighted utility of each 1-item set, and the transaction weighted utility of each 1-item set is stored in ascending order.

[0033] In one possible implementation, the elements in the first column of the utility matrix are expressed as:

[0034]

[0035] Among them, m(k,0) represents the element in the zero-placeholder utility matrix, k represents the matrix row, j represents the matrix column, sumu(i k ) represents element i k The sum of the utility values, sumru(i k ) represents element i k The sum of the residual utility values, u(i k ).row represents element i k The matrix rows storing utility values, ru(i k ).row represents element i k The matrix rows storing the residual utility values, T j Represents transaction j, i k represents the item set k.

[0036] The elements in the even-numbered rows of the utility matrix except the first column are expressed as:

[0037]

[0038] Among them, m(2(k-1),j) represents the elements of the even rows in the zero-occupied matrix, and u(i k ,T j ) represents the utility value of item set k in transaction j;

[0039] The elements in the odd rows of the utility matrix except the first column are expressed as:

[0040]

[0041] Among them, ru(i k ,T j ) represents the residual utility value of item set k in transaction j. Specifically, the original database is scanned by transaction, and the transaction utility of each transaction is recorded in the first transaction utility vector (tV vector).

[0042] The tV vector is defined as:

[0043]

[0044] Furthermore, the utility value of each 1-item contained in each transaction is stored in the utility matrix. The utility matrix is ​​For a database with j transactions and k items, the utility matrix containing 1-items consists of 2k rows and j+1 columns. The rows and columns of the matrix represent utility, remaining utility, and Tid (Transaction ID), respectively. Define u(i k ).row is where u(i k ) of the line, ru(i k ).row is where ru(i k ). Therefore, the 1-term utility matrix is ​​defined as:

[0045] Even Behavior:

[0046]

[0047] Odd Behavior:

[0048]

[0049] The first column of the utility matrix stores the sum of the utilities following the corresponding row, namely:

[0050]

[0051] When scanning the original database, only the even-numbered rows in the matrix are stored, that is, only the 1-item utility is recorded, and the utility value is accumulated to (u(i k ).row, 0), is used to record the sum of the utility values ​​of the 1-item.

[0052] S102 . Determine an item set position hash table according to the initial utility matrix corresponding to the 1-item set.

[0053] Among them, the item set position hash table is used to record the position of each item set in the utility matrix.

[0054] Specifically, two computationally assisted hash tables are constructed: the initial transaction weighted utility hash table (twuM) and the itemset location hash table (lM).

[0055] twuM is used to record 1-item transaction weighted utility (TWU) and is stored in ascending order of TWU. It is defined as:

[0056]

[0057] Define ru(i k ).row=(u(i k ).row+1); therefore lM only needs to store the u(i k).row facilitates the location of items when searching and is defined as:

[0058] lM={{i1:u(i1).row}}...{i k :u(i k ).row}}.

[0059] S103: Update the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold to obtain a second transaction utility vector and a target transaction weighted utility hash table.

[0060] In a possible implementation, updating the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold to obtain the second transaction utility vector and the target transaction weighted utility hash table includes:

[0061] A1. Determine whether there is a target element in the initial transaction weighted utility hash table whose transaction weighted utility of the 1-item set is less than a preset threshold. If so, delete the target element from the initial transaction weighted utility hash table to obtain a target transaction weighted utility hash table.

[0062] A2. Get the 1-item set corresponding to the target element.

[0063] A3. Obtain the elements in the first transaction utility vector that contain the 1-item set corresponding to the target element, and delete the utility value of the 1-item set corresponding to the target element from the obtained elements to obtain a second transaction utility vector.

[0064] Specifically, since twuM has been initialized when scanning the database, the algorithm directly traverses the hash table to implement TWU pruning, that is, all 1-items with TWU values ​​less than the threshold have no mining value and need to be discarded. Since the TWU value and the corresponding 1-item in twuM are stored in pairs, if the TWU value of the item is less than the threshold during the traversal process, the two will be discarded in pairs. All transactions containing the item will also subtract the utility value of the item in this transaction from their own transaction utility.

[0065] S104: Determine a utility matrix including residual utility corresponding to the 1-itemset according to the second transaction utility vector, the item set position hash table, the target transaction weighted utility hash table and the initial utility matrix corresponding to the 1-itemset.

[0066] In a possible implementation, determining a utility matrix including residual utility corresponding to the 1-itemset according to the second transaction utility vector, the item set position hash table, the target transaction weighted utility hash table, and the initial utility matrix corresponding to the 1-itemset includes:

[0067]

[0068] Among them, ru(ik ,T j ) represents the residual utility of item set k in transaction j, i k Represents item set k, i k ' represents the itemsets after itemset k in transaction j in ascending order of TWU, T j Represents transaction j, u(i k ',T j ) represents the utility value of the item set in transaction j that is located after item set k in ascending order of TWU, u(i k ,T j ) represents the utility value of item set k in transaction j, tu(T j ) represents the transaction utility of transaction j, TWU(i k ') represents the TWU value of the item set after item set k in transaction j, which is arranged in ascending order of TWU. TWU(i k ) represents the TWU value of item set k.

[0069] Specifically, the residual utility of an item in each transaction is defined as the sum of the utilities of all items following the item after being sorted in ascending order of TWU.

[0070] The definition and actual calculation method are:

[0071]

[0072] Therefore, the calculation of the residual utility requires calling the utility of each transaction in ascending order of TWU. That is, when traversing twuM, call the transaction containing the 1-item, subtract the utility of the 1-item from the transaction utility, update this value to the tV vector, and store it in the corresponding position in the utility matrix (ru(i k ).row,T j ). During the calculation process, if the 1-item does not exist in the transaction, then its corresponding residual utility does not exist either, and there is no need to perform redundant calculations to waste running time and memory. The judgment basis for this condition is that if the item corresponds to At the same time, each residual utility in the calculation process will be accumulated to (ru(i k ).row, 0), is used to record the sum of the residual utility values ​​of the 1-item.

[0073] S105 , based on a depth-first strategy, expand the itemset according to the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-itemset to obtain the itemset to be judged, and determine whether the itemset to be judged is a high-utility itemset.

[0074] In a possible implementation, based on a depth-first strategy, the item set is expanded according to the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-item set to obtain the item set to be judged, including:

[0075] Decision 1: In the i-th round of expansion, determine whether the pointer p points to the end of the target transaction weighted utility hash table. If so, execute the steps in operation a. If not, execute the steps in decision 2.

[0076] Decision 2: Determine whether the first return value corresponding to the i-1th round of expansion process is true. If so, execute the steps in operation b. If not, execute the steps in operation c.

[0077] Operation a: Delete the last item of the i-1th item set to be judged stored in the item set vector ciV to obtain the first item set vector. The control pointer p searches for the last item of the first item set vector in the target transaction weighted utility hash table and points to the search result. Delete the last item of the first item set vector. The control pointer p moves one bit backward. Add the item currently pointed to by the pointer p to the first item set vector to obtain the i-th item set to be judged.

[0078] Operation b: Delete the last item of the i-1th item set to be judged stored in the item set vector ciV to obtain the third item set vector. The control pointer p searches for the last item of the third item set vector in the target transaction weighted utility hash table and points to the search result. The control pointer p moves one bit backward and adds the item currently pointed to by the pointer p to the third item set vector to obtain the i-th item set to be judged.

[0079] Operation c: Add the item currently pointed to by pointer p to the end of the i-1th item set to be judged stored in the item set vector ciV to obtain the i-th item set to be judged.

[0080] In the i-th round of expansion, in a possible implementation, determining whether the item set to be judged is a high-utility item set includes:

[0081] S61, determine whether the i-th item set to be determined is a 1-item set, if so, set the first return value to true, if not, execute S62.

[0082] S62, judging whether the i-th item set to be judged satisfies the preset first-stage pruning strategy, if not, setting the first return value to false, if satisfied, executing S63.

[0083] S63, constructing a utility matrix of the i-th set of items to be judged according to the transaction.

[0084] S64, based on the logical AND operation, determine whether the item set consisting of items other than the last item in the i-th item set to be determined and the last item of the i-th item set to be determined exist in the same transaction at the same time; if not, execute S65; if yes, execute S66.

[0085] S65, judging whether the i-th item set to be judged satisfies the preset second-stage pruning strategy, if so, returning to execute S63, if not, setting the first return value to false.

[0086] S66, recording the utility value and residual utility value of the i-th item set to be judged in the target transaction;

[0087] S67, determine whether to traverse all transactions, if not, return to execute S63, if yes, determine whether the i-th item set to be determined is a high-utility item set according to preset rules, and whether the i-th item set to be determined needs to be expanded.

[0088] The preset rules include:

[0089] If sumu(Pxy)>minutil, then the item set to be judged {Pxy} is a high-utility item set; if sumu(Pxy)+sumru(Pxy)>minutil, then the item set to be judged needs to be expanded.

[0090] sumu(Pxy) represents the sum of the utility values ​​of the i-th item set to be judged, minutil represents the preset threshold, and sumru(Pxy) represents the sum of the residual utility values ​​of the i-th item set to be judged.

[0091] Further, optionally, before executing judgment 1, the method further includes:

[0092] Initialization: Set the item set vector ciV to an empty set, control the pointer p to point to the start bit of the target transaction weighted utility hash table, set the first return value to true, and the item set vector ciV is used to store the item set to be judged.

[0093] Further, optionally, the method further includes:

[0094] Decision 3: In the i-th round of expansion, determine whether the first return value obtained in the i-th round is true, or whether the pointer p points to the end of the target transaction weighted utility hash table. If so, execute the step of operation d; if not, execute the step of decision 4;

[0095] Decision 4: Determine whether the pointer p points to the end of the target transaction weighted utility hash table, or whether the size of the item set vector ciV is 1. If so, execute the step of decision 1. If not, end.

[0096] Operation d: Control pointer p to move one position backward and execute step 4.

[0097] The present invention can reduce the cost of executing each connection and improve the operation efficiency by using the prefix-free item set expansion method maintained by pointers; and can compress the search space, reduce the data volume and reduce the waste of resources by using a two-stage local pruning strategy.

[0098] Exemplarily, step S105 can be divided into two major steps, where step 1 and step 2 are nested and executed, as follows:

[0099] like Figure 2 As shown, step one includes:

[0100] In the subsequent mining process of high-utility item sets, the algorithm needs to expand the item set according to the "depth-first" strategy to obtain the item set to be judged. The present invention proposes a prefix-free item set expansion method maintained by pointers, vectors and hash tables. The specific implementation method is as follows:

[0101] Define the item set vector as ciV = {I}, which stores the item set to be judged. When step 2 is executed for the first time, define Define pointer p to point to twuM, and obtain the 1-item of the pointed position. When step 2 is executed for the first time, define p to point to the start position of twuM. And when step 2 is executed for the first time, define the return value to be true.

[0102] The relevant judgment operations in the method are:

[0103] Judgment 1: First determine whether pointer p points to the end of twuM. If true, perform operation a; if false, perform judgment 2.

[0104] Judgment 2: Determine whether the return value of step 3 executed last time is true. If it is true, perform operation b; if it is false, perform operation c.

[0105] Judgment 3: Determine whether the return value of step 3 is true or whether the pointer p points to the end of twuM. If true, perform operation d; if false, perform judgment 4.

[0106] Judgment 4: Determine whether the pointer p points to the end of twuM or the size of the ciV vector is 1. If it is true, judge 1, otherwise jump out of the loop.

[0107] The operations after the relevant judgment operations in the method are:

[0108] Operation a: ciV deletes the last item, pointer p searches for the last item of ciV vector in twuM and points to it, ciV deletes the last item again, pointer p moves one position backward, and the item currently pointed to by pointer p is added to ciV vector, and then proceed to step 2.

[0109] Operation b: ciV deletes the last item, pointer p searches for the last item of ciV vector in twuM and points to it, pointer p moves one position backward, the item currently pointed to by pointer p is added to ciV vector, and step 2 is performed.

[0110] Operation c: Add the item currently pointed to by pointer p to the ciV vector and proceed to step 2.

[0111] Operation d: Move pointer p one position backward and perform judgment 4.

[0112] like Figure 3 As shown, step two includes:

[0113] After obtaining the set of items to be judged, the algorithm needs to construct the corresponding two-row utility matrix for judgment. Let the set of items to be judged be {Pxy}.

[0114] If the item set to be judged is a 1-item set at this time, the true value is directly returned to judgment 3 in step 1, and the subsequent operations of step 2 are not performed.

[0115] Since the construction process is complicated, in order to save resources, the first stage of pruning is performed to determine whether {Pxy} is worthy of constructing the utility matrix.

[0116] The pruning strategy for the first phase is:

[0117]

[0118] If not satisfied, return false and return to judgment 3 in step 1, and do not perform subsequent operations in step 2. If satisfied, start building the utility matrix of {Pxy}.

[0119] According to the prefix-free itemset expansion, the construction of the utility matrix of {Pxy} requires the convenience of the utility values ​​and residual utility values ​​of {Px} and {y}.

[0120] Due to the characteristics of the zero-placeholder utility matrix, the present invention proposes a merging method based on a "logical AND" operation. j When u(Px,T j )&&u(y,T j )==1 means {Px} and {y} exist in transaction T at the same time j , at this time u(Pxy,T j )=u(Px,T j )+u(y,T j ),ru(Pxy,T j )=ru(y,T j ). And store the calculated utility values ​​in (u(Pxy).row, T j ), (ru(Pxy).row, T j ), then accumulate to (u(Pxy).row, 00, (ru(Pxy).row, 0). If u(Px,T t )&&u(y,T t )==0 means {Px} and {y} do not exist in transaction T at the same time t , at this time, a two-stage pruning strategy is required, namely

[0121] If the condition is satisfied, the utility list merging operation is continued; if not satisfied, a false return is made to judgment 3 in step 1, and the subsequent operation of step 2 is not performed to start expanding the next set of items to be judged.

[0122] After traversing all transactions, if sumu(Pxy)>minutil, then {Pxy} is a high-utility item set; if sumu(Pxy)+sumru(Pxy)>minutil, then {Pxy}' may become a high-utility item set, return a true value to judgment 3 in step 1, and start the expansion of the next item set; if {Pxy}' cannot become a high-utility item set, return a false value to judgment 3 in step 1.

[0123] In order to facilitate understanding of the present invention, a set of examples are also provided, which are as follows:

[0124] The existing original database is shown in the following table. Assume that the user initially sets the threshold to 20.

[0125] Table 1 Original database

[0126] Transaction Item Transaction Utility (Tu) Utility <![CDATA[T1]]> {a,b,c,d,e,f} 28 {5,8,1,6,3,5} <![CDATA[T2]]> {b,c,d,e} 21 {8,4,6,3} <![CDATA[T3]]> {a,c,d} 6 {3,1,2} <![CDATA[T4]]> {a,c,e,g} 29 {10,7,6,6} <![CDATA[T5]]> {b,c,e,g} 11 {4,2,3,2}

[0127] The method provided by the present invention scans the database row by row (by transaction), stores the transaction utility in the utility matrix in the reading order, and constructs tV, twuM and lM at the same time. The specific operation is shown as follows Figure 4 shown.

[0128] Figure 4 In the figure, it is shown that when scanning transaction T1, the transaction utility is first recorded, and then the utility value, TWU value, and u(i k After scanning the original database, the initial utility matrix is ​​shown in Table 2, the first transaction utility vector is shown in Table 3, the initial transaction weighted utility hash table is shown in Table 4, and the item set position hash table is shown in Table 5.

[0129] Table 2 Initial zero-place utility matrix

[0130]

[0131]

[0132] Table 3 First transaction utility vector

[0133] TID T T T T T tV 28 21 6 29 11

[0134] Table 4 Initial transaction weighted utility hash table

[0135] oeLh {f：28} {g：40} {d：55} {b：60} {a：63} {e：89} {c：95}

[0136] Table 5 Itemset location hash table

[0137] lM {a：0} {b：2} {c：4} {d：6} {e：8} {f：10} {g：12}

[0138] The user initially set the threshold to 20. Since the TWU value of each 1-item set is greater than the threshold, no 1-item set is discarded, and its utility value does not need to be deleted from the corresponding transaction utility. Therefore, the updated tV vector is shown in Table 6. In this process, the total utility value of each 1-item is also convenient to determine whether it belongs to a high-utility item set.

[0139] Table 6 Second transaction utility vector

[0140] TID <![CDATA[T1]]> <![CDATA[T2]]> <![CDATA[T3]]> <![CDATA[T4]]> <![CDATA[T5]]> tV 28 21 6 29 11

[0141] At this time, the remaining utility of the remaining item needs to be calculated in ascending order of TWU, such as Figure 5 The storage status of the final utility matrix after initialization is shown in Table 7.

[0142] Table 7 1 The zero-place utility matrix corresponding to a set of items including residual utility

[0143]

[0144]

[0145] According to the ascending order of TWU and the "depth first" rule, the item sets will be sorted according to Figure 6 order of expansion.

[0146] Since this iterative process is too cumbersome, only the expansion of the {aec} item and the utility matrix construction process are illustrated here. Since the {ae} item belongs to a high-utility item set and its extended item set may also become a high-utility item set, according to the judgment in step two, the pointer p does not point to the end of twuM at this time, so the ciV vector directly adds the item pointed to by pointer p, namely {c}, to obtain the {aec} item. According to step three, search the utility matrix u(c).row and u(ae).row. First, obtain U(u(c).row,0)+U(u(ae).row,0)+U(ru(c).row,0)=39, and judge that this value is greater than the threshold. Therefore, it is time to construct the utility matrix of the {aec} item, which is convenient for affairs. When U(u(c).row,T j )&&U(u(ae).row,T j )≠0, record the utility and residual utility of item {aec} in the corresponding position, and the operation is as follows Figure 7 The final construction results are shown in Table 8.

[0147] Table 8 Constructed zero-place utility matrix

[0148]

[0149]

[0150] Among them, during the construction of the utility matrix of the {aec} item, when scanning to transaction T2, since U(u(c).row,T2)&&U(u(ae).row,T2)=0, it is necessary to perform the second stage of the two-stage pruning, that is, U(u(c).row,0)+U(u(ae).row,0)+U(ru(c).row,0)-U(u(c).row,T2)=35>20. Since the upper bound still meets the pruning condition, the merge operation continues.

[0151] After the utility matrix of the {aec} item is constructed and in the previous search process, the algorithm should expand the item set and construct the matrix according to the proposed expansion method and expansion order, and thereby obtain all item sets that meet the threshold conditions.

[0152] The high-utility item set mining method provided by the present invention determines the initial zero-place utility matrix, the first transaction utility vector and the initial transaction weighted utility hash table corresponding to the 1-item set according to the acquired transaction database, the first transaction utility vector is used to record the utility of each transaction, the initial transaction weighted utility hash table is used to record the transaction weighted utility of each 1-item set, and the transaction weighted utility of each 1-item set is stored in ascending order; according to the initial utility matrix corresponding to the 1-item set, the item set position hash table is determined, and the item set position hash table records the position of each item set in the utility matrix; according to the preset threshold, the first transaction utility vector and the initial transaction weighted utility hash table are updated to obtain the second transaction utility vector and the item set position hash table. The weighted utility hash table of the target transaction is used; according to the utility vector of the second transaction, the item set position hash table, the weighted utility hash table of the target transaction and the initial utility matrix corresponding to the 1-item set, the zero-placeholder utility matrix containing the residual utility corresponding to the 1-item set is determined; based on the depth-first strategy, the item set is expanded according to the weighted utility hash table of the target transaction and the utility matrix containing the residual utility corresponding to the 1-item set, and the item set to be judged and the corresponding zero-placeholder utility matrix are obtained, and the high-utility item set is determined therefrom. The two-dimensional utility matrix structure based on the zero-placeholder matrix can simplify the construction and merging process of the utility list in the high-utility item set mining algorithm, reduce the computational complexity, improve the mining efficiency, and reduce resource occupation.

[0153] The present invention also provides a structure of an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus.

[0154] Memory, used to store computer programs;

[0155] The processor is used to implement the steps provided in the above method embodiment when executing the program stored in the memory.

[0156] The communication interface is used for communication between the above electronic device and other devices.

[0157] The method provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic device can be: a desktop computer, a portable computer, an intelligent mobile terminal, a server, etc. This is not limited here, and any electronic device that can implement the present invention belongs to the protection scope of the present invention.

[0158] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps provided in the above method embodiment are implemented.

[0159] As for the electronic device / storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the specific content and beneficial effects and other related matters can be referred to the partial description of the method embodiment.

[0160] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0161] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A high-utility item set mining method, characterized in that: include: According to the acquired transaction database, determine the initial utility matrix, the first transaction utility vector and the initial transaction weighted utility hash table corresponding to the 1-item set, wherein the utility matrix is ​​a zero-occupied matrix, the first transaction utility vector is used to record the utility of each transaction, and the initial transaction weighted utility hash table is used to record the transaction weighted utility of each 1-item set, and the transaction weighted utility of each 1-item set is stored in ascending order; According to the initial utility matrix corresponding to the 1-item set, determine an item set position hash table, the item set position hash table records the occupied rows of each item set in the utility matrix; updating the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold to obtain a second transaction utility vector and a target transaction weighted utility hash table; Determine a utility matrix including residual utility corresponding to the 1-itemset according to the second transaction utility vector, the item set position hash table, the target transaction weighted utility hash table, and the initial utility matrix corresponding to the 1-itemset; Based on the depth-first strategy, the item set is expanded according to the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-item set to obtain the item set to be judged, and it is determined whether the item set to be judged is a high-utility item set.

2. The method according to claim 1, characterized in that: Each element in the first column of the utility matrix is ​​expressed as: Where m(k,0) represents the element in the zero-placeholder utility matrix, k represents the matrix row, j represents the matrix column, and sumu(ik) represents the element i k The sum of the utility values ​​of element i. k The sum of the residual utility values ​​of u(ik).row represents the element i k Matrix row storing utility values, ru(ik).row represents element i k The matrix rows storing the residual utility values, Tj represents transaction j, ik represents item set k; The other elements except the first column in the even-numbered rows of the utility matrix are expressed as: Where m(2(k-1),j) represents the elements of the even-numbered rows in the zero-occupied matrix, and u(ik,Tj) represents the utility value of item set k in transaction j; The elements other than the first column in the odd rows of the utility matrix are expressed as: Among them, ru(ik,Tj) represents the residual utility value of item set k in transaction j.

3. The method according to claim 1, characterized in that The updating of the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold to obtain a second transaction utility vector and a target transaction weighted utility hash table includes: Determine whether there is a target element in the initial transaction weighted utility hash table whose transaction weighted utility of the 1-item set is less than the preset threshold; if so, delete the target element from the initial transaction weighted utility hash table to obtain the target transaction weighted utility hash table; Get the 1-itemset corresponding to the target element; The elements in the first transaction utility vector that include the 1-item set corresponding to the target element are obtained, and the utility values ​​of the 1-item set corresponding to the target element are deleted from the obtained elements to obtain the second transaction utility vector.

4. The method according to claim 1, characterized in that: The determining, according to the second transaction utility vector, the itemset position hash table, the target transaction weighted utility hash table and the initial utility matrix corresponding to the 1-itemset, a utility matrix including residual utility corresponding to the 1-itemset includes: Among them, ru(i k ,T j ) represents the residual utility of item set k in transaction j, i k represents item set k, ik' represents the item set after item set k in transaction j in ascending order of TWU, Tj represents transaction j, u(ik',Tj) represents the utility value of the item set after item set k in transaction j in ascending order of TWU ... k ,Tj) represents the utility value of item set k in transaction j, tu(T j ) represents the transaction utility of transaction j, TWU(i k ') represents the TWU value of the item set after item set k in transaction j, which is arranged in ascending order of TWU. TWU(i k ) represents the TWU value of item set k.

5. The method according to any one of claims 1 to 4, characterized in that: The depth-first strategy is based on which the item set is expanded according to the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-item set to obtain the item set to be judged, including: Decision 1: In the i-th round of expansion, determine whether the pointer p points to the end of the target transaction weighted utility hash table. If so, execute the steps in operation a; if not, execute the steps in decision 2; Decision 2: Determine whether the first return value corresponding to the i-1th round of expansion process is true. If so, execute the steps in operation b. If not, execute the steps in operation c. Operation a: Delete the last item of the i-1th to-be-judged itemset stored in the itemset vector ciV to obtain the first itemset vector, control the pointer p to search the last item of the first itemset vector in the target transaction weighted utility hash table and point to the search result, delete the last item of the first itemset vector, control the pointer p to move one position backward, add the item currently pointed to by the pointer p to the first itemset vector, and obtain the i-th to-be-judged itemset; Operation b: Delete the last item of the i-1th to-be-judged itemset stored in the itemset vector ciV to obtain a third itemset vector, control the pointer p to search for the last item of the third itemset vector in the target transaction weighted utility hash table and point to the search result, control the pointer p to move one position backward, add the item currently pointed to by the pointer p to the third itemset vector, and obtain the i-th to-be-judged itemset; Operation c: Add the item currently pointed to by pointer p to the end of the i-1th item set to be judged stored in the item set vector ciV to obtain the i-th item set to be judged.

6. The method according to claim 5, characterized in that The determining whether the item set to be determined is a high-utility item set includes: In the i-th round of expansion: S61, determining whether the i-th item set to be determined is a 1-item set, if so, setting the first return value to true, if not, executing S62; S62, judging whether the i-th item set to be judged satisfies the preset first-stage pruning strategy, if not, setting the first return value to false, if satisfied, executing S63; S63, constructing a utility matrix of the i-th item set to be judged according to the transaction; S64, based on a logical AND operation, determine whether the item set consisting of items other than the last item in the i-th item set to be determined and the last item in the i-th item set to be determined exist in the same transaction at the same time, if not, execute S65, if yes, execute S66; S65, judging whether the i-th item set to be judged satisfies the preset second-stage pruning strategy, if so, returning to execute S63, if not, setting the first return value to false; S66, recording the utility value and residual utility value of the i-th item set to be judged in the target transaction; S67, determining whether to traverse all transactions, if not, returning to execute S63, if yes, determining whether the i-th item set to be determined is a high-utility item set and whether the i-th item set to be determined needs to be expanded according to preset rules.

7. The method according to claim 5, characterized in that Before executing judgment 1, the method further includes: Initialization: Set the item set vector ciV to an empty set, control the pointer p to point to the start bit of the target transaction weighted utility hash table, set the first return value to true, and the item set vector ciV is used to store the item set to be judged.

8. The method according to claim 6 or 7, characterized in that: The method further comprises: Decision 3: In the i-th round of expansion, determine whether the first return value obtained in the i-th round is true, or whether the pointer p points to the end of the target transaction weighted utility hash table. If so, execute the step of operation d; if not, execute the step of decision 4; Decision 4: determine whether the pointer p points to the end of the target transaction weighted utility hash table, or determine whether the size of the item set vector ciV is 1. If so, execute the step of decision 1. If not, end. Operation d: Control the pointer p to move backward by one position, and execute the step of determination 4.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-8 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images