High utility item set mining method and apparatus

Abstract

The application provides a high-efficiency item set mining method and device, which comprises the following steps: determining an initial zero-occupancy utility matrix of a 1-item set, a first transaction utility vector and an initial transaction weighted utility hash table according to an obtained transaction database; determining an item set position hash table according to the initial utility matrix; updating the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold to obtain a second transaction utility vector and a target transaction weighted utility hash table; determining a zero-occupancy utility matrix of the 1-item set containing residual utility according to the second transaction utility vector, the item set position hash table, the target transaction weighted utility hash table and the initial utility matrix of the 1-item set; and performing item set expansion according to the target transaction weighted utility hash table and the zero-occupancy utility matrix containing residual utility based on a depth-first strategy to obtain to-be-judged item sets and corresponding zero-occupancy utility matrices, and determining high-utility item sets from the to-be-judged item sets and the corresponding zero-occupancy utility matrices, so that high-efficiency mining is realized.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a high-utility item set mining method and device. Background Art

[0002] With the advent of the information explosion, extracting valuable information from massive, unstructured data has become increasingly crucial in internet technology. Data mining is a computational process that uses specific algorithms to extract interesting, understandable, and actionable data from real, massive, and noisy data. Among these, the high-utility itemset mining algorithm stands out for its unique advantages. It inherits the fundamental concepts of data mining while also optimizing for data characteristics, focusing on frequency and utility calculation. The mined utility combinations not only represent frequent patterns of user behavior but also, in practice, reflect their high profit contribution to merchants or platforms, providing strong data support for precision marketing, inventory management, and user experience optimization. In summary, in the context of the information explosion, the high-utility itemset mining algorithm, with its efficient ability to process massive amounts of data, has become an indispensable data analysis tool in internet technology and across various industries, opening up new avenues for deep data mining and value discovery.

[0003] Current research on high-utility item set mining algorithms primarily focuses on two approaches: utility list-based and database mapping. Utility list-based approaches, which use a vertical structure, merge utility lists to complete the item set evaluation process. Database mapping approaches, which use a horizontal data structure, continuously compress the original database size through projection.

[0004] However, with the continuous expansion of data size, the problems of high-efficiency item set mining algorithms in terms of computational complexity and memory consumption have become increasingly prominent, becoming a bottleneck restricting their widespread application and in-depth development. First, the frequent scanning of the original database by high-efficiency item set mining algorithms is the main reason for their low computational efficiency. In the face of massive data, this traditional scanning mode is particularly cumbersome, not only time-consuming and laborious, but also likely to cause delays and blockages in data access, thereby affecting the real-time and accuracy of data mining. Second, the number of candidate item sets generated by the algorithm during the mining process is staggering, and each candidate item set needs to go through item set expansion and utility list construction to complete the complex utility evaluation process. This process not only involves a large amount of computation, but also involves the comprehensive consideration of multiple parameters and variables, increasing the complexity and uncertainty of the algorithm. This complexity not only prolongs the running time of the algorithm, but also may affect the accuracy and comprehensiveness of the mining results. High-efficiency item set mining algorithms also have drawbacks in terms of memory consumption. During the mining process, the algorithm generates a large number of intermediate results, including candidate item sets, utility lists, and remaining utility lists, which need to be stored in memory for subsequent processing. However, as the size of the data set expands, the number of intermediate results increases dramatically, putting a huge pressure on memory resources. When memory resources are insufficient, the running speed of the algorithm will decrease significantly, and it may even crash due to memory overflow. This not only wastes computing resources, but also may cause the failure of data mining tasks and the loss of data. SUMMARY

[0005] To solve the above problems existing in the prior art, the present application provides a high-efficiency item set mining method and device, specifically comprising:

[0006] In a first aspect, the present application provides a high-efficiency item set mining method, comprising:

[0007] According to the obtained transaction database, the initial utility matrix corresponding to the 1-item set, the first transaction utility vector, and the initial transaction weighted utility hash table are determined, the utility matrix is a zero placeholder matrix, the first transaction utility vector is used to record the utility of each transaction, and the initial transaction weighted utility hash table is used to record the transaction weighted utility of each 1-item set, and the transaction weighted utility of each 1-item set is stored in ascending order;

[0008] According to the initial utility matrix corresponding to the 1-item set, the item set position hash table is determined, which records the position of each item set in the utility matrix;

[0009] The first transaction utility vector and the initial transaction weighted utility hash table are updated according to the preset threshold, to obtain the second transaction utility vector and the target transaction weighted utility hash table;

[0010] Determine the utility matrix corresponding to the 1-itemset containing the residual utility based on the second transaction utility vector, the item set position hash table, the target transaction weighted utility hash table, and the initial utility matrix corresponding to the 1-itemset;

[0011] Based on the depth-first strategy, the item set is expanded according to the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-item set to obtain the item set to be judged, and determine whether the item set to be judged is a high-utility item set.

[0012] In a second aspect, the present invention further provides an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0013] Memory for storing computer programs;

[0014] The processor is configured to implement any one of the methods provided in the first aspect when executing a program stored in the memory.

[0015] In a third aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any one of the methods provided in the first aspect is implemented.

[0016] Beneficial effects of the present invention:

[0017] The high-utility item set mining method and device provided by the present invention determines the initial utility matrix, the first transaction utility vector and the initial transaction weighted utility hash table corresponding to the 1-item set based on the acquired transaction database, wherein the utility matrix is ​​a zero-occupied matrix, the first transaction utility vector is used to record the utility of each transaction, the initial transaction weighted utility hash table is used to record the transaction weighted utility of each 1-item set, and the transaction weighted utility of each 1-item set is stored in ascending order; according to the initial utility matrix corresponding to the 1-item set, the item set position hash table is determined, and the item set position hash table records the position of each item set in the utility matrix; the first transaction utility vector and the initial transaction weighted utility hash table are updated according to a preset threshold, and the high-utility item set mining method and device provided by the present invention determines the initial utility matrix, and the item set position hash table records the position of each item set in the utility matrix; the first transaction utility vector and the initial transaction weighted utility hash table are updated according to the preset threshold, and the high-utility item set mining method and device provided by the present invention determines the initial utility matrix, and ... to the second transaction utility vector and the target transaction weighted utility hash table; according to the second transaction utility vector, the item set position hash table, the target transaction weighted utility hash table and the initial utility matrix corresponding to the 1-item set, the utility matrix containing the residual utility corresponding to the 1-item set is determined; based on the depth-first strategy, the item set is expanded according to the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-item set to obtain the item set to be judged, and determine whether the item set to be judged is a high-utility item set. Its two-dimensional utility matrix structure based on the zero-occupancy matrix can simplify the construction and merging process of the utility list in the high-utility item set mining algorithm, reduce the computational complexity, improve the mining efficiency, and reduce resource occupation.

[0018] The application will be described in further detail below with reference to the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A flowchart of a high utility item set mining method provided by the application;

[0020] Figure 2 A process diagram for determining a to-be-judged item set provided by the application;

[0021] Figure 3 A process diagram for determining a high utility item set from a to-be-judged item set provided by the application;

[0022] Figure 4 A process diagram for an experiment provided by the application;

[0023] Figure 5 Another process diagram for an experiment provided by the application;

[0024] Figure 6 An experimental data diagram provided by the application;

[0025] Figure 7 Still another process diagram for an experiment provided by the application. DETAILED DESCRIPTION

[0026] The application will be described in further detail below with reference to the drawings and embodiments, but the embodiments of the application are not limited thereto.

[0027] To solve the problems in the prior art, the application provides a high utility item set mining method, which adopts a two-dimensional utility matrix structure based on a zero-occupancy matrix, can simplify the construction and merging process of the utility list in the high utility item set mining algorithm, and uses a pointer-maintained prefix-free item set extension method to reduce the cost of executing each connection. Meanwhile, the method proposes a novel two-stage local pruning strategy to compress the search space.

[0028] Figure 1 A flowchart of a high utility item set mining method provided by the application is shown in FIG. 1, and the method comprises the following steps. Figure 1

[0029] S101, determining an initial utility matrix corresponding to a 1-item set, a first transaction utility vector and an initial transaction weighted utility hash table according to an acquired transaction database.

[0030] The utility matrix is a zero-occupancy matrix.

[0031] The first transaction utility vector is used to record the utility of each transaction.

[0032] ​The initial transaction weighted utility hash table is used to record the transaction weighted utility of each 1-item set, and the transaction weighted utility of each 1-item set is stored in ascending order.

[0033] In one possible implementation, the elements in the first column of the utility matrix are expressed as:

[0034]

[0035] Among them, m(k,0) represents the element in the zero-placeholder utility matrix, k represents the matrix row, j represents the matrix column, sumu(i k ) represents element i k The sum of the utility values, sumru(i k ) represents element i k The sum of the residual utility values, u(i k ).row represents element i k The matrix rows storing utility values, ru(i k ).row represents element i k Matrix rows storing residual utility values, T j Represents transaction j, i k represents the item set k.

[0036] The elements in the even rows of the utility matrix except the first column are expressed as:

[0037]

[0038] Among them, m(2(k-1),j) represents the elements of the even rows in the zero-occupied matrix, u(i k ,T j ) represents the utility value of item set k in transaction j;

[0039] The elements in the odd rows of the utility matrix except the first column are expressed as:

[0040]

[0041] Among them, ru(i k ,T j ) represents the residual utility value of item set k in transaction j. Specifically, the original database is scanned by transaction, and the transaction utility of each transaction is recorded in the first transaction utility vector (tV vector).

[0042] The tV vector is defined as:

[0043]

[0044] Further, the utility value of the 1-item included in each transaction is stored in the utility matrix. The utility matrix is for a database with j transactions and k items, and the utility matrix of the 1-item is composed of 2k rows and j+1 columns, and the rows and columns of the matrix represent utility, remaining utility and Tid (Transaction ID) respectively. Define u(i k ).row is a row storing u(i k ).row is a row storing ru(i k ).row is a row storing ru(i k ). Thus, the utility matrix of the 1-item is defined as:

[0045] Even rows:

[0046]

[0047] Odd rows:

[0048]

[0049] The first column of the utility matrix stores the sum of the utility after the corresponding row, that is:

[0050]

[0051] When scanning the original database, only the even rows in the matrix are stored, that is, only the utility of the 1-item is recorded, and at the same time, the utility value is accumulated to (u(i k ).row, 0) for recording the sum of the utility values of the 1-item.

[0052] S102, determining the item set position hash table according to the initial utility matrix corresponding to the 1-item set.

[0053] The item set position hash table is used to record the position of each item set in the utility matrix.

[0054] Specifically, two auxiliary hash tables for calculation are constructed: an initial transaction weighted utility hash table (twuM) and an item set position hash table (lM).

[0055] The twuM is used to record the transaction weighted utility (TWU) of the 1-item, and is stored in ascending order of TWU. It is defined as:

[0056]

[0057] Define ru(i k ).row=(u(i k ).row+1); thus lM only needs to store u(i k).row facilitates the location of items when searching and is defined as:

[0058] lM={{i1:u(i1).row}}...{i k :u(i k ).row}}.

[0059] S103: Update the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold to obtain a second transaction utility vector and a target transaction weighted utility hash table.

[0060] In one possible implementation, updating the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold to obtain the second transaction utility vector and the target transaction weighted utility hash table includes:

[0061] A1. Determine whether there is a target element in the initial transaction weighted utility hash table whose transaction weighted utility of the 1-item set is less than a preset threshold. If so, delete the target element from the initial transaction weighted utility hash table to obtain a target transaction weighted utility hash table.

[0062] A2. Get the 1-itemset corresponding to the target element.

[0063] A3. Obtain the elements in the first transaction utility vector that contain the 1-itemset corresponding to the target element, and delete the utility value of the 1-itemset corresponding to the target element from the obtained elements to obtain a second transaction utility vector.

[0064] Specifically, since the twuM is initialized during database scanning, the algorithm directly traverses the hash table to implement TWU pruning. This means that all 1-items with TWU values ​​below a threshold are not worth mining and must be discarded. Since TWU values ​​and corresponding 1-items are stored in pairs in the twuM, if an item's TWU value falls below the threshold during traversal, both items are discarded. All transactions containing that item also have their utility value subtracted from their own transaction utility.

[0065] S104 , determining a utility matrix including residual utility corresponding to the 1-itemset according to the second transaction utility vector, the itemset position hash table, the target transaction weighted utility hash table, and the initial utility matrix corresponding to the 1-itemset.

[0066] In one possible implementation, determining a utility matrix including residual utility corresponding to the 1-itemset based on the second transaction utility vector, the item set position hash table, the target transaction weighted utility hash table, and the initial utility matrix corresponding to the 1-itemset includes:

[0067]

[0068] Among them, ru(ik ,T j ) represents the residual utility of item set k in transaction j, i k Represents item set k, i k ' represents the itemsets after itemset k in transaction j in ascending order of TWU, T j Represents transaction j, u(i k ',T j ) represents the utility value of the item set after item set k in transaction j in ascending order of TWU, u(i k ,T j ) represents the utility value of item set k in transaction j, tu(T j ) represents the transaction utility of transaction j, TWU(i k ') represents the TWU value of the item set after item set k in transaction j in ascending order of TWU, TWU(i k ) represents the TWU value of item set k.

[0069] Specifically, the residual utility of an item in each transaction is defined as the sum of the utilities of all items following the item after being sorted in ascending order of TWU.

[0070] The definition and actual calculation method are:

[0071]

[0072] Therefore, the calculation of the residual utility requires calling the utility of each transaction in ascending order of TWU. That is, when traversing twuM, call the transaction containing the 1-item, subtract the utility of the 1-item from the transaction utility, update this value to the tV vector, and store it in the corresponding position in the utility matrix (ru(i k ).row,T j ). During the calculation process, if the 1-item does not exist in the transaction, then its corresponding residual utility does not exist, and there is no need to perform redundant calculations to waste running time and memory. The judgment basis of this condition is that if the item corresponds to At the same time, each residual utility in the calculation process will be accumulated to (ru(i k ).row, 0), is used to record the sum of the residual utility values ​​of the 1-item.

[0073] S105 , based on a depth-first strategy, expand the itemset according to the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-itemset to obtain the itemset to be judged, and determine whether the itemset to be judged is a high-utility itemset.

[0074] In one possible implementation, based on a depth-first strategy, the itemset is expanded according to the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-itemset to obtain the itemset to be judged, including:

[0075] Decision 1: In the i-th round of expansion, determine whether pointer p points to the end of the target transaction weighted utility hash table. If so, execute the steps in operation a; if not, execute the steps in decision 2.

[0076] Decision 2: Determine whether the first return value corresponding to the i-1th round of expansion process is true. If so, execute the steps in operation b; if not, execute the steps in operation c;

[0077] Operation a: Delete the last item of the i-1th to-be-judged itemset stored in the itemset vector ciV to obtain the first itemset vector. The control pointer p searches for the last item of the first itemset vector in the target transaction weighted utility hash table and points to the search result. Delete the last item of the first itemset vector. The control pointer p moves one position backward and adds the item currently pointed to by pointer p to the first itemset vector to obtain the i-th to-be-judged itemset.

[0078] Operation b: Delete the last item of the i-1th to-be-judged itemset stored in the itemset vector ciV to obtain the third itemset vector. The control pointer p searches for the last item of the third itemset vector in the target transaction weighted utility hash table and points to the search result. The control pointer p moves one position backward and adds the item currently pointed to by pointer p to the third itemset vector to obtain the i-th to-be-judged itemset.

[0079] Operation c: Add the item currently pointed to by pointer p to the end of the i-1th item set to be judged stored in the item set vector ciV to obtain the i-th item set to be judged.

[0080] In the i-th round of expansion, in one possible implementation, determining whether the item set to be determined is a high-utility item set includes:

[0081] S61, determine whether the i-th item set to be determined is a 1-item set, if so, set the first return value to true, if not, execute S62.

[0082] S62, determine whether the i-th item set to be determined meets the preset first-stage pruning strategy. If not, set the first return value to false. If satisfied, execute S63.

[0083] S63: Construct a utility matrix of the i-th item set to be judged according to the transaction.

[0084] S64, based on the logical AND operation, determine whether the item set consisting of items other than the last item in the i-th item set to be determined and the last item of the i-th item set to be determined exist in the same transaction at the same time; if not, execute S65; if so, execute S66.

[0085] S65, judging whether the i-th item set to be judged satisfies a preset second-stage pruning strategy, if yes, returning to execute S63, if no, setting the first return value to false.

[0086] S66, recording the utility value and the residual utility value of the i-th item set to be judged in the target transaction;

[0087] S67, determining whether all transactions are traversed, if no, returning to execute S63, if yes, determining whether the i-th item set to be judged is a high-utility item set and whether the i-th item set to be judged has an expansion necessity according to a preset rule.

[0088] The preset rule comprises:

[0089] if sumu(Pxy) > minutil, the item set {Pxy} to be judged is a high-utility item set; and if sumu(Pxy) + sumru(Pxy) > minutil, the item set to be judged has an expansion necessity.

[0090] sumu(Pxy) represents the sum of the utility values of the i-th item set to be judged, minutil represents a preset threshold, and sumru(Pxy) represents the sum of the residual utility values of the i-th item set to be judged.

[0091] Further, before executing the judging 1, the method further comprises:

[0092] initialization: setting an item set vector ciV as an empty set, setting a control pointer p to point to the start bit of the target transaction weighted utility hash table, and setting a first return value as true, wherein the item set vector ciV is used for storing the item set to be judged.

[0093] Further, the method further comprises:

[0094] judging 3: in the i-th round of expansion, determining whether the first return value obtained in the i-th round is true or whether the control pointer p points to the end of the target transaction weighted utility hash table, if yes, executing the step of operation d, and if no, executing the step of judging 4;

[0095] judging 4: judging whether the control pointer p points to the end of the target transaction weighted utility hash table or whether the size of the item set vector ciV is 1, if yes, executing the step of judging 1, and if no, ending;

[0096] operation d: moving the control pointer p by one bit and executing the step of judging 4.

[0097] By using the pointer-maintained prefix-free item set expansion manner, the cost of executing each connection can be reduced, and the operation efficiency can be improved; by using the two-stage local pruning strategy, the search space can be compressed, the data amount can be reduced, and resource waste can be reduced.

[0098] Exemplarily, step S105 can be divided into two steps, where step 1 and step 2 are nested and executed, as follows:

[0099] like Figure 2 As shown, step one includes:

[0100] In the subsequent mining process of high-utility itemsets, the algorithm needs to expand the itemsets according to the "depth-first" strategy to obtain the itemsets to be judged. This paper proposes a prefix-free itemset expansion method maintained by pointers, vectors, and hash tables. The specific implementation method is as follows:

[0101] Define the item set vector as ciV = {I}, which stores the item set to be judged. When step 2 is executed for the first time, define Define pointer p to point to twuM and obtain the 1-item of the pointed bit. When step 2 is executed for the first time, define p to point to the start bit of twuM. And when step 2 is executed for the first time, define the return value to be true.

[0102] The relevant judgment operations in the method are:

[0103] Judgment 1: First determine whether the pointer p points to the end of twuM. If true, perform operation a; if false, perform judgment 2.

[0104] Judgment 2: Determine whether the return value of the last execution of step 3 is true. If it is true, perform operation b; if it is false, perform operation c.

[0105] Judgment 3: Determine whether the return value of step 3 is true or whether the pointer p points to the end of twuM. If true, perform operation d; if false, perform judgment 4.

[0106] Judgment 4: Determine whether the pointer p points to the end of twuM or the size of the ciV vector is 1. If true, perform judgment 1, otherwise exit the loop.

[0107] The operations after the relevant judgment operations in the method are:

[0108] Operation a: cIV deletes the last item, pointer p searches for the last item of the ciV vector in twuM and points to it, cIV deletes the last item again, pointer p moves one position backward, and the item currently pointed to by pointer p is added to the cIV vector, and step 2 is performed.

[0109] Operation b: cIV deletes the last item, pointer p searches for the last item of the ciV vector in twuM and points to it, pointer p moves one position backward, adds the item currently pointed to by pointer p to the cIV vector, and proceeds to step 2.

[0110] Operation c: Add the item currently pointed to by pointer p to the cIV vector and proceed to step 2.

[0111] Operation d: Move pointer p one position backward and perform judgment 4.

[0112] like Figure 3 As shown, step two includes:

[0113] After obtaining the set of items to be judged, the algorithm needs to construct its corresponding two-row utility matrix for judgment. Let the set of items to be judged be {Pxy}.

[0114] If the item set to be judged is a 1-item set, the true value is directly returned to judgment 3 in step 1, and the subsequent operations of step 2 are not performed.

[0115] Since the construction process is complex, in order to save resources, the first stage of pruning is performed to determine whether {Pxy} is worthy of constructing the utility matrix.

[0116] The pruning strategy for the first phase is:

[0117]

[0118] If not satisfied, return false and return to judgment 3 in step 1, and do not perform subsequent operations in step 2. If satisfied, start constructing the utility matrix of {Pxy}.

[0119] According to the prefix-free item set expansion, the construction of the utility matrix of {Pxy} requires the convenience of the utility values ​​and residual utility values ​​of {Px} and {y}.

[0120] Due to the characteristics of zero-place utility matrix, the present invention proposes a merging method based on "logical AND" operation. j When u(Px,T j )&&u(y,T j )==1 means {Px} and {y} exist in transaction T at the same time j , at this time u(Pxy,T j )=u(Px,T j )+u(y,T j ),ru(Pxy,T j )=ru(y,T j ). And store the calculated utility values ​​into (u(Pxy).row, T j ), (ru(Pxy).row, T j ), then add to (u(Pxy).row, 00, (ru(Pxy).row, 0). If u(Px,T t )&&u(y,T t )==0 means {Px} and {y} do not exist in transaction T at the same time t , at this time, a two-stage pruning strategy is required, namely

[0121] If the condition is met, the utility list merging operation is continued; if not, a false return is made to judgment 3 in step 1, and subsequent operations in step 2 are not performed to start expanding the next set of items to be judged.

[0122] After traversing all transactions, if sumu(Pxy)>minutil, then {Pxy} is a high-utility itemset; if sumu(Pxy)+sumru(Pxy)>minutil, then {Pxy}' may become a high-utility itemset, return a true value to judgment 3 in step 1, and start the expansion of the next itemset; if {Pxy}' cannot become a high-utility itemset, return a false value to judgment 3 in step 1.

[0123] In order to facilitate understanding of the present invention, a set of examples are also provided, which are as follows:

[0124] The existing original database is shown in the following table. Assume that the user's initial threshold is 20.

[0125] Table 1 Original database

[0126] Transaction Item Transaction Utility (Tu) Utility <![CDATA[T1]]> {a, b, c, d, e, f} 28 {5,8,1,6,3,5} <![CDATA[T2]]> {b, c, d, e} 21 {8,4,6,3} <![CDATA[T3]]> {a, c, d} 6 {3,1,2} <![CDATA[T4]]> {a, c, e, g} 29 {10,7,6,6} <![CDATA[T5]]> {b, c, e, g} 11 {4,2,3,2}

[0127] The method provided by the present invention scans the database row by row (by transaction) and stores the transaction utility into the utility matrix in the order of reading, and constructs tV, twuM and lM at the same time. The specific operation is shown as follows Figure 4 shown.

[0128] Figure 4 In the figure, when scanning transaction T1, the utility of the transaction is first recorded, and then the utility value, TWU value and u(i k After scanning the original database, the initial utility matrix is ​​shown in Table 2 below, the first transaction utility vector is shown in Table 3, the initial transaction weighted utility hash table is shown in Table 4, and the item set position hash table is shown in Table 5.

[0129] Table 2 Initial zero-occupancy utility matrix

[0130]

[0131]

[0132] Table 3 First transaction utility vector

[0133] TID T T T T T tV 28 21 6 29 11

[0134] Table 4 Initial transaction weighted utility hash table

[0135] twuM {f：28} {g：40} {d：55} {b：60} {a：63} {e：89} {c：95}

[0136] Table 5 Itemset location hash table

[0137] lM {a：0} {b：2} {c：4} {d：6} {e：8} {f：10} {g：12}

[0138] The user initially set a threshold of 20. Since the TWU value of each item set is greater than this threshold, no item set is discarded, and its utility value does not need to be deleted from the corresponding transaction utility. Therefore, the updated tV vector is shown in Table 6. This process also facilitates the total utility value of each item to determine whether it belongs to a high-utility item set.

[0139] Table 6 Second transaction utility vector

[0140] TID <![CDATA[T1]]> <![CDATA[T2]]> <![CDATA[T3]]> <![CDATA[T4]]> <![CDATA[T5]]> tV 28 21 6 29 11

[0141] At this time, the remaining utility of the remaining item needs to be calculated in ascending order of TWU, such as Figure 5 The storage status of the final utility matrix after initialization is shown in Table 7.

[0142] Table 7.1 Zero-placeholder utility matrix including residual utility corresponding to one item set

[0143]

[0144]

[0145] According to the ascending order of TWU and the "depth first" rule, the item sets will be sorted according to Figure 6 order of expansion.

[0146] Since this iterative process is too cumbersome, only the expansion of the {aec} item and the utility matrix construction process are illustrated here. Since the {ae} item belongs to the high-utility item set and its extended item set may also become a high-utility item set, according to the judgment in step 2, the pointer p does not point to the end of twuM at this time, so the ciV vector directly adds the item pointed to by pointer p, namely {c}, to obtain the {aec} item. According to step 3, search the utility matrix u(c).row and u(ae).row. First, obtain U(u(c).row,0)+U(u(ae).row,0)+U(ru(c).row,0)=39, and judge that this value is greater than the threshold, so it is time to construct the utility matrix of the {aec} item, which is convenient for affairs. When U(u(c).row,T j )&&U(u(ae).row,T j )≠0, record the utility and residual utility of item {aec} in the corresponding position, and the operation is as follows Figure 7 The final construction results are shown in Table 8.

[0147] Table 8 Constructed zero-place utility matrix

[0148]

[0149]

[0150] During the utility matrix construction process for the {aec} item, when the scan reaches transaction T2, U(u(c).row,T2)&&U(u(ae).row,T2)=0. Therefore, the second stage of the two-stage pruning process is required: U(u(c).row,0)+U(u(ae).row,0)+U(ru(c).row,0)-U(u(c).row,T2)=35>20. Since this upper bound still meets the pruning condition, the merge operation continues.

[0151] After the utility matrix of the {aec} item is constructed and in the previous search process, the algorithm should expand the item set and construct the matrix according to the proposed expansion method and expansion order, and thereby obtain all item sets that meet the threshold conditions.

[0152] The high-utility item set mining method provided by the present invention determines the initial zero-placeholder utility matrix, the first transaction utility vector and the initial transaction weighted utility hash table corresponding to the 1-item set based on the acquired transaction database, the first transaction utility vector is used to record the utility of each transaction, the initial transaction weighted utility hash table is used to record the transaction weighted utility of each 1-item set, and the transaction weighted utility of each 1-item set is stored in ascending order; according to the initial utility matrix corresponding to the 1-item set, the item set position hash table is determined, and the item set position hash table records the position of each item set in the utility matrix; the first transaction utility vector and the initial transaction weighted utility hash table are updated according to a preset threshold to obtain the second transaction utility vector and the item set position hash table. The target transaction weighted utility hash table is used; according to the second transaction utility vector, the item set position hash table, the target transaction weighted utility hash table and the initial utility matrix corresponding to the 1-item set, the zero-placeholder utility matrix containing the residual utility corresponding to the 1-item set is determined; based on the depth-first strategy, the item set is expanded according to the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-item set to obtain the item set to be judged and the corresponding zero-placeholder utility matrix, and the high-utility item set is determined therefrom. The two-dimensional utility matrix structure based on the zero-placeholder matrix can simplify the construction and merging process of the utility list in the high-utility item set mining algorithm, reduce the computational complexity, improve the mining efficiency and reduce resource occupation.

[0153] The present invention also provides a structure of an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus.

[0154] Memory for storing computer programs;

[0155] The processor is configured to implement the steps provided in the above method embodiment when executing the program stored in the memory.

[0156] The communication interface is used for communication between the above electronic device and other devices.

[0157] The method provided in the embodiments of the present invention can be applied to electronic devices. Specifically, the electronic devices can be desktop computers, portable computers, smart mobile terminals, servers, etc. This is not limited here; any electronic device that can implement the present invention falls within the scope of protection of the present invention.

[0158] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps provided in the above method embodiment are implemented.

[0159] As for the electronic device / storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the specific content and beneficial effects, please refer to the partial description of the method embodiment.

[0160] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature specified as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0161] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A high-utility item set mining method, characterized in that: include: Based on the acquired transaction database, determine the initial utility matrix, first transaction utility vector, and initial transaction weighted utility hash table corresponding to the 1-item set. The utility matrix is ​​a zero-occupied matrix. The first transaction utility vector is used to record the utility of each transaction. The initial transaction weighted utility hash table is used to record the transaction weighted utility of each 1-item set, and the transaction weighted utility of each 1-item set is stored in ascending order. The elements in the first column of the utility matrix are expressed as follows: ， in, represents the element in the zero-placeholder utility matrix, represents the matrix rows, represents the matrix columns, Represents element i k The sum of the utility values ​​of Represents element i k The sum of the residual utility values ​​of Represents element i k Matrix rows storing utility values, Represents element i k The matrix rows storing the residual utility values, represents transaction j, represents item set k; The other elements except the first column in the even-numbered rows of the utility matrix are expressed as: ， in, represents the elements of the even rows in the zero-occupied matrix, represents the utility value of item set k in transaction j; The elements other than the first column in the odd-numbered rows of the utility matrix are expressed as: ， in, represents the residual utility value of item set k in transaction j; Determine an item set position hash table according to the initial utility matrix corresponding to the 1-item set, wherein the item set position hash table records the occupied rows of each item set in the utility matrix; updating the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold to obtain a second transaction utility vector and a target transaction weighted utility hash table; Determining a utility matrix including residual utility corresponding to the 1-itemset according to the second transaction utility vector, the itemset position hash table, the target transaction weighted utility hash table, and the initial utility matrix corresponding to the 1-itemset; Based on a depth-first strategy, an item set is expanded according to the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-item set to obtain an item set to be judged, and it is determined whether the item set to be judged is a high-utility item set.

2. The method according to claim 1, characterized in that The updating of the first transaction utility vector and the initial transaction weighted utility hash table according to a preset threshold to obtain a second transaction utility vector and a target transaction weighted utility hash table includes: Determine whether there is a target element in the initial transaction weighted utility hash table whose transaction weighted utility of the 1-item set is less than the preset threshold; if so, delete the target element from the initial transaction weighted utility hash table to obtain the target transaction weighted utility hash table; Get the 1-itemset corresponding to the target element; The element containing the 1-itemset corresponding to the target element in the first transaction utility vector is obtained, and the utility value of the 1-itemset corresponding to the target element is deleted from the obtained element to obtain the second transaction utility vector.

3. The method according to claim 1, characterized in that The determining, based on the second transaction utility vector, the itemset position hash table, the target transaction weighted utility hash table, and the initial utility matrix corresponding to the 1-itemset, a utility matrix corresponding to the 1-itemset containing residual utility includes: ， in, represents the residual utility of item set k in transaction j, represents the item set k, represents the itemset following itemset k in transaction j in ascending order of TWU, represents transaction j, It represents the utility value of the item set after item set k in transaction j in ascending order of TWU. represents the utility value of item set k in transaction j, represents the transaction utility of transaction j, It represents the TWU value of the item set after item set k in transaction j, sorted in ascending order of TWU. Represents the TWU value of item set k.

4. The method according to any one of claims 1 to 3, characterized in that: The depth-first strategy is based on the target transaction weighted utility hash table and the utility matrix containing the residual utility corresponding to the 1-item set to expand the item set to obtain the item set to be judged, including: Decision 1: In the i-th round of expansion, determine whether pointer p points to the end of the target transaction weighted utility hash table. If so, execute the steps in operation a; if not, execute the steps in decision 2. Decision 2: Determine whether the first return value corresponding to the i-1th round of expansion process is true. If so, execute the steps in operation b; if not, execute the steps in operation c; Operation a: Delete the last item of the i-1th to-be-determined itemset stored in the itemset vector ciV to obtain a first itemset vector, control the pointer p to search for the last item of the first itemset vector in the target transaction weighted utility hash table and point it to the search result, delete the last item of the first itemset vector, control the pointer p to move one position backward, add the item currently pointed to by the pointer p to the first itemset vector, and obtain the i-th to-be-determined itemset; Operation b: Delete the last item of the i-1th to-be-judged itemset stored in the itemset vector ciV to obtain a third itemset vector, control the pointer p to search for the last item of the third itemset vector in the target transaction weighted utility hash table and point it to the search result, control the pointer p to move one position backward, add the item currently pointed to by the pointer p to the third itemset vector, and obtain the i-th to-be-judged itemset; Operation c: Add the item currently pointed to by pointer p to the end of the i-1th item set to be judged stored in the item set vector ciV to obtain the i-th item set to be judged.

5. The method according to claim 4, characterized in that The determining whether the to-be-judged itemset is a high-utility itemset includes: In the i-th round of expansion: S61, determining whether the i-th item set to be determined is a 1-item set, if so, setting the first return value to true, if not, executing S62; S62, determining whether the i-th item set to be determined satisfies the preset first-stage pruning strategy; if not, setting the first return value to false; if satisfied, executing S63; S63, constructing a utility matrix of the i-th item set to be judged according to the transaction; S64, based on a logical AND operation, determining whether the item set consisting of items other than the last item in the i-th item set to be determined and the last item in the i-th item set to be determined are both in the same transaction; if not, executing S65; if so, executing S66; S65, determining whether the i-th item set to be determined satisfies the preset second-stage pruning strategy. If so, returning to S63; if not, setting the first return value to false. S66, recording the utility value and residual utility value of the i-th item set to be judged in the target transaction; S67, determining whether all transactions have been traversed, if not, returning to executing S63, if yes, determining whether the i-th item set to be determined is a high-utility item set and whether the i-th item set to be determined needs to be expanded according to preset rules.

6. The method according to claim 4, characterized in that Before executing judgment 1, the method further includes: Initialization: Setting the itemset vector is an empty set, the control pointer p points to the start bit of the target transaction weighted utility hash table, the first return value is set to true, and the item set vector Used to store the set of items to be judged.

7. The method according to claim 5 or 6, characterized in that The method further comprises: Decision 3: In the i-th round of expansion, determine whether the first return value obtained in the i-th round is true, or whether the pointer p points to the end of the target transaction weighted utility hash table. If so, execute the step of operation d; if not, execute the step of decision 4; Decision 4: Determine whether the pointer p points to the end of the target transaction weighted utility hash table, or determine the item set vector Is the size 1? If so, execute the step 1 of judgment; if not, end; Operation d: Control the pointer Move back one position and execute the step of judgment 4.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 7 when executing a program stored in a memory.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images