A fast and efficient method for mining quantitative itemsets based on length constraints

Abstract

The present invention relates to a fast, high-utility quantitative item set mining method based on length constraints. The method comprises obtaining a quantitative transaction database, a minimum utility threshold, a minimum length, a maximum length, a quantitative correlation coefficient, and a merging method; performing a first scan of the quantitative transaction database to calculate the LTWU of all quantitative item sets, pruning them based on the LTWU, and storing the unpruned quantitative item sets in a set of promising quantitative item sets; sorting the promising quantitative item sets in ascending order of LTWU; performing a second scan of the revised database to create a redefined utility list and an LTQCS structure; storing the promising quantitative item sets in separate sets based on the utility values ​​of each item in the redefined utility list; merging weak-utility quantitative item sets, high-utility quantitative item sets, and candidate quantitative item sets; and outputting the high-utility quantitative item sets in the merged set based on the minimum and maximum lengths. This method can filter out itemsets with high utility but that do not meet the length constraints and can be applied to stock analysis scenarios.

Description

Technical Field

[0001] The present invention belongs to the field of computer information technology processing, and in particular relates to a fast and efficient utility quantized item set mining method based on length constraints. Background Art

[0002] The rapid development of information technologies such as the Internet, the Internet of Things, and cloud computing has led to the continuous integration of information technology with traditional applications in politics, economics, military affairs, scientific research, and everyday life, generating vast amounts of data unprecedented in history. Furthermore, smart mobile devices, sensors, e-commerce websites, and social networks around the world continuously generate a wide variety of data. Faced with this vast amount of data, how to analyze it promptly and effectively, and extract underlying patterns closely related to people's lifestyles, is a pressing issue for governments and businesses in the information age. To address this challenge, data mining technology has emerged.

[0003] Data mining refers to the process of finding important, unknown, and potentially useful patterns in databases. In simple terms, it involves discovering information of interest to users from a dataset, processing this information to obtain useful knowledge, and ultimately using this knowledge to guide future production or implementation. In many real-life applications, data mining techniques are used to extract valuable information from databases to support decision-making. High-utility itemset mining is a technique used in data mining to discover relationships between data. In the commercial service sector, utility represents the profit of a particular combination of products. High-utility itemset mining can identify sets of items in the data with relatively high utility values—high-utility itemsets. Consequently, high-utility itemset mining has garnered increasing attention and research in recent years.

[0004] Current high-utility itemset mining techniques primarily aim to discover all sets of items whose utility is no less than a user-defined minimum threshold. This technique comprehensively considers the number of item occurrences in transactions and the user's weighted preferences for individual items, but does not address the quantitative relationships between items. Therefore, in 2007, Yen et al. first introduced the concept of weakly utility-quantified itemsets in the HUQA algorithm and proposed a k-support bound method for pruning, laying the foundation for subsequent high-utility-quantified itemset mining. In 2014, Li et al. proposed the VHUQI algorithm, which uses a utility list structure to store item set utility information, further improving the efficiency of high-utility-quantified item mining. However, this algorithm only uses the k-support bound method for pruning, which does not significantly reduce the search space. In 2019, Li et al. proposed the HUQI-Miner algorithm, which utilizes transaction-weighted utility (TWU) and residual utility (RU) to rapidly eliminate low-utility-quantified itemsets. Experimental results show that the HUQI-Miner algorithm significantly outperforms the previous two algorithms, but it still requires a large number of union calculations during the mining process, resulting in a long runtime. In 2021, Mourad et al. proposed a new improved algorithm, the FHUQI-Miner algorithm, and improved and expanded the EUCP structure used in the FHM algorithm. They proposed two new pruning strategies for processing quantized item sets, greatly reducing the number of union operations.

[0005] Current high-utility quantitative itemset mining techniques can only mine high-utility quantitative itemsets with quantitative relationships, but do not consider long itemsets. This is because long itemsets generally represent special cases with a limited frequency of occurrence, and storing long itemsets consumes a large amount of memory and time during the mining process. Summary of the Invention

[0006] In order to solve the above problems existing in the prior art, the present invention provides a fast and efficient utility quantized itemset mining method based on length constraints. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] The embodiment of the present invention provides a fast and efficient utility quantized itemset mining method based on length constraints, comprising the steps of:

[0008] S1. Obtain the quantitative transaction database, customized minimum utility threshold, minimum length, maximum length, quantitative correlation coefficient, and merging method;

[0009] S2. Performing a first scan on the quantitative transaction database to calculate the maximum transaction weighted utility of all quantized item sets, and pruning all quantized item sets based on the maximum transaction weighted utility property using the minimum utility threshold and the quantitative correlation coefficient, and storing the pruned quantized item sets into a promising quantized item set set;

[0010] S3. Arranging the pruned quantized item sets in ascending order according to the values ​​of the maximum transaction weighted utilities, and revising the quantitative transaction database according to the arrangement results to obtain a revised database;

[0011] S4. Scan the revised database a second time to create a redefined utility list and LTQCS structure;

[0012] S5. Dividing the promising quantized item set into a high-utility quantized item set, a candidate quantized item set, and a weak-utility quantized item set according to the utility values ​​of each item in the redefined utility list;

[0013] S6. According to the merging method, merging the quantized item sets in the weak utility quantized item set set to generate a range quantized item set, and then merging the range quantized item set with the high utility quantized item set and the candidate quantized item set into one set to obtain a merged set;

[0014] S7. When the minimum length is less than or equal to 1, output the high-utility quantized item sets in the merged set, and when the maximum length is greater than 1, perform a depth-first search method using the redefined utility list and LTQCS structure until all high-utility quantized item sets in the merged set are output.

[0015] In one embodiment of the present invention, the maximum transaction weighted utility of the quantized itemset is:

[0016]

[0017] Among them, LTU is the maximum transaction utility, which is defined as: transaction T r ={x1,x2,…,x k The maximum utility under the maximum length maxL is recorded as LTU(T r ), the expression is: LTU(T r )=∑M(T r ), M(T r ) is the maximum utility set of the transaction, transaction T r The maximum utility set M(T r ) is {u((i1,l1,u1),T r ),u((i2,l2,u2),T r ),…,u((i k ,l k ,u k ),T r )}, the maximum length of the set, x j (1≤j≤k)=(i j ,lj ,u j ) is a quantitative item, T r ={x1,x2,…,x k} is a transaction, i j Indicates the name of the item, l j Represents item i j The lower limit of the number, u j Represents item i j The maximum number of .

[0018] In one embodiment of the present invention, according to the maximum transaction-weighted utility property of the quantized itemsets, all quantized itemsets are pruned using the minimum utility threshold and the quantized correlation coefficient, and the pruned quantized itemsets are stored in a promising quantized itemset set, including:

[0019] According to the maximum transaction weighted utility of the quantified item set having the downward closure property, it is judged whether the quantified item set satisfies Among them, θ is the customized minimum utility threshold, qrc is the quantitative correlation coefficient;

[0020] If so, the quantized item set is retained, and the retained quantized item set is stored as the pruned quantized item set in the promising quantized item set set; if not, the quantized item set is deleted.

[0021] In one embodiment of the present invention, the redefined utility list includes the item set name, the transaction in which the item set is located, the actual utility of the quantized item set in the transaction, the maximum residual utility set of the quantized item set in the transaction, the maximum residual utility of the quantized item set, the actual utility sum, the residual utility sum, and the maximum transaction weighted utility of the quantized item set.

[0022] In one embodiment of the present invention, the maximum residual utility of the quantized item set is the maximum residual utility of the quantized item set in the revised database, which is expressed as:

[0023]

[0024] Among them, lru(X,T r ) is the quantified item set X in transaction T r The maximum residual utility in lru(X,T r )=∑M(T r ,X),M(T r ,X) is defined as: Quantized item set X in transaction T r The maximum residual utility set in is {u((n1,l1,u1),T r ),u((n2,l2,u2),T r ),…,u((n m ,l m ,um ),T r )}; maxE(X) = maxL-|X|, the maximum number of items that can be expanded in X should meet the length constraint, |X| is the number of quantized items contained in X, v j (1≤j≤m)=(n j ,l j ,u j ) represents the quantified items in the set V, V(T r ,X)=v1,v2,…,v m Is the quantified item set X in transaction T r The set of items that can be expanded in n j Represents the name of the item in the set V, l j Represents item n j The lower limit of the number, u j Represents item n j The maximum number of .

[0025] In one embodiment of the present invention, the actual utility of the quantized item set in the transaction is: EU(X,T r )=u(X,T r ), where u represents the item set X in transaction T r The utility of .

[0026] In one embodiment of the present invention, the maximum residual utility set of the quantified item set in the transaction is: eList = M(T r ,X).

[0027] In one embodiment of the present invention, step S5 includes:

[0028] When the actual utility of the quantized item set in the transaction is greater than or equal to the minimum utility threshold, storing the quantized item set in the high-utility quantized item set;

[0029] When the actual utility of the quantized item set in the transaction plus the maximum residual utility is greater than or equal to the minimum utility threshold, storing the quantized item set in the candidate quantized item set;

[0030] When the actual utility of the quantized item set in the transaction plus the maximum residual utility is less than the minimum utility threshold, the quantized item set is stored in the weak utility quantized item set.

[0031] In one embodiment of the present invention, the LTQCS structure is created using the TQCS creation method in the FHUQI algorithm;

[0032] The LTQCS structure consists of a set of tuples of the form (a, b, c), where a and b are two quantified items that appear simultaneously in the revised database, and c is the maximum transaction utility of {ab}.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1. The fast and high-utility quantified item set mining method based on length constraints of the present invention proposes two more compact upper limits. The first more compact upper limit is the maximum transaction weighted utility, and the second more compact upper limit is the maximum residual utility in the redefined utility list. This greatly reduces the search space, improves the running speed of the algorithm, reduces memory consumption, and can filter out item sets with high utility but not meeting the length constraints. In particular, for data sets with too many long item sets, it can quickly mine high-utility quantified item sets with length constraints, thereby improving the efficiency of the algorithm.

[0035] 2. The fast and high-utility quantized item set mining method based on length constraint of the present invention redefines the utility list structure to facilitate the storage of the utility of quantized item sets of maximum length. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of a fast and efficient utility quantized itemset mining method based on length constraints provided by an embodiment of the present invention;

[0037] Figure 2 A flowchart of another method for mining fast and efficient utility quantized itemsets based on length constraints provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of a redefined utility list structure provided by an embodiment of the present invention;

[0039] Figure 4 A comparison chart of the number of modes of different algorithms provided by embodiments of the present invention;

[0040] Figure 5 A comparison chart of the mode running time of different algorithms provided in an embodiment of the present invention;

[0041] Figure 6 A comparison chart of memory usage of different algorithms provided in an embodiment of the present invention;

[0042] Figure 7 This is a schematic diagram of fund data in the Choice financial terminal;

[0043] Figure 8 A schematic diagram of part of the data converted into a quantitative transaction database;

[0044] Figure 9This is a schematic diagram of some data converted into an external income statement;

[0045] Figure 10 A diagram for mapping each stock name into a number;

[0046] Figure 11 This is a schematic diagram of a single stock in the mining results;

[0047] Figure 12 This is a schematic diagram of the ranking of the growth rate of a single stock in the third quarter;

[0048] Figure 13 A schematic diagram of the top 20 stocks in terms of growth rate and economic benefits;

[0049] Figure 14 This is a schematic diagram of the combined stocks in the mining results. DETAILED DESCRIPTION

[0050] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0051] Example 1

[0052] See Figure 1 and Figure 2 , Figure 1 A flowchart of a method for mining fast and efficient utility quantized itemsets based on length constraints provided by an embodiment of the present invention is provided. Figure 2 A flowchart of another method for mining fast and efficient utility quantized itemsets based on length constraints provided by an embodiment of the present invention. The method comprises the following steps:

[0053] S1. Obtain the quantitative transaction database D, the customized minimum utility threshold θ, the minimum length minL, the maximum length maxL, the quantitative correlation coefficient qrc, and the merging method CM.

[0054] Specifically, the quantitative transaction database D is composed of a set of transactions, denoted as D = {T1, T2, ..., T M}, where each transaction T r ∈D(1≤r≤M) has a unique identifier T id . Let I={i1,i2,…,i N} is the set of all items that appear in D, where each transaction T r All contain some items and are subsets of I Transaction T r Each item i that appears in c ∈I(1≤c≤N) has a corresponding integer q(i c ,T r ) means ic In T r The quantity in is called internal utility. r Project i c There is a number (usually positive) that can be expressed as profit p(i c ), called external utility.

[0055] The minimum utility threshold θ, minimum length minL, maximum length maxL, quantization correlation coefficient qrc, and merging method CM are all user-defined.

[0056] S2. Perform a first scan on the quantitative transaction database to calculate the maximum transaction weighted utility of all quantized item sets, and prune all quantized item sets based on the maximum transaction weighted utility property using the minimum utility threshold and the quantitative correlation coefficient, and store the pruned quantized item sets into a promising quantized item set set.

[0057] Specifically, the maximum transaction-weighted utility of the quantified itemset X is the first compact upper bound proposed, which is defined as Where LTU is the maximum transaction utility, which is defined as: when transaction T r ={x1,x2,…,x k The maximum utility under the maximum length maxL is recorded as LTU(T r ), the expression is: LTU(T r )=∑M(T r ); where M(T r ) is the maximum utility set of a transaction, which is defined as: let the user-defined maximum length be maxL, transaction T r ={x1,x2,…,x k}, quantify item x j (1≤j≤k)=(i j ,l j ,u j ), i j Indicates the name of the item, l j Represents item i j The lower limit of the number, u j Represents item i j The upper limit of the number of transactions T r The maximum utility set M(T r ) is {u((i1,l1,u1),T r ),u((i2,l2,u2),T r ),…,u((i k ,l k ,u k ),T r )} with maximum length maxL.

[0058] According to the definition of LTWU, LTWU has a downward closure property if the quantized item set X satisfies Then all supersets of the quantized item set X are not high-utility quantized item sets with length constraints. This pruning strategy is called LTWU pruning.

[0059] Specifically, determine whether the quantified item set satisfies Where θ is the user-defined minimum utility threshold, and qrc is the user-defined quantization correlation coefficient. If so, the quantized itemset is retained and stored as the pruned quantized itemset in the promising quantized itemset set. If not, the quantized itemset is deleted, thus completing the quantized itemset pruning.

[0060] In this embodiment, since a length restriction is added to the LTU, the LTWU is a more compact upper bound, which can further reduce the search space and improve the algorithm performance.

[0061] S3. Arrange the pruned quantized item sets in ascending order according to the values ​​of the maximum transaction weighted utilities to obtain promising quantized item sets, and revise the quantitative transaction data to obtain a revised database.

[0062] Specifically, the maximum transaction weighted utility (LTWU) of the quantified item sets is a specific value. The quantified item sets are sorted in ascending order according to the LTWU value to obtain promising quantified item sets. The pruned quantified item sets are used to update and revise the quantitative transaction database to obtain a revised database.

[0063] S4. Scan the revised database a second time to create a redefined utility list and LTQCS structure.

[0064] Specifically, the redefined utility list structure contains a second more compact upper bound for the maximum remaining utility LRU. Let transaction T r and quantized item set X, let V(T r ,X)=v1,v2,…,v m It's X in T r The itemset that can be expanded in v j (1≤j≤m)=(n j ,l j ,u j ), n j Represents the name of the item in the set V, l j Represents item n j The lower limit of the number, u j Represents item n jThe maximum number of items that can be expanded in X should satisfy the length constraint, which is defined as maxE(X) = maxL-|X|, where |X| is the number of quantized items contained in X. Then X is in T r The maximum residual utility set in is {u((n1,l1,u1),T r ),u((n2,l2,u2),T r ),…,u((n m ,l m ,u m ),T r )}, denoted as M(T r ,X). Then X is in transaction T r The maximum residual utility in is denoted as lru(X,T r ), defined as: lru(X,T r )=∑M(T r ,X). Then the maximum residual utility of the quantified item set X in the revised database D is recorded as LRU(X), which is defined as:

[0065] like Figure 3 As shown, Figure 3 A schematic diagram of a redefined utility list structure provided by an embodiment of the present invention. The redefined utility list includes the item set name, the transaction Tid where the item set is located, and the quantified item set in the transaction T r The utility EU(X,T r )=u(X,T r ),u represents the item set X in transaction T r The utility of the quantified item set in the transaction is the maximum residual utility set eList = M(T r ,X), quantified item set maximum residual utility LRU(X,T r )=lru(X,T r ), actual utility and SumU, residual utility and SumEU, and the maximum transaction-weighted utility LTWU(X) of the quantized itemset. By storing the utility information of the quantized itemset and the heuristic information used to construct the search space, the algorithm avoids scanning the database multiple times and improves its efficiency.

[0066] The LTWU of Q-items Co-occurrence-based Structure (LTQCS) structure is created in the same way as the TWU of Q-items Co-occurrence-based Structure (TQCS) in the Fast High Utility Quantitative Itemset Mining (FHUQI) algorithm. LTQCS consists of a set of tuples of the form (a, b, c), where a and b are two quantitative items that appear simultaneously in the revised database, and c is the maximum transaction utility of {ab}.

[0067] S5. Divide the promising quantization item set into a high-utility quantization item set, a candidate quantization item set, and a weak-utility quantization item set according to the utility values ​​of each item in the redefined utility list.

[0068] Specifically, when the actual utility EU of the quantized item set in the transaction is greater than or equal to the minimum utility threshold θ, the quantized item set is stored in the high-utility quantized item set;

[0069] When the actual utility EU of the quantized item set in the transaction plus the maximum residual utility LRU is greater than or equal to the minimum utility threshold θ, the quantized item set is stored in the candidate quantized item set;

[0070] When the actual utility EU of the quantized item set in the transaction plus the maximum residual utility LRU is less than the minimum utility threshold θ, the quantized item set is stored in the weak utility quantized item set;

[0071] If none of the above conditions are met, the corresponding quantized item set is deleted.

[0072] S6. According to the merging method CM, the quantized item sets in the weak utility quantized item set are merged to generate a range quantized item set, and then the range quantized item set is merged with the high utility quantized item set and the candidate quantized item set into one set to obtain a merged set.

[0073] S7. When the minimum length is less than or equal to 1, output the high-utility quantized item sets in the merged set, and when the maximum length is greater than 1, perform a depth-first search method using the redefined utility list and LTQCS structure until all high-utility quantized item sets in the merged set are output.

[0074] Specifically, the algorithm determines whether the minimum length satisfies minL≤1. If so, it outputs the high-utility quantized itemsets in the merged set. Otherwise, the algorithm ends the minimum length determination and proceeds to the next step of determining the maximum length. The algorithm determines whether the maximum length satisfies maxL>1. If so, it performs a depth-first search until all high-utility quantized itemsets with length constraints are output. If not, the algorithm ends.

[0075] This embodiment performs a depth-first search method, and can mine high-utility quantized item sets whose item set lengths are within a user-defined range through recursive calls.

[0076] The fast and high-utility quantified item set mining method based on length constraints of the present invention proposes two more compact upper limits. The first more compact upper limit is the maximum transaction weighted utility, and the second more compact upper limit is the maximum residual utility in the redefined utility list. It greatly reduces the search space, improves the running speed of the algorithm, reduces memory consumption, and can filter out item sets with high utility but not meeting the length constraints. In particular, for data sets with too many long item sets, it can quickly mine high-utility quantified item sets with length constraints, thereby improving the efficiency of the algorithm.

[0077] The fast and high-utility quantized item set mining method based on length constraint of the present invention redefines the utility list structure so as to store the utility of the quantized item set with the maximum length.

[0078] Furthermore, this embodiment further verifies the fast and high utility quantized itemset mining method based on length constraints (hereinafter referred to as FHUQI-LC) through simulation.

[0079] This example was tested on a computer equipped with a 64-bit AMD Ryzen 7 4800U processor and 16GB of RAM, running Windows 10. The database used is the Foodmart dataset from the SPMF website. This dataset is a sparse dataset containing transaction records of 4,141 customers in a retail store, with 1,559 unique items and an average length of 4.42 per transaction. In the experiment, the FHUQI-LC minimum length threshold was set to 1, and five different maximum length thresholds (maxL = 1, 2, 3, 4, and 5) were used to compare the number of patterns, execution time, and memory usage with a state-of-the-art, fast, high-utility quantitative itemset mining algorithm.

[0080] See Figure 4 、 Figure 5 and Figure 6 , Figure 4 A comparison chart of the number of modes of different algorithms provided by the embodiments of the present invention, Figure 5A comparison chart of the mode running time of different algorithms provided in an embodiment of the present invention is shown. Figure 6 A comparison chart of memory usage of different algorithms provided by embodiments of the present invention.

[0081] like Figure 4 As shown in the figure, when the minimum utility threshold is less than 0.1%, the number of patterns of the FHUQI-LC algorithm with maxL=1 is reduced by 55%-98% compared to the FHUQI algorithm, and the number of patterns of the FHUQI-LC algorithm with maxL=5 is reduced by 21%-88% compared to the FHUQI algorithm. This greatly reduces the storage of long itemsets, shortens the search space, and performs better when the maximum length and minimum threshold are small. Figure 5 As shown in Figure 1, when maxL=1, the running speed of the FHUQI-LC algorithm is 3-15 times that of the FHUQI algorithm. When maxL is larger, the running speed is slower. When maxL=5, the running speed of the FHUQI-LC algorithm is 1.15-2.89 times that of the FHUQI algorithm. Figure 6 As shown in the figure, the memory usage of FHUQI-LC is 1%-61% less than that of FHUQI, indicating that the use of length constraints can reduce the search space and reduce the number of connection operations, further saving memory usage.

[0082] In summary, this embodiment adopts a fast and high-utility quantitative item set mining algorithm based on length constraints. Compared with other algorithms, it can filter out item sets with high utility but not meeting the length constraints, greatly reducing the number of patterns. In addition, the search space is further reduced by proposing two more compact upper limits, thereby improving the algorithm's running speed and reducing memory consumption.

[0083] Example 2

[0084] Based on the first embodiment, this embodiment provides an application scenario to illustrate the application of the fast and efficient utility quantized itemset mining method based on length constraints.

[0085] 1. Case Analysis

[0086] Funds play a very important role in the domestic financial market. With the rapid development of China's economy and the continuous opening of its financial markets, funds have become the choice of more and more investors. As of the end of February 2023, the scale of management of China's fund industry has reached 27.25 trillion yuan, of which stock funds are the largest type of funds. The development of funds not only provides investors with convenient investment channels, but also promotes the development of China's stock market and provides new channels for corporate financing. With the continuous development of funds, fund companies have become important investment institutions in my country. In order to meet the different needs of fund investors, fund managers of various fund companies will adopt differentiated investment strategies and form different investment styles. Therefore, this embodiment mainly explains which stocks fund investors should choose to invest in, or how fund investors can determine which fund company's fund manager has the strongest investment capabilities.

[0087] Current high-utility quantitative item set mining algorithms are mainly used in application scenarios such as supermarket marketing. To demonstrate the application value of high-utility quantitative item set technology in contemporary society, this embodiment proposes the FHUQI-LC algorithm and applies it to the field of financial analysis to mine and analyze fund holdings.

[0088] 2. Data Acquisition and Processing

[0089] The data of this embodiment comes from the Choice financial terminal, and the Choice financial terminal is used to select all domestic ordinary stock funds and equity-oriented mixed funds. Due to the nature of the fund itself and the differences in the fund manager's personal operating style and holding philosophy, the fund's holdings and holding ratios will change dynamically, so this embodiment writes a conditional expression in Choice to extract the top 10 holdings information in each fund's 2022 third quarter report and the price increase and decrease data of each fund and its constituent stocks in the past quarter. This data is mined using the FHUQI-LC algorithm proposed in Example 1. By setting different thresholds and lengths, individual stocks and stock portfolios that bring high economic benefits are mined. The mining results are analyzed and evaluated, and investment recommendations are given to evaluate the profitability of the fund and the fund manager's stock selection ability.

[0090] Figure 7This is a diagram of fund data from the Choice financial terminal, which primarily includes 2,047 funds. Each column in the table header displays the fund's stock code, security name, fund inception date, top ten holdings, percentage of holdings of the top ten holdings, and price fluctuations. The main reason for selecting each fund's top ten holdings is the complex composition of the funds (equity-oriented hybrid funds typically consist of stocks, bonds, insurance, etc.). These top ten holdings account for the largest portion of the fund's total market value and, in practice, have the greatest impact on the fund's overall performance. Therefore, selecting the top ten holdings provides ample reference and analytical value.

[0091] Export the data from the Choice financial terminal, clean the exported data, remove the funds containing non-stock components in the top ten heavily-weighted components, and finally select 1,225 funds and 1,295 stocks for mining. Since the dataset used for high-utility quantitative item set mining is based on the SPMF platform, the data needs to meet the format of the SPMF platform. SPMF mainly contains two datasets, namely the quantitative transaction database and the external utility table. The format in the quantitative transaction database: each line is a transaction, the data before "," refers to the items in the transaction, and the data after "," refers to the number of items in the transaction, and the two items are separated by a space. The format of the external utility table: the items and profit values ​​are separated by ",". For example Figure 8 As shown, Figure 8 This is a schematic diagram of some data converted into a quantitative transaction database. Figure 9 The following is a diagram of some data converted into an external profit statement. In the fund data analyzed in this case study, each fund represents a transaction, and each stock represents a project. The proportion of each stock in the fund is rounded off, and the rounded integer is used as the number of stocks in the fund. The increase or decrease of each stock is rounded off to the nearest integer and used as the external profit of the item. The external profit value can be positive or negative. To facilitate the analysis of the results, Figure 10 As shown, Figure 10 A diagram showing the mapping of each stock name to a number.

[0092] 3. Results Analysis

[0093] The algorithm FHUQI-LC is used to mine the quantitative transaction database mentioned above, where the minimum utility threshold is set to 1500, the minimum length threshold is set to 1, and the maximum length threshold is set to 1. The economic benefit ranking data of each single stock in the 1295 stocks relative to the entire fund is obtained. Figure 11 As shown, Figure 11 This is a schematic diagram of a single stock in the mining results. Figure 11It can be found that the single-stock configuration with the highest economic benefit is (478, 4), that is, Xinlai Materials holds 4% of the shares. Among all 1,225 funds, the total economic benefit brought by (478, 4) is 63.92%, followed by (265, 6), that is, Junda shares holds 6% of the shares, and the total economic benefit brought is 46.56%.

[0094] In order to distinguish the difference and impact of the rise and fall of a single stock and the economic benefits it brings, the following is a list of the rise and fall of the 1,295 constituent stocks from July 1 to September 30, arranged from high to low. Figure 12 As shown, Figure 12 This is a schematic diagram of the ranking of the growth rate of a single stock in the third quarter, starting from Figure 12 It can be seen that the stock 820 (Southern Power Grid Technology) has increased or decreased by 101% during the specified time period.

[0095] The economic benefit data and the top 20 individual stock price increase data obtained from mining are selected for comparative analysis. Figure 13 As shown, Figure 13 This is a schematic diagram of the top 20 stocks in terms of individual stock growth and economic benefits.

[0096] from Figure 13 It's clear that the common perception that "the greater the increase, the higher the return" is misguided. This overlooks the importance of holding percentage and frequency. China Southern Power Grid Technology clearly had the highest increase within the period, at 101%. However, its economic benefits were not the highest. For the 1,225 funds, it contributed only 24.24% to the overall economic growth. Conversely, Yanzhou Coal Mining Energy, with a mere 35% increase, delivered 28% economic benefits, surpassing China Southern Power Grid Technology. This is consistent with economic trends: high growth implies high volatility and risk. High risk often leads fund managers and individual investors to avoid or even eliminate positions in stocks like China Southern Power Grid Technology. The data in the chart also shows that all of the top 20 stocks with the highest gains exceeded 55%, while the vast majority of the top 20 stocks with the highest economic benefits saw increases between 10% and 50%. The candlestick chart also indicates a generally steady upward trend. Stable returns indicate a more stable investment, which is why more funds are allocating to these stocks. This can help individual investors better understand the financial market and provide a basis for individual stock allocations. You can also determine whether the fund manager is aggressive or conservative in his investment by analyzing the economic benefits and growth data of the constituent stocks in the fund portfolio, thereby helping individuals decide whether the fund they are about to choose is suitable for them.

[0097] In addition, according to Figure 13The performance of individual stocks in the fourth quarter of 2022 also shows that 90% of the top 20 stocks with the highest economic returns suffered losses in the fourth quarter. In contrast, only 65% ​​of the top 20 stocks with the highest gains suffered losses in the fourth quarter, while 35% still achieved profitability. This is because investing in the financial market follows the rules of a "zero-sum game." Under strict competition, a gain for one party necessarily means a loss for the other. The sum of the gains and losses of all parties involved in the game always equals zero, making cooperation impossible. Therefore, investing in these stocks that have already generated significant economic returns in the fourth quarter would be tantamount to taking over the market. Therefore, analyzing this data suggests that it is recommended to avoid "chasing high" stocks that have already generated high economic returns. For stocks that have seen significant gains but have generated low economic returns, consider buying small positions.

[0098] From the above experiment, it can be seen that theoretically the optimal investment portfolio should be the top 10 stocks in terms of economic efficiency as heavy holdings, but in reality, this combination is obviously unpredictable by fund managers. After traversal and comparison, it is also proved that there is no fund with the above combination among the 1,225 funds. In order to evaluate the fund manager's stock selection and allocation ability, the following experiment is conducted. The threshold is set to 2800, the minimum threshold length is 2, and the maximum threshold length is 10. All fund data are mined to obtain 81 combinations with multiple stocks under the premise of meeting the economic efficiency of 28%. The 81 combinations are arranged in descending order of economic efficiency to obtain some combinations as follows Figure 14 As shown, Figure 14 This is a schematic diagram of the combined stocks in the mining results.

[0099] from Figure 14 As can be seen, under the premise of meeting the minimum threshold, the combination with the highest utility is {(284,8), (301,5), (299,7), (797,9), (48,6), (478,4)}, corresponding to 8% of Homai Holdings, 5% of Tianyue Advanced, 7% of APEC Technology, 9% of Tuojing Technology, 6% of North China Innovation, and 4% of New Lai Materials, with a utility of 38.10%. Comparing with the original fund data, it is found that only two funds have the above-mentioned holdings: the Guolian An Preferred Industry Hybrid Fund and the Guolian An Technology Power Fund, both managed by the same fund manager, Pan Ming. Based on the above holdings, it can be seen that Pan Ming prefers to invest in high-tech companies such as semiconductors and photovoltaics. Therefore, it can be concluded that, under the background of encouraging technological development, fund manager Pan Ming, who excels in investing in technology companies, has outstanding performance and high fund returns. For investors who want to invest in technology-related stock funds or equity-oriented hybrid funds, Pan Ming is a good choice.

[0100] In summary, the FHUQI-LC algorithm can mine the economic benefit data of individual stocks in a fund portfolio. Combined with the price fluctuations of individual stocks, it can provide individual investors with a basis for holding and allocating stocks, and provide data support for individual investors to choose aggressive or conservative funds according to their personal preferences. It can also mine fund portfolio data that meets the minimum utility threshold. By matching it with the original fund data, it can help investors find funds and fund managers with high economic benefits and evaluate and analyze fund managers.

[0101] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A fast and high-utility quantitative itemset mining method based on length constraints, characterized by: Including steps: S1. Obtain the quantitative transaction database, customized minimum utility threshold, minimum length, maximum length, quantitative correlation coefficient, and merging method; S2. Performing a first scan on the quantitative transaction database to calculate the maximum transaction weighted utility of all quantized item sets, and pruning all quantized item sets based on the maximum transaction weighted utility property using the minimum utility threshold and the quantitative correlation coefficient, and storing the pruned quantized item sets into a promising quantized item set set; S3. Arranging the pruned quantized item sets in ascending order according to the values ​​of the maximum transaction weighted utilities, and revising the quantitative transaction database according to the arrangement results to obtain a revised database; S4. Scan the revised database a second time to create a redefined utility list and LTQCS structure; S5. Dividing the promising quantized item set into a high-utility quantized item set, a candidate quantized item set, and a weak-utility quantized item set according to the utility values ​​of each item in the redefined utility list; S6. According to the merging method, merging the quantized item sets in the weak-utility quantized item set set to generate a high-utility range quantized item set, and then merging the high-utility range quantized item set, the high-utility quantized item set, and the candidate quantized item set into one set to obtain a merged set; S7. When the minimum length is less than or equal to 1, output the high-utility quantized item sets in the merged set, and when the maximum length is greater than 1, perform a depth-first search method using the redefined utility list and LTQCS structure until all high-utility quantized item sets in the merged set are output.

2. The fast and high-utility quantified itemset mining method based on length constraints according to claim 1 is characterized in that: The maximum transaction-weighted utility of the quantized itemset is: Among them, LTU is the maximum transaction utility, which is defined as: transaction T r ={x1,x2,…,x k The maximum utility under the maximum length maxL is recorded as LTU(T r ), the expression is: LTU(T r )=∑M(T r ), M(T r ) is the maximum utility set of the transaction, transaction T r The maximum utility set M(T r ) is {u((i1,l1,u1),T r ),u((i2,l2,u2),T r ),…,u((i k ,l k ,u k ),T r )}, the maximum length of the set, x j (1≤j≤k)=(i j ,l j ,u j ) is a quantitative item, T r ={x1,x2,…,x k } is a transaction, i j Indicates the name of the item, l j Represents item i j The lower limit of the number, u j Represents item i j The maximum number of .

3. The fast and high-utility quantified itemset mining method based on length constraints according to claim 1, characterized in that: According to the maximum transaction weighted utility property of the quantized itemset, all quantized itemsets are pruned using the minimum utility threshold and the quantized correlation coefficient, and the pruned quantized itemsets are stored in a promising quantized itemset set, including: According to the maximum transaction weighted utility of the quantified item set having the downward closure property, it is judged whether the quantified item set satisfies Among them, θ is the customized minimum utility threshold, qrc is the quantitative correlation coefficient; If so, the quantized item set is retained, and the retained quantized item set is stored as the pruned quantized item set in the promising quantized item set set; if not, the quantized item set is deleted.

4. The fast and high-utility quantified itemset mining method based on length constraints according to claim 1, characterized in that: The redefined utility list includes the item set name, the transaction in which the item set is located, the actual utility of the quantized item set in the transaction, the maximum residual utility set of the quantized item set in the transaction, the maximum residual utility of the quantized item set, the actual utility sum, the residual utility sum and the maximum transaction weighted utility of the quantized item set.

5. The fast and high-utility quantified itemset mining method based on length constraints according to claim 4 is characterized in that: The maximum residual utility of the quantized item set is the maximum residual utility of the quantized item set in the revised database, which is recorded as: Among them, lru(X,T r ) is the quantified item set X in transaction T r The maximum residual utility in lru(X,T r )=∑M(T r ,X),M(T r ,X) is defined as: Quantized item set X in transaction T r The maximum residual utility set in is {u((n1,l1,u1),T r ),u((n2,l2,u2),T r ),…,u((n m ,l m ,u m ),T r )}; maxE(X) = maxL-|X|, the maximum number of items that can be expanded in X should meet the length constraint, |X| is the number of quantized items contained in X, v j (1≤j≤m)=(n j ,l j ,u j ) represents the quantified items in the set V, V(T r ,X)=v1,v2,…,v m Is the quantified item set X in transaction T r The set of items that can be expanded in n j Represents the name of the item in the set V, l j Represents item n j The lower limit of the number, u j Represents item n j The maximum number of .

6. The fast and high-utility quantified itemset mining method based on length constraints according to claim 4, characterized in that: The actual utility of the quantized item set in the transaction is: EU(X,T r )=u(X,T r ), where u represents the item set X in transaction T r The utility of .

7. The fast and high-utility quantified itemset mining method based on length constraints according to claim 4, characterized in that: The maximum residual utility set of the quantified item set in the transaction is: eList = M(T r ,X).

8. The fast and high-utility quantified itemset mining method based on length constraints according to claim 4, characterized in that: Step S5 includes: When the actual utility of the quantized item set in the transaction is greater than or equal to the minimum utility threshold, storing the quantized item set in the high-utility quantized item set; When the actual utility of the quantized item set in the transaction plus the maximum residual utility is greater than or equal to the minimum utility threshold, storing the quantized item set in the candidate quantized item set; When the actual utility of the quantized item set in the transaction plus the maximum residual utility is less than the minimum utility threshold, the quantized item set is stored in the weak utility quantized item set.

9. The fast and high-utility quantified itemset mining method based on length constraints according to claim 1, characterized in that: The LTQCS structure is created using the TQCS creation method in the FHUQI algorithm; The LTQCS structure consists of a set of tuples of the form (a, b, c), where a and b are two quantified items that appear simultaneously in the revised database, and c is the maximum transaction utility of {ab}.

Images