A fuzzy high-utility pattern mining method based on medical scenarios

Abstract

This paper proposes a fuzzy high-utility pattern mining method based on medical scenarios. By introducing fuzzy set theory and fuzzy function models, the high-utility pattern mining algorithm is applicable not only to transaction database scenarios, but also to medical data scenarios, and can enhance the interpretability of mining results. This method combines the characteristics of a one-stage high-utility pattern mining algorithm with a two-stage high-utility pattern mining algorithm. When faced with data sets with different characteristics, the time and space consumption of the fuzzification process is relatively stable, and the mining process has better performance than traditional single algorithms.

Description

Technical Field

[0001] The present invention belongs to the field of data mining, specifically a fuzzy high-utility pattern mining method based on medical scenarios. Background Art

[0002] With the advent of the first computer, data emerged. The emergence of ultra-large-scale databases, such as commercial data warehouses, and the widespread use of computer-automated data collection and recording have led to an exponential growth in the amount of human data. The increasing application of statistical methods in data processing has provided methods for mining databases. Advanced computer technologies, such as faster and greater computing power and parallel architectures, have enabled the discovery of knowledge within ultra-large databases. Against this backdrop, data mining technology emerged. Data mining has a wide range of applications, such as association rule mining, sequential pattern mining, text classification, web log analysis, and collaborative filtering. Over the years, high-utility pattern mining technology has also rapidly developed. However, high-utility pattern mining is primarily applied to databases containing profit information, considering only the numerical information of these items and resulting in poor interpretability.

[0003] Data mining is the process of carefully analyzing data to discover meaningful relationships, trends, and patterns. The Apriori algorithm, proposed by Agrawal et al. in 1994, and the FP-growth algorithm, proposed by Han et al. in 2000, are representative methods for frequent itemset mining (FIM), often used to discover relationships between items. Subsequently, numerous FIM mining algorithms have been developed and applied in various scenarios. However, FIM only considers whether an itemset occurs, ignoring the value of the itemset itself. Therefore, the Two-Phase algorithm, proposed by Liu et al. in 2005, is a classic method for high-utility itemset mining (HUIM). HUIM fully considers both the frequency of an itemset's occurrence and the value inherent in the itemset through external and internal utility tables, making it more practically oriented than FIM. However, most current high-utility itemset mining algorithms are based on one-stage or two-stage mining models. These two types of mining algorithms have significantly different time and space complexities when processing the same dataset. Therefore, technical personnel are often required to have rich experience and be able to choose appropriate mining algorithms based on the characteristics of the data set, which invisibly increases the learning cost.

[0004] Fuzzy methods are a common approach in data mining. In his 1965 paper "Fuzzy Sets," Zadeh, an American cybernetics expert and mathematician, proposed membership functions as characteristic functions of fuzzy sets, which are the core of fuzzy sets. As the application of fuzzy sets continues to expand, extended fuzzy sets, including interval-valued fuzzy sets, intuitionistic fuzzy sets, vague sets, and type II fuzzy sets, have gradually been used in various fields. Many concepts in life are difficult to define precisely and are often vague.

[0005] In medical scenarios, the use of fuzzy set theory in rule-based classifiers can effectively blur thresholds and boundaries, thereby resolving the problem of sharp points. While the various indicator data we obtain in traditional medical scenarios is often floating-point data, the classic FIM and HUIM algorithms excel at processing Boolean, binary, and other categorical data. They can mine the relationships between different data features, offering advantages in disease prediction and diagnosis, and possess unique significance for the future development of diagnostics. In traditional medical scenarios, the various indicator data we obtain is often floating-point data. Therefore, after processing the dirty data, we fuzzify the various data features based on reference values ​​for various human indicators, converting them into Boolean, binary, or categorical data. High-utility pattern mining algorithms are then used for data mining.

[0006] Direct and fuzzy interval methods are commonly used to convert floating-point data from physical examinations into Boolean data. The direct interval method is relatively simple and straightforward, but it can lead to sharp edges in certain scenarios. For example, consider the fasting blood glucose level, a common measurement in diabetes exams. Suppose someone goes to the hospital for a physical examination, which includes a fasting blood glucose level. The reference standard for fasting blood glucose levels is 3.2-6.5 mmol / L. If the fasting blood glucose level exceeds 7.0 mmol / L, the individual is considered to have diabetes and further testing is recommended. If the fasting blood glucose level is 8.0 mmol / L, the probability of diabetes is high and a thorough examination is warranted. However, if the fasting blood glucose level is 6.4 mmol / L, how should we classify the patient? Should we consider the level below 6.5 mmol / L and therefore not categorize the patient as diabetic, or should we consider the level high and warrant further testing? In this case, the 6.5 mmol / L threshold seems somewhat useless. Summary of the Invention

[0007] In response to the above technical problems, the present invention proposes a fuzzy high-utility pattern mining method based on medical scenarios. This method introduces fuzzy set theory, uses fuzzy functions and combines it with a high-utility pattern mining algorithm, thereby expanding the application field of the high-utility pattern mining algorithm and enhancing the interpretability of the mining results. And the characteristics of one-stage and two-stage high-utility pattern mining algorithms. In a large number of experiments, when facing data sets with different characteristics, the fuzzy process of this method is relatively stable in terms of time and space consumption, and the average complexity of time and space in the mining process has better performance than traditional one-stage and two-stage mining algorithms. The use of this method ensures that it still has good performance when facing data sets under different circumstances.

[0008] In order to achieve the above technical objectives, the present invention adopts the following specific technical solutions:

[0009] A fuzzy high-utility pattern mining method based on medical scenarios includes the following steps:

[0010] S1. Collect various physical examination data of patients in medical scenarios and preprocess them. The preprocessing includes referring to the intervals and standard values ​​of various human indicators in medical standards, filling the values ​​outside the interval range with boundary values ​​to obtain the original database data;

[0011] S2. After the original database data obtained in step S1 is passed through the fuzzy function, the membership value of the interval is obtained, which is used as the internal utility value, and the external utility table is manually specified, thereby obtaining a fuzzy database that can be mined by the high-utility pattern mining algorithm;

[0012] S3. The fuzzy database obtained in step S2 is mined using a two-stage mining algorithm. The switching module is periodically called to determine whether the switching condition is met. If so, the one-stage mining algorithm is used to mine fuzzy high-utility patterns.

[0013] In step S1 , the data is pre-processed by filling in the medical standard values ​​of human attributes, and then normalized so that all the data fall within the interval determined by the fuzzy function.

[0014] In step S2, a type I fuzzy function or a type II fuzzy function is selected, and after the data passes through the fuzzy module, the membership values ​​of the low, medium, and high intervals are calculated respectively; after the first scan, the main interval is obtained, and then the membership value of the main interval is obtained. This value is used as the internal utility value in the high-utility pattern mining algorithm, and then the external utility values ​​of various attributes in the physical examination data are set to the same value.

[0015] Step S3 specifically includes the following sub-steps:

[0016] S3.1. Construct a header table based on the fuzzy database content and calculate relevant parameters. Based on the header table, find the path from each leaf node to the root node to obtain the conditional pattern base. Once the traversal is complete, the fuzzy high-utility pattern of all item sets is obtained.

[0017] S3.2. Design a switching module that combines the characteristics of the one-stage mining algorithm and the two-stage mining algorithm in the high-utility pattern mining algorithm. The switching module makes the algorithm adaptive to various scenarios and then screens out candidate high-utility item sets. When calculating the true utility value, a list-based algorithm is used to screen out the high-utility item sets.

[0018] S3.3. Use utility-list to mine high-utility patterns. In the first scan of the database, the algorithm finds a set of items whose TWU is higher than the minimum utility threshold and arranges them in ascending order. When the database is scanned for the second time, the database is reorganized according to the order of the item sets, and a utility list for each item set is generated. After generating the utility list for each item set, the utility list of the {k-1}-length item set is continuously combined to generate a utility list of the {k}-length item set.

[0019] In step S3.2, the switching module is designed to make the algorithm adaptive to various scenarios. The algorithm includes:

[0020] Only a two-stage mining algorithm is used to calculate the overestimated utility value and then perform preliminary screening, and then calculate the true utility value to mine and obtain high utility patterns;

[0021] Or only one-stage mining algorithm is used to directly calculate the true utility value and mine the fuzzy high-utility pattern;

[0022] Or use a tree structure to construct a utility tree and calculate the overestimated utility value.

[0023] The beneficial effects of the present invention are:

[0024] The present invention proposes a fuzzy high-utility pattern mining method based on medical scenarios. By introducing fuzzy set theory and fuzzy function models, the high-utility pattern mining algorithm can be applied not only to transaction database scenarios, but also to medical data scenarios, and can enhance the interpretability of mining results. This method combines the characteristics of a one-stage high-utility mining algorithm and a two-stage high-utility pattern mining algorithm. When faced with data sets of different characteristics, the time and space consumption in the fuzzification process is relatively stable, and the mining process has better performance than the traditional single algorithm. As a result, when faced with different data, the method has better performance as a whole, which is described in detail below.

[0025] By referring to the guidance provided by medical experts and processing the data using fuzzy clustering algorithms, we can obtain fuzzy membership function models for different attributes. We then innovatively use these models to process medical databases to give them utility values, which can then be mined using high-utility pattern mining algorithms.

[0026] By adopting the concept of a hybrid framework, using both a one-stage mining algorithm and a two-stage mining algorithm to mine the database, the fuzzy high-utility mining algorithm can adapt to datasets with different characteristics in various medical scenarios and has good performance in terms of time and space complexity.

[0027] Introducing fuzzy set theory into the mining process makes the mining results no longer cold numbers, but converted into more easily understandable semantic results, thereby increasing the interpretability of the results and solving the problem of poor understandability and interpretability in high-utility pattern mining algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the method design proposed by the present invention;

[0029] Figure 2 is the fuzzy membership function;

[0030] Figure 3 This is the database form before fuzzification;

[0031] Figure 4 It is the database form after obfuscation processing;

[0032] Figure 5 This is the UFH algorithm framework diagram;

[0033] Figure 6-Figure 12 It is the tree-forming process of the UFH algorithm;

[0034] Figure 13 It is the EUCS structure diagram of the database. Specific implementation plan

[0035] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0036] For the original data set in the medical scenario, the boundary value is processed, and the empty values ​​are filled with the standard values ​​of the reference values ​​of various human indicators to obtain the original database. After the database data passes through the fuzzy function, the membership value of the interval is obtained, which is used as the internal utility value, and the external utility table is manually specified. In this way, a fuzzy database that can be mined by a high-utility pattern mining algorithm is obtained. A two-stage mining algorithm is used for mining, and the switching module is periodically called to determine whether the switching conditions are met. If so, the one-stage mining algorithm is used for processing. The design flow chart of the method proposed in the present invention is as follows: Figure 1 shown.

[0037] 1. Architecture

[0038] The improved UFH algorithm proposed in this invention is similar to the traditional UFH algorithm, consisting of a one-stage high-utility pattern mining algorithm and a two-stage high-utility pattern mining algorithm. However, the present invention differs in that the related structures of the one-stage high-utility pattern mining algorithm and the two-stage pattern mining algorithm in the traditional UFH algorithm are redesigned and modified to enable them to store and process fuzzified data. Furthermore, functional code modules are added to the algorithm that utilize fuzzy set theory to fuzzify the original database and process the mining results to generate highly interpretable semantic results. Ultimately, this improved algorithm can mine fuzzy high-utility patterns.

[0039] 2. Methodology

[0040] 1. Data Preprocessing

[0041] In the traditional medical data scenario, after obtaining the original physical examination data of the patient, the values ​​outside the interval range are filled with boundary values, referring to the intervals and standard values ​​of various human indicators in the medical standards. Since the values ​​outside the interval range and the boundary values ​​have a membership of 1 to the interval when the fuzzy membership function is used for processing, it does not affect the final result. For blank values, the standard values ​​of various human indicators are used to fill them. If there are many missing values ​​in a certain data, they are directly deleted. Since different attribute values ​​belong to different ranges, they need to be normalized. In the method of the present invention, the following is adopted: Figure 2 The fuzzy membership function shown in FIG, so for different attribute values ​​of the human body, after normalization processing, all data values ​​are made to fall between 0 and 11, thereby obtaining the preprocessed original database.

[0042] Fuzzy processing

[0043] In traditional medical settings, the various indicator data we obtain is often floating-point data. Classic FIM and HUIM algorithms excel at processing Boolean, binary, and other categorical data. They can mine the relationships between different data features, offering advantages in disease prediction and diagnosis, and possessing unique significance for the future development of diagnostics. In traditional medical settings, the various indicator data we obtain is often floating-point data. Therefore, after processing the dirty data, we fuzzify the various data features based on reference values ​​for various human indicators, converting them into Boolean, binary, or other categorical data. We then utilize high-utility pattern mining algorithms for data mining.

[0044] Direct and fuzzy partitioning are commonly used to convert floating-point data from physical examinations into Boolean data. The direct partitioning method is relatively simple and straightforward, but it can lead to edge sharpening in certain scenarios. Take, for example, the fasting blood glucose level, a common measurement in diabetes exams. Suppose someone goes to the hospital for a physical examination, which includes a fasting blood glucose level. The reference standard for fasting blood glucose levels is 3.2-6.5 mmol / L. If the fasting blood glucose level exceeds 7.0 mmol / L, the individual is considered to have diabetes and further testing is performed. If the fasting blood glucose level is 8.0 mmol / L, there is a high probability of diabetes and a thorough examination is warranted. However, if the fasting blood glucose level is 6.4 mmol / L, how should we classify it? Should we consider it as not exceeding 6.5 mmol / L and therefore not categorize it as diabetic, or should we consider it as high and require further testing? The 6.5 mmol / L threshold seems somewhat useless. Using the fuzzy partitioning method can effectively avoid the problem of edge sharpening. It expands the data space into a fuzzy space, dividing it into three intervals: low, medium, and high. Floating-point numbers in the interval [0,1] are used as the membership of the divided data in each interval. After the first scan, the main interval is obtained, and then the membership value of the main interval is obtained. This value is used as the internal utility value in the high-utility pattern mining algorithm. The external utility values ​​of various attributes in the physical examination data are then set to the same value. The external utility value of the disease can be increased to exceed the external utility values ​​of other attributes, thereby increasing the probability of the disease appearing in the high-utility pattern set.

[0045] For example, a fasting blood glucose value of 6.4mmol / L can be considered as hyperglycemia with a membership of 0.99, normal blood glucose with a membership of 0.05, and hypoglycemia with a membership of 0. Figure 3 As shown, the processed fuzzy database is in the form of Figure 4 shown.

[0046] Improved UFH algorithm and mining process

[0047] There's no clear distinction between one-stage and two-stage mining algorithms; they simply perform differently in different scenarios. Therefore, Dawar et al. proposed the UFH algorithm in 2017. The UFH algorithm is essentially a hybrid framework that integrates one-stage and two-stage algorithms. For different scenarios, it can dynamically choose to execute one of the two algorithm systems, or execute part of one system before switching to the other. The difficulty of this approach lies in defining the switching criteria and how to switch. The processing can be divided into the following situations:

[0048] 1) Use only a two-stage mining algorithm. Scan the dataset and construct a tree. Calculate the overestimated utility value. Compare the overestimated utility value with the minimum utility value to eliminate some itemsets. Finally, calculate the true utility value and compare it with the minimum utility threshold to select the high-utility itemsets.

[0049] 2) Use only a single-stage mining algorithm. The database is scanned and stored in lists. Generating candidate itemsets is skipped, and the true utility value is directly calculated. This process is similar to the Apriori algorithm, which generates itemsets of varying lengths. The utility value is then compared with a minimum utility threshold to select high-utility itemsets.

[0050] 3) Use a tree structure to construct a utility tree, calculate overestimated utility values, and first screen out candidate high-utility item sets. A list-based algorithm is then used to calculate the true utility values, thereby screening out high-utility item sets.

[0051] 4) First execute the tree-based algorithm, and at a certain point in the execution, switch to the list-based algorithm. This switching module can define the switching criteria to help select the best switching point to maximize overall efficiency. For example, the switching criteria can be determined by observing the distribution of data. The overall framework of the UFH algorithm is shown in the figure below. Figure 5 shown

[0052] 1) During the first pass of the database scan, the dataset is fuzzified to generate a fuzzy database with an internal utility table. Based on the characteristics of the fuzzy database, the UFH algorithm constructs a header table, calculates overestimated utility values, and preliminarily screens candidate high-utility itemsets. Each node in the header table consists of the item name (item), overestimated utility value (N.nu), support (N.count), a pointer to the parent node (N.parent), and pointers (N.hlink) to other nodes. The root node is an empty node pointing to its child nodes. The itemset formed by the path from the root node to any node in the tree is called a prefix itemset. A header table is also maintained, which primarily contains the item set name, TWU value, and a pointer to the header table. Nodes in the UP tree are sorted in descending order of TWU values ​​in the header table. Identical items in different transactions are linked together using a linked list. During the database scan, the UFH algorithm calculates the transaction-weighted utility (TWU) of all items. Entries with TWU less than the preset minimum utility threshold will be deleted from the database, and the items in the database transaction will be sorted in descending order of TWU value in the header table. The header table is generated according to the data of the new database. The header table process of the data in the sample database is as follows Figures 6 to 12 shown.

[0053] 2) A fuzzy high-utility pattern is identified using the paper mulberry tree algorithm. A one-stage high-utility pattern mining algorithm is then used to directly calculate the true utility value and compare it with a minimum utility threshold to identify all fuzzy high-utility patterns. In this process, a utility list is used to mine high-utility patterns. A utility list is a compact data structure used to store item sets and their utility values ​​in a transaction database. It consists of three columns: TID, Iutils, and Rutils. TID uniquely identifies each transaction in the database. is the actual utility value of item set I in the transaction. represents the residual utility value of item set I. The internal items of each transaction in the database are reorganized in ascending order according to the TWU value of the item set. This reorganized database is called the reorganized database. After the initial database scan, the algorithm identifies item sets with a TWU above the minimum utility threshold and sorts them in ascending order. During the next database scan, the database is reorganized according to the item set's sort order, and a utility list for each item set is generated. After generating a utility list of an item set, the algorithm continuously generates a utility list of a {k}-length item set by combining the utility lists of {k-1}-length item sets. In this process, no high-utility candidate item sets are generated, and in order to reduce the number of connection operations, the EUCP pruning strategy (estimated utility co-occirrence pruning) is used. This pruning data strategy relies on a data structure EUCS. This data structure is constructed when the database is scanned for the second time. For any item set I, determine whether its superset is a high-utility item set. Based on the EUCS data structure, we can directly determine whether there is a superset whose sum of Iutils and Rutils is greater than the minimum utility threshold. If it does not exist, it will not be explored further. This reduces the number of connections. The EUCS structure of the database is as follows Figure 13 shown.

Claims

1. A fuzzy high-utility pattern mining method based on medical scenarios, characterized by: The following steps are involved: S1. Collect various physical examination data of patients in medical scenarios and preprocess them. The preprocessing includes referring to the intervals and standard values ​​of various human indicators in medical standards, filling the values ​​outside the interval range with boundary values ​​to obtain the original database data; S2. After the original database data obtained in step S1 is passed through the fuzzy function, the membership value of the interval is obtained, which is used as the internal utility value, and the external utility table is manually specified, thereby obtaining a fuzzy database that can be mined by the high-utility pattern mining algorithm; S3, the fuzzy database obtained in step S2 is mined using a two-stage mining algorithm, and the switching module is periodically called to determine whether the switching condition is met. If so, the fuzzy database is switched to a one-stage mining algorithm to mine fuzzy high-utility patterns; In step S2, a type I fuzzy function or a type II fuzzy function is selected, and after the data passes through the fuzzy module, the membership values ​​of the low, medium, and high intervals are calculated respectively; after the first scan, the main interval is obtained, and then the membership value of the main interval is obtained. This value is used as the internal utility value in the high-utility pattern mining algorithm, and then the external utility values ​​of various attributes in the physical examination data are set to the same value; Step S3 specifically includes the following sub-steps: S3.

1. Construct a header table based on the fuzzy database content and calculate relevant parameters. Based on the header table, find the path from each leaf node to the root node to obtain the conditional pattern base. Once the traversal is complete, the fuzzy high-utility pattern of all item sets is obtained. S3.

2. Design a switching module that combines the characteristics of the one-stage mining algorithm and the two-stage mining algorithm in the high-utility pattern mining algorithm. The switching module makes the algorithm adaptive to various scenarios and then screens out candidate high-utility item sets. When calculating the true utility value, a list-based algorithm is used to screen out the high-utility item sets. S3.

3. Use utility-list to mine high-utility patterns. In the first scan of the database, the algorithm finds a set of items whose TWU is higher than the minimum utility threshold and arranges them in ascending order. When the database is scanned for the second time, the database is reorganized according to the order of the item sets, and a utility list for each item set is generated. After generating the utility list for each item set, the utility list of the {k-1}-length item set is continuously combined to generate a utility list of the {k}-length item set.

2. The fuzzy high-utility pattern mining method based on medical scenarios according to claim 1 is characterized in that: In step S1 , the data is pre-processed by filling in the medical standard values ​​of human attributes, and then normalized so that all the data fall within the interval determined by the fuzzy function.

3. The fuzzy high-utility pattern mining method based on medical scenarios according to claim 1 is characterized in that: In step S3.2, the switching module is designed to make the algorithm adaptive to various scenarios. The algorithm includes: Only a two-stage mining algorithm is used to calculate the overestimated utility value and then perform preliminary screening, and then calculate the true utility value to mine and obtain high utility patterns; Or only one-stage mining algorithm is used to directly calculate the true utility value and mine the fuzzy high-utility pattern; Or use a tree structure to construct a utility tree and calculate the overestimated utility value.

Images