Data mining method and device based on frequent item set algorithm

Abstract

This application relates to the field of artificial intelligence, and more particularly to a data mining method, apparatus, medium, electronic device, and computer program product based on a frequent itemset algorithm. This application introduces a dynamic expansion mechanism and feedback mechanism for frequent itemsets, supports multi-level association rule mining, employs intelligent frequency statistics, integrates expert evaluation methods, and ensures the timeliness and accuracy of association rules.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a data mining method, device, medium, electronic device, and computer program product based on a frequent itemset algorithm. Background Art

[0002] In clinical trials, data quality is directly related to the accuracy and reliability of research results. Therefore, rigorous logical verification of clinical trial data is crucial. Traditional clinical trial data verification methods often rely on static Data Verification Plan (DVP) rule sets, which are defined before the trial begins and remain unchanged throughout the trial. However, as clinical trial projects progress, static DVP rule sets gradually reveal their limitations due to potential protocol changes or other business requirements, resulting in the addition or deletion of data fields, changes in data types, and changes in data relationships. Summary of the Invention

[0003] The embodiments of the present application provide a data mining method, apparatus, medium, electronic device, and computer program product based on a frequent itemset algorithm.

[0004] In a first aspect, an embodiment of the present application provides a data mining method based on a frequent itemset algorithm, for use in an electronic device, the method comprising:

[0005] For the acquired historical data set, a frequent itemset algorithm is used to mine a plurality of frequent itemsets of the historical data set, wherein, during the mining process, based on the association strength between a new item and a specific frequent itemset in the multiple frequent itemsets, the specific frequent itemset is dynamically expanded to obtain the dynamically expanded specific frequent itemset;

[0006] generating a plurality of association rules for the plurality of frequent item sets, wherein the plurality of association rules include single-level association rules for two frequent item sets and multi-level association rules for at least three frequent item sets;

[0007] The adjusted frequency of a specific association rule among the plurality of association rules is counted, and an association rule base is generated.

[0008] In a second aspect, an embodiment of the present application provides a data mining device based on a frequent itemset algorithm, the device comprising:

[0009] a mining unit, for mining the acquired historical data set using a frequent itemset algorithm to obtain a plurality of frequent itemsets from the historical data set, wherein, during the mining process, based on the association strength between a new item and a specific frequent itemset in the multiple frequent itemsets, the specific frequent itemset is dynamically expanded to obtain the dynamically expanded specific frequent itemset;

[0010] a generating unit, configured to generate a plurality of association rules for the plurality of frequent item sets, wherein the plurality of association rules include single-level association rules for two frequent item sets and multi-level association rules for at least three frequent item sets;

[0011] The statistical unit counts the adjusted frequency of a specific association rule among the plurality of association rules and generates an association rule base.

[0012] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed by a computer, the computer executes the data mining method based on the frequent itemset algorithm described in the first aspect above.

[0013] In a fourth aspect, an embodiment of the present application provides an electronic device comprising one or more processors and one or more memories; one or more programs are stored on the one or more memories, and when the one or more programs are executed by the one or more processors, the electronic device executes the data mining method based on the frequent item set algorithm described in the first aspect above.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer-executable instructions, which are executed by a processor to implement the data mining method based on the frequent itemset algorithm according to the first aspect above.

[0015] This application introduces a dynamic expansion mechanism for frequent itemsets, which allows itemsets to be dynamically added or adjusted according to the current situation of frequent itemsets during the mining process to capture more subtle association patterns.

[0016] In addition, this application introduces a feedback mechanism, which allows business personnel to evaluate and provide feedback on the performance of the frequent itemset algorithm based on actual application results. These feedbacks will be used to adjust the algorithm parameters to improve the adaptability and accuracy of the algorithm.

[0017] In addition, in addition to single-level association rule mining, this application also supports multi-level association rule mining, so that complex association rules spanning multiple levels, such as chain rules, and more complex network-like associations can be identified.

[0018] In addition, this application adopts an intelligent frequency statistics method, which not only records the number of times association rules appear, but also considers the mutual influence and dependency between association rules. By introducing the correlation coefficient, the weight of the association rules can be evaluated more accurately, rather than based on simple frequency counting.

[0019] In addition, this application integrates the expert evaluation method, allowing experts to directly adjust the weights of the mined association rules. For example, based on specific information or conditions in the project, low-frequency association rules can be additionally weighted to reflect their importance in practical applications.

[0020] In addition, as new data is added or the business environment changes, the present application can re-run the method and update the association rule library to ensure the timeliness and accuracy of the association rules. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 According to an embodiment of the present application, a flow chart of a data mining method based on a frequent itemset algorithm is shown;

[0022] Figure 2 According to an embodiment of the present application, a schematic diagram of a data mining device based on a frequent itemset algorithm is shown;

[0023] Figure 3 According to an embodiment of the present application, a block diagram of an electronic device is shown. DETAILED DESCRIPTION

[0024] The illustrative embodiments of the present application include, but are not limited to, data mining methods, apparatuses, media, electronic devices, and computer program products based on frequent item set algorithms.

[0025] The embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0026] Figure 1 The data mining method based on the frequent item set algorithm of the embodiment of the present application is shown. Figure 3 Electronic devices shown.

[0027] like Figure 1 As shown, in step S11, a historical data set is obtained. Specifically, the historical data set includes a historical DVP rule set.

[0028] Continuing in step S11, for the acquired historical data set, a frequent itemset algorithm is used to mine multiple frequent itemsets in the historical data set. Specifically, a minimum support is set, and the support of each k-itemset in the historical data set is calculated to obtain a set of k-item frequent itemsets containing k-itemsets whose support is not less than the minimum support, i.e., multiple k-item frequent itemsets.

[0029] Continuing in step S11, during the mining process, based on the association strength between the new item and a specific frequent item set in multiple frequent item sets, the specific frequent item set is dynamically expanded to obtain a dynamically expanded specific frequent item set. With a specific frequent itemset The strength of the association is the confidence level of the two ,in, For item sets The support of Greater than the confidence threshold When you confirm that the new item Add to a specific frequent itemset The specific frequent itemsets after dynamic expansion are formed in Understandably, the new With a specific frequent itemset The association strength can be defined according to the specific situation, for example, it can be other functional relationships of support.

[0030] This application introduces a dynamic expansion mechanism for frequent itemsets, which allows itemsets to be dynamically added or adjusted according to the current situation of frequent itemsets during the mining process to capture more subtle association patterns.

[0031] More specifically, based on feedback from business personnel To adjust the confidence threshold ,in, For example, including accuracy, recall rate, etc., to obtain the confidence threshold after feedback ,in, Based on feedback indicators The calculated threshold adjustment amount.

[0032] This application introduces a feedback mechanism, which allows business personnel to evaluate and provide feedback on the performance of the frequent itemset algorithm based on actual application results. This feedback will be used to adjust the algorithm parameters to improve the adaptability and accuracy of the algorithm.

[0033] Continue as Figure 1 As shown, in step S12, multiple association rules of multiple frequent item sets are generated, wherein the multiple association rules include single-level association rules of two frequent item sets and multi-level association rules of at least three frequent item sets.

[0034] Specifically, for the first frequent itemset and the second frequent itemset , when the first frequent itemset and the second frequent itemset Confidence When it is greater than the minimum confidence, the first frequent item set is determined and the second frequent itemset There is a single-level association rule .

[0035] Similarly, for the second frequent itemset and the third frequent itemset , when the second frequent itemset and the third frequent itemset Confidence When it is greater than the minimum confidence, the second frequent item set is determined and the third frequent itemset There is a single-level association rule .

[0036] Furthermore, for the first frequent itemset , the second frequent itemset and the third frequent itemset , when the first frequent itemset and the second frequent itemset Single-level association rules Support and the second frequent itemset and the third frequent itemset Single-level association rules Support Both are greater than the support threshold When , determine the first frequent itemset , the second frequent itemset and the third frequent itemset There are multi-level association rules .

[0037] In addition to single-level association rule mining, this application also supports multi-level association rule mining, so that complex association rules spanning multiple levels can be identified, such as " "Such chain rules, as well as more complex network-like associations.

[0038] Continue as Figure 1 As shown, in step S13, the adjusted frequency of a specific association rule among the multiple association rules is counted. Specifically, the specific association rule The adjusted frequency is ,in, For specific association rules The original frequency, For specific association rules Its related rule set The correlation coefficient, and Association rules Association rules The degree of association and the value range is between.

[0039] This application adopts an intelligent frequency statistics method, which not only records the number of times association rules appear, but also considers the mutual influence and dependency between association rules. By introducing the correlation coefficient, the weight of association rules can be evaluated more accurately, rather than based on simple frequency counting.

[0040] More specifically, specific association rules The adjusted frequency is ,in, Adjustment factor for experts.

[0041] This application integrates the expert evaluation method, allowing experts to directly adjust the weights of the mined association rules. For example, based on specific information or conditions in the project, low-frequency association rules can be given additional weighting to reflect their importance in practical applications.

[0042] Continuing in step S13, an association rule base is generated. Specifically, when new data is added, the data mining method can be re-executed to generate a new association rule base.

[0043] As new data is added or the business environment changes, the method can be re-run and the association rule library can be updated to ensure the timeliness and accuracy of the association rules.

[0044] Figure 2 The data mining device based on the frequent itemset algorithm according to an embodiment of the present application is shown.

[0045] like Figure 2 As shown, the mining unit 201 mines multiple frequent item sets from the acquired historical data set using a frequent item set algorithm, wherein during the mining process, based on the association strength between the new item and a specific frequent item set in the multiple frequent item sets, the specific frequent item set is dynamically expanded to obtain the dynamically expanded specific frequent item set; the generating unit 202 generates multiple association rules for the multiple frequent item sets, wherein the multiple association rules include single-level association rules for two frequent item sets and multi-level association rules for at least three frequent item sets; the statistical unit 203 counts the adjusted frequencies of specific association rules in the multiple association rules, and generates an association rule base.

[0046] It is understandable that the mining unit 201, the generating unit 202 and the statistical unit 203 can be Figure 3 The processor 302 in the electronic device having the functions of these modules or units is implemented.

[0047] The present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed by a computer, the computer executes Figure 1 The method shown in .

[0048] The present application also provides a computer program product comprising computer executable instructions, which are executed by a processor to implement Figure 1 The method shown in .

[0049] Figure 3 An electronic device according to an embodiment of the present application is shown.

[0050] like Figure 3 As shown, the electronic device may include one or more processors 302 , system control logic 308 connected to at least one of the processors 302 , system memory 304 connected to the system control logic 308 , non-volatile memory (NVM) 306 connected to the system control logic 308 , and a communication interface 310 connected to the system control logic 308 .

[0051] The processor 302 may include one or more single-core or multi-core processors. The processor 302 may include any combination of general-purpose processors and specialized processors (e.g., graphics processors, application processors, baseband processors, etc.). In the embodiments herein, the processor 302 may be configured to execute Figure 1 The method shown in .

[0052] In some embodiments, system control logic 308 may include any suitable interface controller to provide any suitable interface to at least one of processors 302 and / or any suitable device or component in communication with system control logic 308 .

[0053] In some embodiments, system control logic 308 may include one or more memory controllers to provide an interface to system memory 304. System memory 304 may be used to load and store data and / or instructions. In some embodiments, system memory 304 of an electronic device may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).

[0054] NVM / memory 306 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, NVM / memory 306 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of an HDD (Hard Disk Drive), a CD (Compact Disc) drive, and a DVD (Digital Versatile Disc) drive.

[0055] NVM / storage 306 may include a portion of storage resources installed on a device of an electronic device, or it may be accessible to the device but not necessarily a part of the device. For example, NVM / storage 306 may be accessed over a network via communication interface 310 .

[0056] In particular, system memory 304 and NVM / storage 306 may include, respectively, a temporary copy and a permanent copy of instructions 320. Instructions 320 may include instructions that, when executed by at least one of processors 302, cause the electronic device to perform Figure 1 In some embodiments, instructions 320 , hardware, firmware, and / or software components thereof may additionally or alternatively be located in system control logic 308 , communication interface 310 , and / or processor 302 .

[0057] The communication interface 310 may include a transceiver for providing a radio interface for the electronic device to communicate with any other suitable device (such as a front-end module, antenna, etc.) via one or more networks. In some embodiments, the communication interface 310 may be integrated with other components of the electronic device. For example, the communication interface 310 may be integrated with at least one of the processor 302, the system memory 304, the NVM / storage 306, and a firmware device (not shown) having instructions. When at least one of the processors 302 executes the instructions, the electronic device implements Figure 1 The method shown in .

[0058] The communication interface 310 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the communication interface 310 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0059] In one embodiment, at least one of the processors 302 may be packaged together with logic for one or more controllers of the system control logic 308 to form a system-in-package (SiP). In one embodiment, at least one of the processors 302 may be integrated on the same die with logic for one or more controllers of the system control logic 308 to form a system-on-chip (SoC).

[0060] The electronic device may further include an input / output (I / O) device 312. The I / O device 312 may include a user interface that enables a user to interact with the electronic device, and peripheral component interfaces designed to enable peripheral components to interact with the electronic device. In some embodiments, the electronic device also includes a sensor for determining at least one of environmental conditions and location information related to the electronic device.

[0061] In some embodiments, the user interface may include, but is not limited to, a display (e.g., an LCD display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., an LED flash), and a keyboard.

[0062] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0063] In some embodiments, the sensors may include, but are not limited to, a gyroscope sensor, an accelerometer, a proximity sensor, an ambient light sensor, and a positioning unit. The positioning unit may also be part of or interact with the communication interface 310 to communicate with components of a positioning network (e.g., Global Positioning System (GPS) satellites).

[0064] It is understood that the structures illustrated in the embodiments of the present application do not constitute specific limitations on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0065] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0066] Program code can be implemented with a high-level programming language or an object-oriented programming language to communicate with the processing system. Where necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described herein is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0067] One or more aspects of at least one embodiment may be implemented as representative instructions stored on a computer-readable storage medium, which represent various logic within a processor and, when read by a machine, causes the machine to fabricate logic for performing the techniques described herein. These representations, known as "IP cores," may be stored on a tangible, computer-readable storage medium and supplied to various customers or manufacturing facilities to load into fabrication machines that actually manufacture the logic or processor.

[0068] Unless the context dictates otherwise, the terms "comprising," "having," and "including" are synonymous. The phrase "A / B" means "A or B." The phrase "A and / or B" means "(A and B) or (A or B)."

[0069] As used herein, the term “module” or “unit” may refer to, be or include: an application specific integrated circuit (ASIC), an electronic circuit, a (shared, dedicated or group) processor and / or memory that executes one or more software or firmware programs, a combinational logic circuit and / or other suitable components that provide the described functionality.

[0070] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, the instructions may be distributed over a network or via other computer-readable media. Thus, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including, but not limited to, floppy disks, compact disks, CD-ROMs, magneto-optical disks, ROMs, random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or tangible machine-readable storage for transmitting information via electrical, optical, acoustical, or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.) over the Internet. Thus, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0071] In the accompanying drawings, some structural or method features are shown in a specific arrangement and / or order. However, it should be understood that such specific arrangement and / or order may not be required. In some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0072] It should be understood that although the terms "first," "second," and the like may be used herein to describe various elements or data, these elements or data should not be limited by these terms. These terms are used only to distinguish one feature from another. For example, a first feature may be referred to as a second feature, and similarly, a second feature may be referred to as a first feature without departing from the scope of the exemplary embodiments.

[0073] It should be noted that in this specification, similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

Claims

1. A data mining method based on a frequent itemset algorithm, used in electronic equipment, characterized in that: The method comprises: For the acquired historical data set, a frequent itemset algorithm is used to mine a plurality of frequent itemsets of the historical data set, wherein, during the mining process, based on the association strength between a new item and a specific frequent itemset in the multiple frequent itemsets, the specific frequent itemset is dynamically expanded to obtain the dynamically expanded specific frequent itemset; generating a plurality of association rules for the plurality of frequent item sets, wherein the plurality of association rules include single-level association rules for two frequent item sets and multi-level association rules for at least three frequent item sets; Counting the adjusted frequencies of specific association rules among the plurality of association rules, and generating an association rule library; Among them, for the first frequent item set and the second frequent itemset , when the first frequent itemset and the second frequent itemset Confidence When it is greater than the minimum confidence, the first frequent item set is determined and the second frequent itemset There is a single-level association rule ; For the second frequent itemset and the third frequent itemset , when the second frequent itemset and the third frequent itemset Confidence When it is greater than the minimum confidence, the second frequent item set is determined and the third frequent itemset There is a single-level association rule ; For the first frequent itemset , the second frequent itemset and the third frequent itemset , when the first frequent itemset and the second frequent itemset Single-level association rules Support And the second frequent itemset and the third frequent itemset Single-level association rules Support Both are greater than the support threshold When the first frequent itemset is determined , the second frequent itemset and the third frequent itemset There are multi-level association rules .

2. The method according to claim 1, characterized in that New Item With a specific frequent itemset The strength of the association is the confidence level of the two ,in, For item sets The support, and when the confidence Greater than the confidence threshold When the new item is determined Add to the specific frequent itemset The specific frequent itemsets after dynamic expansion are formed in .

3. The method according to claim 2, characterized in that Feedback indicators based on business personnel To adjust the confidence threshold The confidence threshold after feedback ,in, Based on feedback indicators The calculated threshold adjustment amount.

4. The method according to claim 3, characterized in that Specific association rules The adjusted frequency is ,in, For the specific association rule The original frequency, For the specific association rule Its related rule set The correlation coefficient, and Association rules Association rules The degree of association and the value range is between.

5. The method according to claim 4, characterized in that Specific association rules The adjusted frequency is ,in, Adjustment factor for experts.

6. A data mining device based on a frequent itemset algorithm, characterized in that: The device comprises: a mining unit, for mining the acquired historical data set using a frequent itemset algorithm to obtain a plurality of frequent itemsets from the historical data set, wherein, during the mining process, based on the association strength between a new item and a specific frequent itemset in the multiple frequent itemsets, the specific frequent itemset is dynamically expanded to obtain the dynamically expanded specific frequent itemset; a generating unit, configured to generate a plurality of association rules for the plurality of frequent item sets, wherein the plurality of association rules include single-level association rules for two frequent item sets and multi-level association rules for at least three frequent item sets; a statistical unit for counting the adjusted frequency of a specific association rule among the plurality of association rules and generating an association rule base; Among them, for the first frequent item set and the second frequent itemset , when the first frequent itemset and the second frequent itemset Confidence When it is greater than the minimum confidence, the first frequent item set is determined and the second frequent itemset There is a single-level association rule ; For the second frequent itemset and the third frequent itemset , when the second frequent itemset and the third frequent itemset Confidence When it is greater than the minimum confidence, the second frequent item set is determined and the third frequent itemset There is a single-level association rule ; For the first frequent itemset , the second frequent itemset and the third frequent itemset , when the first frequent itemset and the second frequent itemset Single-level association rules Support And the second frequent itemset and the third frequent itemset Single-level association rules Support Both are greater than the support threshold When the first frequent itemset is determined , the second frequent itemset and the third frequent itemset There are multi-level association rules .

7. A computer-readable storage medium, characterized in that The storage medium stores instructions, and when the instructions are executed by a computer, the computer is enabled to execute the data mining method based on the frequent itemset algorithm according to any one of claims 1 to 5.

8. An electronic device, characterized in that: The electronic device comprises one or more processors and one or more memories; one or more programs are stored on the one or more memories, and when the one or more programs are executed by the one or more processors, the electronic device executes the data mining method based on the frequent item set algorithm according to any one of claims 1 to 5.

9. A computer program product, characterized in that The method comprises computer-executable instructions, wherein the instructions are executed by a processor to implement the data mining method based on the frequent item set algorithm according to any one of claims 1 to 5.

Images