Data mining method and device based on frequent item set algorithm

Abstract

The invention relates to the field of artificial intelligence, in particular to a data mining method and device based on a frequent item set algorithm, a medium, electronic equipment and a computer program product. According to the method, a dynamic expansion mechanism and a feedback mechanism of the frequent item set are introduced, multi-level association rule mining is supported, an intelligent frequency statistical method is adopted, an expert evaluation method is fused, and timeliness and accuracy of association rules are ensured.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and particularly to a data mining method, device, medium, electronic device, and computer program product based on the frequent item set algorithm. Background Art

[0002] In clinical trials, the quality of data is directly related to the accuracy and reliability of research results. Therefore, it is crucial to conduct strict logical verification on the data collected in clinical trials. Traditional clinical trial data verification methods often rely on static data verification plan (DVP) rule sets, which are defined before the start of the trial and remain unchanged throughout the trial process. However, as the clinical trial project progresses, changes in the protocol or other business requirements may lead to the addition or deletion of data fields, changes in data types, and changes in the association relationships between data. The static DVP rule set gradually reveals its limitations. Summary of the Invention

[0003] Embodiments of this application provide a data mining method, device, medium, electronic device, and computer program product based on the frequent item set algorithm.

[0004] In a first aspect, an embodiment of this application provides a data mining method based on the frequent item set algorithm for an electronic device. The method includes: For the obtained historical data set, use the frequent item set algorithm to mine multiple frequent item sets of the historical data set. During the mining process, based on the association strength between a new item and a specific frequent item set among the multiple frequent item sets, dynamically expand the specific frequent item set to obtain the dynamically expanded specific frequent item set; Generate multiple association rules for the multiple frequent item sets, where the multiple association rules include single-level association rules between two frequent item sets and multi-level association rules between at least three frequent item sets; Statistically analyze the adjusted frequency of a specific association rule among the multiple association rules, and generate an association rule library.

[0005] In a second aspect, an embodiment of this application provides a data mining device based on the frequent item set algorithm. The device includes: A mining unit, which, for the obtained historical data set, uses the frequent item set algorithm to mine multiple frequent item sets of the historical data set. During the mining process, based on the association strength between a new item and a specific frequent item set among the multiple frequent item sets, dynamically expand the specific frequent item set to obtain the dynamically expanded specific frequent item set; A generating unit that generates multiple association rules for the multiple frequent item sets, where the multiple association rules include single-level association rules of two frequent item sets and multi-level association rules of at least three frequent item sets; A statistical unit that statistically calculates the adjusted frequency of a specific association rule among the multiple association rules and generates an association rule library.

[0006] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed by a computer, the computer is caused to execute the data mining method based on the frequent item set algorithm according to the first aspect above.

[0007] In a fourth aspect, an embodiment of the present application provides an electronic device, including one or more processors and one or more memories; one or more programs are stored on the one or more memories. When the one or more programs are executed by the one or more processors, the electronic device is caused to execute the data mining method based on the frequent item set algorithm according to the first aspect above.

[0008] In a fifth aspect, an embodiment of the present application provides a computer program product, including computer-executable instructions that are executed by a processor to implement the data mining method based on the frequent item set algorithm according to the first aspect above.

[0009] The present application introduces a dynamic expansion mechanism for frequent item sets, which allows adding or adjusting item sets dynamically according to the situation of the current frequent item sets during the mining process to capture more subtle association patterns.

[0010] In addition, the present application introduces a feedback mechanism. Business personnel can evaluate and provide feedback on the performance of the frequent item set algorithm according to the actual application effects, and these feedbacks will be used to adjust the algorithm parameters to improve the adaptability and accuracy of the algorithm.

[0011] In addition, in addition to single-level association rule mining, the present application also supports multi-level association rule mining, so as to be able to identify complex association rules spanning multiple levels, such as chain rules, and more complex network-like associations.

[0012] In addition, the present application adopts an intelligent frequency statistics method, which not only records the number of occurrences of association rules, but also considers the mutual influence and dependence relationships between association rules. By introducing a correlation coefficient, it can more accurately evaluate the weights of association rules, rather than based on simple frequency counting.

[0013] In addition, this application incorporates the expert evaluation method, allowing experts to directly adjust the weights of the mined association rules. For example, according to specific information or conditions in the project, additional weights can be assigned to association rules with low frequencies to reflect their importance in practical applications.

[0014] In addition, with the addition of new data or changes in the business environment, this application can re-run the method and update the association rule library to ensure the timeliness and accuracy of the association rules. Brief Description of the Drawings

[0015] Figure 1 According to an embodiment of the present application, a flowchart of a data mining method based on the frequent itemset algorithm is shown; Figure 2 According to an embodiment of the present application, a schematic diagram of a data mining device based on the frequent itemset algorithm is shown; Figure 3 According to an embodiment of the present application, a block diagram of an electronic device is shown. Detailed Description of the Embodiments

[0016] Exemplary embodiments of the present application include, but are not limited to, a data mining method, device, medium, electronic device, and computer program product based on the frequent itemset algorithm.

[0017] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0018] Figure 1 A data mining method based on the frequent itemset algorithm according to an embodiment of the present application is shown for Figure 3 the electronic device shown.

[0019] As Figure 1 shown, in step S11, a historical data set is obtained. Specifically, the historical data set includes a historical DVP rule set.

[0020] Continuing in step S11, for the obtained historical data set, multiple frequent itemsets of the historical data set are mined using the frequent itemset algorithm. Specifically, a minimum support is set, and the support of each k-itemset in the historical data set is calculated to obtain a set of k-item frequent itemsets containing k-itemsets with a support not less than the minimum support, that is, multiple k-item frequent itemsets.

[0021] Continuing in step S11, during the mining process, based on the association strength between a new item and a specific frequent itemset among the multiple frequent itemsets, the specific frequent itemset is dynamically expanded to obtain a dynamically expanded specific frequent itemset. Specifically, the new item and the specific frequent itemset The association strength is the confidence of the two , where For an item set of support, and when the confidence is greater than the confidence threshold it is determined to add the new item to a specific frequent item set to form a dynamically expanded specific frequent item set . It can be understood that the association strength between the new item and the specific frequent item set can be defined according to specific circumstances. For example, it can be other functional relationships of support.

[0022] This application introduces a dynamic expansion mechanism for frequent item sets. This mechanism allows adding or adjusting item sets dynamically according to the current frequent item sets during the mining process to capture more subtle association patterns.

[0023] More specifically, based on the feedback metrics of business personnel the confidence threshold is adjusted, where for example, it includes accuracy, recall, etc., to obtain the feedback confidence threshold , where is the threshold adjustment amount calculated according to the feedback metrics .

[0024] This application introduces a feedback mechanism. Business personnel can evaluate and provide feedback on the performance of the frequent item set algorithm according to the actual application effects, and these feedbacks will be used to adjust the algorithm parameters to improve the adaptability and accuracy of the algorithm.

[0025] Continuing as Figure 1 shown, in step S12, multiple association rules of multiple frequent item sets are generated, where the multiple association rules include single-level association rules of two frequent item sets and multi-level association rules of at least three frequent item sets.

[0026] Specifically, for the first frequent item set and the second frequent item set , when the confidence of the first frequent item set and the confidence of the second frequent item set and the second frequent item set are greater than the minimum confidence, it is determined that there is a single-level association rule between the first frequent item set

[0027] Similarly, for the second frequent item set and the third frequent item set , when the confidence of the second frequent item set of the second frequent item set When it is greater than the minimum confidence, the second frequent item set is determined and the third frequent itemset There is a single-level association rule .

[0028] Furthermore, for the first frequent itemset , the second frequent itemset and the third frequent itemset , when the first frequent itemset and the second frequent itemset Single-level association rules Support And the second frequent itemset and the third frequent itemset Single-level association rules Support Both are greater than the support threshold When , determine the first frequent itemset , the second frequent itemset and the third frequent itemset There are multi-level association rules .

[0029] In addition to single-level association rule mining, this application also supports multi-level association rule mining, so that complex association rules spanning multiple levels can be identified, such as " "Such chain rules, as well as more complex network-like associations.

[0030] Continue as Figure 1 As shown, in step S13, the adjusted frequency of a specific association rule among the multiple association rules is counted. The adjusted frequency is ,in, For a specific association rule The original frequency, For a specific association rule The set of rules associated with it The correlation coefficient of Association rules Association rules The degree of association and the value range is between.

[0031] This application adopts an intelligent frequency statistics method, which not only records the number of times association rules appear, but also considers the mutual influence and dependency between association rules. By introducing the association coefficient, the weight of the association rule can be evaluated more accurately, rather than based on simple frequency counting.

[0032] More specifically, for a specific association rule the adjusted frequency is , where is the expert adjustment factor.

[0033] This application incorporates the expert evaluation method, allowing experts to directly adjust the weights of the mined association rules. For example, according to specific information or conditions in the project, additional weights can be assigned to association rules with low frequencies to reflect their importance in practical applications.

[0034] Continuing in step S13, an association rule library is generated. Specifically, when new data is added, the data mining method can be re-executed to generate a new association rule library.

[0035] With the addition of new data or changes in the business environment, this application can re-run the method and update the association rule library to ensure the timeliness and accuracy of the association rules.

[0036] Figure 2 Shows a data mining device based on the frequent item set algorithm according to an embodiment of this application.

[0037] As Figure 2 shown, for the acquired historical data set, the mining unit 201 uses the frequent item set algorithm to mine multiple frequent item sets of the historical data set. Among them, during the mining process, based on the association strength between the new item and a specific frequent item set among the multiple frequent item sets, the specific frequent item set is dynamically expanded to obtain the dynamically expanded specific frequent item set; the generation unit 202 generates multiple association rules for the multiple frequent item sets, where the multiple association rules include single-level association rules for two frequent item sets and multi-level association rules for at least three frequent item sets; the statistical unit 203 statistically calculates the adjusted frequency of a specific association rule among the multiple association rules and generates an association rule library.

[0038] It can be understood that the mining unit 201, the generation unit 202, and the statistical unit 203 can be implemented by a processor 302 with the functions of these modules or units in an Figure 3 electronic device.

[0039] This application also provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed by a computer, the computer is made to execute the Figure 1 method shown in

[0040] This application also provides a computer program product, including computer-executable instructions, and the instructions are executed by a processor to implement the Figure 1 method shown in

[0041] Figure 3 Shows an electronic device according to an embodiment of this application.

[0042] As shown Figure 3 in the figure, the electronic device may include one or more processors 302, system control logic 308 connected to at least one of the processors 302, system memory 304 connected to the system control logic 308, non-volatile memory (NVM) 306 connected to the system control logic 308, and a communication interface 310 connected to the system control logic 308.

[0043] The processor 302 may include one or more single-core or multi-core processors. The processor 302 may include any combination of general-purpose processors and dedicated processors (e.g., graphics processors, application processors, baseband processors, etc.). In the embodiments herein, the processor 302 may be configured to execute Figure 1 the method shown in

[0044] In some embodiments, the system control logic 308 may include any suitable interface controller to provide any suitable interface to at least one of the processors 302 and / or any suitable device or component communicating with the system control logic 308.

[0045] In some embodiments, the system control logic 308 may include one or more memory controllers to provide an interface connected to the system memory 304. The system memory 304 may be used to load and store data and / or instructions. In some embodiments, the system memory 304 of the electronic device may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).

[0046] The NVM / memory 306 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the NVM / memory 306 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of an HDD (Hard Disk Drive), a CD (Compact Disc) drive, and a DVD (Digital Versatile Disc) drive.

[0047] The NVM / memory 306 may include a portion of the storage resources installed on the device of the electronic device, or it may be accessible by the device but not necessarily part of the device. For example, the NVM / storage 306 may be accessed via the communication interface 310 through a network.

[0048] Specifically, the system memory 304 and the NVM / memory 306 may respectively include: a temporary copy and a permanent copy of the instructions 320. The instructions 320 may include: when executed by at least one of the processors 302, causing the electronic device to implementFigure 1 Instructions of the method shown in. In some embodiments, instructions 320, hardware, firmware, and / or their software components may additionally / alternatively be placed in system control logic 308, communication interface 310, and / or processor 302.

[0049] Communication interface 310 may include a transceiver for providing a radio interface for the electronic device, and thus communicating with any other suitable devices (such as a front-end module, an antenna, etc.) via one or more networks. In some embodiments, communication interface 310 may be integrated with other components of the electronic device. For example, communication interface 310 may be integrated with at least one of processor 302, system memory 304, NVM / memory 306, and a firmware device with instructions (not shown), and when at least one of the instructions is executed by processor 302, the electronic device implements Figure 1 the method shown in.

[0050] Communication interface 310 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, communication interface 310 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0051] In one embodiment, at least one of processors 302 may be packaged together with the logic of one or more controllers for system control logic 308 to form a system-in-package (SiP). In one embodiment, at least one of processors 302 may be integrated with the logic of one or more controllers for system control logic 308 on the same die to form a system-on-chip (SoC).

[0052] The electronic device may further include: an input / output (I / O) device 312. I / O device 312 may include a user interface that enables a user to interact with the electronic device; the design of the peripheral component interface enables peripheral components to also interact with the electronic device. In some embodiments, the electronic device further includes sensors for determining at least one of environmental conditions and location information related to the electronic device.

[0053] In some embodiments, the user interface may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., a light-emitting diode flash), and a keyboard.

[0054] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0055] In some embodiments, the sensors may include, but are not limited to, gyro sensors, accelerometers, proximity sensors, ambient light sensors, and positioning units. The positioning unit may also be part of or interact with the communication interface 310 to communicate with components of a positioning network (e.g., Global Positioning System (GPS) satellites).

[0056] It can be understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than those shown, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0057] Program code can be applied to the input instructions to perform the various functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0058] The program code can be implemented in a high-level procedural language or an object-oriented programming language to communicate with the processing system. When needed, the program code can also be implemented in assembly language or machine language. In fact, the mechanisms described herein are not limited to the scope of any particular programming language. In any case, the language can be a compiled language or an interpreted language.

[0059] One or more aspects of at least one embodiment can be implemented by representative instructions stored on a computer-readable storage medium, the instructions representing various logics in a processor, and the instructions, when read by a machine, cause the machine to fabricate the logics for performing the techniques described herein. These representations, referred to as “IP cores,” can be stored on a tangible computer-readable storage medium and provided to multiple customers or production facilities to be loaded into the manufacturing machines that actually fabricate the logics or processors.

[0060] Unless the context otherwise requires, the terms “comprise,” “have,” and “include” are synonyms. The phrase “A / B” means “A or B.” The phrase “A and / or B” means “(A and B) or (A or B).”

[0061] As used herein, the term “module” or “unit” can refer to, be, or include: an application specific integrated circuit (ASIC), an electronic circuit, a (shared, dedicated, or group) processor and / or memory that executes one or more software or firmware programs, combinational logic circuits, and / or other suitable components that provide the described functionality.

[0062] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or via other computer-readable media. Thus, machine-readable media may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), but not limited to, floppy disks, optical disks, CD-ROMs, magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, flash memory, or tangible machine-readable memories for transmitting information via electrical, optical, acoustic, or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.) over the Internet. Thus, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0063] In the drawings, some structural or method features are shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or ordering may not be required. In some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Additionally, the inclusion of a structural or method feature in a particular figure does not imply that such a feature is required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0064] It should be understood that although terms such as "first", "second", etc. may be used herein to describe various elements or data, these elements or data should not be limited by these terms. These terms are only used to distinguish one feature from another. For example, without departing from the scope of the exemplary embodiments, a first feature may be referred to as a second feature, and similarly, a second feature may be referred to as a first feature.

[0065] It should be noted that in this specification, like reference numerals and letters denote like items in the following drawings, and thus, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings.

Claims

1. A data mining method based on the frequent item set algorithm for an electronic device, characterized in that, The method includes: For the obtained historical data set, use the frequent item set algorithm to mine multiple frequent item sets of the historical data set. Among them, during the mining process, based on the association strength between the new item and a specific frequent item set among the multiple frequent item sets, dynamically expand the specific frequent item set to obtain the dynamically expanded specific frequent item set; Generate multiple association rules for the multiple frequent item sets, where the multiple association rules include single-level association rules of two frequent item sets and multi-level association rules of at least three frequent item sets; Statistically analyze the adjusted frequency of a specific association rule among the multiple association rules, and generate an association rule library.

2. The method according to claim 1, wherein New item The association strength with a specific frequent item set is the confidence of the two, where , is the support of the item set , and when the confidence is greater than the confidence threshold , it is determined to add the new item to the specific frequent item set to form a dynamically expanded specific frequent item set .

3. The method according to claim 2, wherein Based on the feedback metrics of business personnel to adjust the confidence threshold to obtain the confidence threshold after feedback , where is the threshold adjustment amount calculated according to the feedback metrics ​ 4. The method according to claim 3, wherein For the first frequent itemset , the second frequent itemset and the third frequent itemset , when the support degree of the single-level association rule between the first frequent itemset and the second frequent itemset and the support degree of the single-level association rule between the second frequent itemset and the third frequent itemset are both greater than the support degree threshold , it is determined that there is a multi-level association rule among the first frequent itemset , the second frequent itemset and the third frequent itemset .

5. The method according to claim 4, wherein Specific association rule The adjusted frequency is , where is the original frequency of the specific association rule , is the correlation coefficient between the specific association rule and its associated rule set , and is the degree of association between association rule and association rule , and its value range is between .

6. The method according to claim 5, characterized in that, Specific association rules The adjusted frequency is , where is the expert adjustment factor.

7. A data mining device based on a frequent item set algorithm, characterized in that, The apparatus includes: A mining unit that, for the obtained historical data set, uses the frequent item set algorithm to mine multiple frequent item sets of the historical data set. Among them, during the mining process, based on the association strength between the new item and a specific frequent item set among the multiple frequent item sets, dynamically expand the specific frequent item set to obtain the dynamically expanded specific frequent item set; A generating unit that generates multiple association rules for the multiple frequent item sets, where the multiple association rules include single-level association rules of two frequent item sets and multi-level association rules of at least three frequent item sets; A statistical unit that statistically analyzes the adjusted frequency of a specific association rule among the multiple association rules, and generates an association rule library.

8. A computer-readable storage medium, characterized in that, Instructions are stored on the storage medium, and when the instructions are executed by a computer, the computer executes the data mining method based on the frequent item set algorithm according to any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes one or more processors and one or more memories; one or more programs are stored on the one or more memories, and when the one or more programs are executed by the one or more processors, the electronic device executes the data mining method based on the frequent item set algorithm according to any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer-executable instructions that are executed by a processor to implement the data mining method based on the frequent item set algorithm according to any one of claims 1 to 6.

Images