Method, device and equipment for carrying out association rule mining through genetic algorithm and medium

Abstract

The invention discloses a method, device and equipment for carrying out association rule mining through a genetic algorithm and a medium. The method comprises the steps of obtaining original log data and performing initialization to obtain initialized log data; associating the initialized data according to a predefined association strategy, and calculating a support degree and a confidence degree of each association rule; encoding each association rule, taking the values of the support degree and the confidence obtained by calculation as real values, and combining the real values with each encoded rule to obtain a plurality of initial individuals in the genetic algorithm to form an initial population; performing multi-round iteration on the initial population according to three modes of selection, crossover and variation until convergence; and extracting the association rule with the highest fitness as an optimal association rule and outputting the optimal association rule to the user side. According to the embodiment of the invention, the log association rule is optimally mined by utilizing the global search characteristic of the genetic algorithm, so that the accuracy of the mining result is enhanced while the rule mining efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data mining and information processing, and in particular to a method, device, equipment and medium for mining association rules using a genetic algorithm. Background Art

[0002] The amount of log data generated by modern information systems is huge, the data structure is complex, and it contains a large amount of unstructured and semi-structured data. Traditional association rule mining methods not only have large computational complexity and high time complexity, but also have difficulty meeting the requirements of real-time and efficiency. In addition, when processing these data, they often adopt greedy strategies, which are easy to fall into local optimal solutions and cannot globally search for the optimal rule combination, affecting the effect of rule mining.

[0003] At the same time, due to the lack of optimization of rules, the association rules generated by traditional methods may have redundancy, duplication or low quality problems, and are difficult to be directly applied to actual business. Therefore, this application mines association rules based on genetic algorithms to obtain the global optimal solution. Summary of the invention

[0004] Based on this, the present invention provides a method, device, equipment and medium for mining association rules using a genetic algorithm to solve the problem that traditional association rule mining methods are inefficient and prone to falling into local optimality.

[0005] In a first aspect, an embodiment of the present invention provides a method for mining association rules using a genetic algorithm, the method comprising:

[0006] In response to an instruction of association rule mining initiated by a user terminal, original log data is acquired and initialized to obtain initialized log data;

[0007] The initialization data is associated according to a predefined association strategy to obtain multiple association rules, and the support and confidence of each association rule are calculated according to the pre-built support function and confidence function;

[0008] Encode each association rule according to a preset encoding rule, and use the support and confidence values ​​calculated under each association rule as real values, and combine them with each encoded rule to obtain multiple initial individuals in the genetic algorithm to form an initial population;

[0009] The initial population is iterated multiple times in three ways: selection, crossover, and mutation until convergence;

[0010] All new individuals participating in the last round of iteration are sorted in descending order according to the fitness value, and the association rules corresponding to the individuals with the highest fitness are extracted according to the sorting results and output to the user end as the optimal association rules.

[0011] In a second aspect, an embodiment of the present invention provides a device for mining association rules using a genetic algorithm, the device comprising:

[0012] The log initialization module is used to respond to the instruction of association rule mining initiated by the user end, obtain the original log data and initialize it to obtain the initialized log data;

[0013] An association rule calculation module is used to associate the initialization data according to a predefined association strategy to obtain multiple association rules, and calculate the support and confidence of each association rule according to a pre-built support function and confidence function;

[0014] The initial population construction module is used to encode each association rule according to a preset encoding rule, and use the support and confidence values ​​calculated under each association rule as real values, and combine them with each encoded rule to obtain multiple initial individuals in the genetic algorithm to form an initial population;

[0015] The population iteration module is used to perform multiple rounds of iterations on the initial population in three ways: selection, crossover, and mutation until convergence;

[0016] The association rule mining module is used to sort all new individuals participating in the last round of iteration in descending order of fitness value, and extract the association rules corresponding to the individuals with the highest fitness according to the sorting results, and output them to the user end as the optimal association rules.

[0017] In a third aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for mining association rules using a genetic algorithm as described in any embodiment of the present invention.

[0021] In a fourth aspect, a computer-readable storage medium is also provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a method for performing association rule mining using a genetic algorithm as described in any embodiment of the present invention when executed.

[0022] The technical solution of the embodiment of the present invention utilizes the global search feature of the genetic algorithm to optimize the mining of log association rules. First, the support and confidence are encoded in real numbers and stored in the gene sequence of the individual. The automatic optimization of parameters is achieved through iteration and evaluation, avoiding the tedious process of manually setting thresholds, which not only improves the efficiency of rule mining, but also enhances the accuracy of mining results. Secondly, the selection, crossover and mutation operations in the genetic algorithm increase the diversity of rules, help the algorithm search for the optimal solution in a wider solution space, and improve the global search capability.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a flow chart of a method for mining association rules using a genetic algorithm provided by Embodiment 1 of the present invention;

[0026] Figure 2 is a flow chart of another method for mining association rules using a genetic algorithm provided in Embodiment 2 of the present invention;

[0027] Figure 3 This is a schematic diagram of the structure of a method and device for performing association rule mining using a genetic algorithm according to Embodiment 3 of the present invention;

[0028] Figure 4 The present invention is a schematic diagram of the structure of an electronic device for implementing a method for mining association rules using a genetic algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] Figure 1 This is a flowchart of a method for mining association rules using a genetic algorithm provided in the first embodiment of the present invention. This embodiment is applicable to mining the optimal association rules in logs. The method can be executed by a device for mining association rules using a genetic algorithm. The device can be implemented in the form of hardware and / or software and can be configured in a data processing system. Figure 1 As shown, the method includes:

[0033] S110 , in response to an instruction for mining association rules initiated by the user terminal, original log data is acquired and initialized to obtain initialized log data.

[0034] The amount of raw log data is huge and the data structure is complex, containing a large amount of unstructured and semi-structured data. Therefore, initialization mainly includes three processes: log cleaning, log formatting, and feature extraction. First, by analyzing the structure in the raw log data, invalid, duplicate, or erroneous log entries are identified. For example, duplicate log records can be detected and deleted, or log entries with abnormal timestamps or missing key fields can be deleted. Secondly, by parsing the structured or semi-structured data of the log, it is standardized to the same format, such as converting all timestamps to UTC format and unifying IP addresses to standard IPV4 / IPV6 representation. Finally, the key fields in the log are extracted through regular expressions or pattern matching algorithms, for example, timestamps, user IDs, and operation types are extracted from text logs. The data obtained after the above operations is the initialization log data.

[0035] In a specific example, the original log data may be: [2024-09-10 12:34:56] IP: 192.168.0.1, UserID: 12345, Action: Login, Status: Success; [2024-09-10 12:34:56] IP: 192.168.0.1, UserID: 12345, Action: Login, Status: Success; [2024-09-10 12:35:05] IP: 192.168.0.2, UserID: 67890, Action: Download, Status: Success. After initialization, we get: {timestamp:2024-09-10T12:34:56Z,ip:192.168.0.1,user:12345,action:Login,status:Success}; {timestamp:2024-09-10T12:35:05Z,ip:192.168.0.2,user:67890,action:Download,status:Success}.

[0036] S120 , associating the initialized data according to a predefined association strategy to obtain a plurality of association rules, and calculating the support and confidence of each association rule according to a pre-constructed support function and confidence function.

[0037] The association strategy specifies the combination relationship between each association rule, where each association rule consists of an antecedent and a consequent. The antecedent is the condition part of the rule, which consists of an attribute and an attribute value pair, and the consequent is the result part of the rule, which consists of an attribute and an attribute value pair. The support is defined as the proportion of the antecedent and the consequent of an association rule that appear simultaneously in the log, and the support of each rule is calculated by counting the frequency; the confidence is defined as the proportion of the consequent that appears when the antecedent appears, and the confidence is calculated by counting the joint probability of the antecedent and the consequent.

[0038] Continuing with the above example, the first item can be: {Action = Login}, and the second item can be:

[0039] {Status=Success}, where Action and Login in the antecedent are an attribute and attribute value pair, and Status and Success in the succesor are an attribute and attribute value pair. Combining the antecedent and succesor results in an association rule of the form {Action=Login}→{Status=Success}, where →the antecedent is the antecedent and →the succesor is the succesor.

[0040] Optionally, calculating the support and confidence of each association rule according to a pre-built support function and confidence function may include:

[0041] In the initialization log data, traverse all initialization log entries and record the total number of initialization log entries;

[0042] Among all the initialized log entries, the number of log entries that meet the current association rule is counted as the first number of log entries;

[0043] Among all the initialized log entries, the number of all log entries containing the preceding attribute information pair of the current association rule is counted as the number of second log entries;

[0044] According to the formula: Calculate and obtain the support S corresponding to the current association rule;

[0045] According to the formula: The confidence C corresponding to the current association rule is calculated.

[0046] Specifically, satisfying the current association rule means that the antecedent and the consequent of the association rule appear in the log at the same time. In a specific example, if there are 1000 initialization log data, among which there are 400 data containing the fields Action=Login and Status=Success, the support for the current association rule is 400 / 1000=0.4; if there are 100 data with the antecedent field of Action=Login in the initialization log data, the confidence for the current association rule is 400 / (400+100)=0.8.

[0047] S130, encoding each association rule according to a preset encoding rule, and taking the support and confidence values ​​calculated under each association rule as real values, and combining them with each encoded rule to obtain multiple initial individuals in the genetic algorithm to form an initial population.

[0048] Encoding the association rules means converting the attributes and attribute values ​​in the antecedent and consequent into codes composed of letters and numbers according to the encoding rules. After combining the calculated support and confidence with the encoded association rules, each initial individual can be expressed as: [antecedent code, consequent code, support, confidence]. For example, [A1, S1, 0.6, 0.9] is the encoded association rule with a support of 0.6 and a confidence of 0.9.

[0049] Furthermore, encoding each association rule according to a preset encoding rule may include:

[0050] Each association rule is encoded according to the constructed attribute value set and the pre-set encoding rules.

[0051] Optionally, encoding each association rule according to the constructed attribute value set and a preset encoding rule may include:

[0052] Respectively identify the preceding attribute name and the succeeding attribute name in the current association rule, and extract the first characters of the preceding attribute name and the succeeding attribute name as the preceding first character code value and the succeeding first character code value;

[0053] Respectively identify the preceding attribute value and the succeeding attribute value in the current association rule, and respectively obtain the positions of the preceding attribute value and the succeeding attribute value in the attribute value set as the preceding last digit code value and the succeeding last digit code value;

[0054] Combine the first digit code value of the previous item and the last digit code value of the previous item to obtain the previous item code value of the current association rule;

[0055] Combine the first digit code value of the subsequent item and the last digit code value of the subsequent item to obtain the subsequent item code value of the current association rule;

[0056] Combine the preceding encoding value with the succeeding encoding value to obtain the encoded current association rule;

[0057] The position of each attribute value in the attribute value set is counted starting from 1.

[0058] Specifically, the specific execution logic of the encoding rule is as follows: Under an association rule, first extract the first character of the attribute in the antecedent and the consequent respectively, such as association rule 1: {Action = Login} →

[0059] {Status = Success}, then the first character of the previous attribute is A, and the first character of the next attribute is S, then A is the first code of the previous attribute, and S is the first code of the next attribute. Secondly, identify the positions of Login and Success in the attribute value set. If Login is the first in the attribute value set, the code is 1; if Success is the second in the attribute value set, the code is 2, then 1 is the last code of the previous attribute, and 2 is the last code of the next attribute.

[0060] The combination of the first digit code of the previous item and the last digit code of the previous item is A1, so A1 is the previous item.

[0061] {Action = Login}; the combination of the first digit code of the latter item and the last digit code of the latter item is S2, and S2 is the code of the latter item {Status = Success}.

[0062] {Status = Success} can be expressed as [A1, S2, a, b] after encoding, where a and b are the real values ​​of the support and confidence of the current association rule.

[0063] S140, performing multiple rounds of iterations on the initial population in the three methods of selection, crossover and mutation until convergence.

[0064] The initial population serves as the starting point of the genetic algorithm and provides a diverse initial solution space for the genetic algorithm. After the initial population is selected, crossed and mutated, new individuals can be introduced into the initial population to gradually optimize the set of association rules.

[0065] Furthermore, the initial population is iterated multiple times in three ways: selection, crossover, and mutation until convergence, which may include:

[0066] Calculate the fitness of each initial individual in the initial population according to the predefined fitness function, and determine the initial individual with the highest fitness;

[0067] Each initial individual in the initial population is used as a parent, and genes are exchanged between two parents according to the preset crossover probability and crossover method to obtain multiple offspring;

[0068] Perform mutation operation on each initial individual in the initial population according to the preset mutation probability and mutation mode to obtain new mutation individuals corresponding to each initial individual;

[0069] The initial individual with the highest fitness, all offspring, and each new mutant individual are used as a new population to enter the next round of iteration;

[0070] The fitness of each initial individual in the initial population is calculated according to a predefined fitness function, specifically including:

[0071] According to the formula, F = α·S + β·C, calculate the fitness F of each initial individual;

[0072] Wherein, α and β are preset weight coefficients, and α+β=1, S is the support of the association rule corresponding to the initial individual, and C is the confidence of the association rule corresponding to the initial individual.

[0073] The purpose of the selection operation is to select individuals with higher fitness from the current initial population, that is, to calculate the fitness value of each initial individual, and select the excellent individuals to enter the next iteration based on the roulette or tournament selection strategy. The fitness function is used to evaluate the quality of individuals, and the support and confidence can be weighted summed, that is, the formula: F = α·S + β·C. In a specific example, α=0.5, β=0.5, if individual 1: [A1, S1, 0.4, 0.8]; individual 2: [A2, S2, 0.15, 0.375]; individual 3: [A3, S1, 0.05, 0.5], then F1=0.5*0.4+0.5*0.8=0.6; F2=0.5*0.15+0.5*0.375=0.2625; F3=0.5*0.05+0.5*0.5=0.275, then individual 1 is the initial individual with the highest fitness.

[0074] The purpose of the crossover operation is to generate new offspring individuals by combining the genes of parent individuals and explore new solution spaces. First, determine the crossover probability, and select individuals from the population for crossover according to the crossover probability. The crossover methods can include: single-point crossover, multi-point crossover, and uniform crossover. In a specific example, parent 1: [A1, S1, 0.4, 0.8], parent 2: [A3, S1, 0.05, 0.5], if the single-point crossover method is selected for crossover, the offspring 1: [A1, S1, 0.05, 0.5], offspring 2: [A3, S1, 0.4, 0.8] are obtained. The generation of new offspring also represents the generation of new association rules.

[0075] The purpose of mutation operation is to introduce random changes, increase the diversity of the population, and prevent the algorithm from falling into the local optimal solution. Based on the pre-set mutation probability, individuals are selected from the population for mutation. The mutation methods may include: attribute mutation and parameter mutation. In a specific example, after the attribute mutation of individual 1: [A1, S1, 0.05, 0.5], [A2, S1, 0.05, 0.5] can be obtained. The generation of new mutated individuals represents the generation of new association rules.

[0076] S150, sorting all new individuals participating in the last round of iteration in descending order of fitness values, and extracting association rules corresponding to individuals with the highest fitness values ​​according to the sorting results, and outputting them to the user end as optimal association rules.

[0077] After multiple genetic operations, the association rules are sorted and displayed, the rules that are beneficial are retained through fitness evaluation, and finally the optimal association rules with high fitness are output.

[0078] The embodiment of the present invention utilizes the global search feature of the genetic algorithm to optimize the mining of log association rules. First, by encoding the support and confidence into real numbers and storing them in the gene sequence of the individual, the automatic optimization of the parameters is achieved through iteration and evaluation, avoiding the tedious process of manually setting the threshold, which not only improves the efficiency of rule mining, but also enhances the accuracy of the mining results. Secondly, the selection, crossover and mutation operations in the genetic algorithm increase the diversity of the rules, which helps the algorithm to search for the optimal solution in a wider solution space and improves the global search capability.

[0079] Embodiment 2

[0080] Figure 2 This is a flow chart of another method for mining association rules using a genetic algorithm provided in the second embodiment of the present invention. This embodiment is refined based on the above embodiment. Figure 2 As shown, the method specifically includes:

[0081] S210 , in response to an instruction for mining association rules initiated by the user terminal, original log data is acquired and initialized to obtain initialized log data.

[0082] S220 , respectively read attribute information pairs from each initialization log data, and perform deduplication processing on each attribute information pair to obtain deduplication attribute information pairs, wherein the attribute information pairs include attribute names and attribute values.

[0083] Each attribute information pair refers to a pair of mapping relationships consisting of an attribute and an attribute value. For example, a piece of data in the initialization log: {timestamp:2024-09-10T12:34:56Z,ip:192.168.0.1,user:12345,action:Login,status:Success}, {status:Success} is an attribute information pair. Identify the attribute information pairs in all initialization log data and remove duplicates to obtain multiple attribute information pairs without duplication.

[0084] Optionally, after reading the attribute information pairs in each initialization log data respectively, the following steps may also be included:

[0085] All attribute values ​​in each attribute information pair read are constructed into an attribute value set.

[0086] S230: extract all attribute names from each pair of deduplicated attribute information, cluster all attribute values ​​under the same attribute name, and obtain an attribute value set corresponding to each attribute name.

[0087] Specifically, attribute values ​​with the same attribute name are extracted, an attribute value set with the same attribute name is constructed according to each attribute name, and an index is constructed for each attribute value stored in the attribute value set to mark its arrangement position in the attribute value set.

[0088] S240, query the pre-built mapping relationship between the previous and next items, and obtain the relationship pairs of the previous and next item attribute names from all the extracted attribute names, wherein the attribute value of the next item attribute name depends on the attribute value of the previous item attribute name.

[0089] The mapping relationship between the preceding and following items specifies which attribute names are the preceding items and which attribute names are the following items. The preceding and following item attribute name relationship pair refers to a mapping relationship between the preceding item attribute name and the following item attribute name. For example, among all the attribute names, Action is the preceding item attribute name, and Status is the following item attribute name. Action and Status are a set of preceding and following item attribute name relationship pairs.

[0090] S250, combining multiple association rules based on the attribute value sets corresponding to the antecedent attribute name and the consequent attribute name in each antecedent attribute name relationship pair, and calculating the support and confidence of each association rule according to the pre-constructed support function and confidence function.

[0091] When Action is used as the antecedent attribute name, the attribute value is extracted from the attribute value set corresponding to Action, such as Login; when Status is used as the succesor attribute name, the attribute value is extracted from the attribute value set corresponding to Status, such as Success, and an association rule of {Action=Login}→{Status=Success} is obtained.

[0092] Optionally, multiple association rules are obtained by combining attribute value sets corresponding to the antecedent attribute name and the consequent attribute name in each antecedent attribute name relationship pair, which may include:

[0093] Extract the target antecedent attribute name and the target subsequent attribute name from the currently processed antecedent attribute name relationship pair;

[0094] Obtain a first attribute value set corresponding to the target antecedent attribute name, and a second attribute value set corresponding to the target consequent attribute name;

[0095] Combining the target antecedent attribute name with each first attribute value in the first attribute value set to obtain a plurality of antecedent attribute information pairs, and combining the target posterior attribute name with each second attribute value in the second attribute value set to obtain a plurality of posterior attribute information pairs;

[0096] Each preceding attribute information pair is combined with each succeeding attribute information pair to obtain an association rule that matches the preceding and succeeding attribute name relationship pair currently being processed.

[0097] Continuing with the above example, when Action is used as the target antecedent attribute name, the attribute value set corresponding to Action can be called the first attribute value set, and when Status is used as the target consequent attribute name, the attribute value set corresponding to Status can be called the second attribute value set, and the first attribute value set can be: {Login, Download, Upload}, and the second attribute value set can be: {Success, Failure}. Combine the target antecedent attribute name and the target consequent attribute name with the first attribute value set and the second attribute value set respectively to obtain the antecedent attribute name relationship pair {Action=Login}, {Action=Download}, {Action=Upload}, and the consequent attribute name relationship pair {Status=Success}, {Status=Failure}. Combine the antecedent attribute name relationship pair with the consequent attribute name relationship pair to obtain multiple association rules such as {Action=Login}→{Status=Success}.

[0098] S260, encoding each association rule according to a preset encoding rule, and taking the support and confidence values ​​calculated under each association rule as real values, and combining them with each encoded rule to obtain multiple initial individuals in the genetic algorithm to form an initial population.

[0099] S270, perform multiple rounds of iterations on the initial population using selection, crossover and mutation until convergence.

[0100] S280, sorting all new individuals participating in the last round of iteration in descending order of fitness values, and extracting the association rules corresponding to the individuals with the highest fitness values ​​according to the sorting results, and outputting them to the user end as the optimal association rules.

[0101] The technical solution of the embodiment of the present invention mainly explains the construction of association rules by refining the overall solution. Specifically, the association rules are composed of antecedents and consequents, and the combination relationship of the antecedents and consequents is determined by the antecedent-consequent mapping relationship. In general, the embodiment of the present invention encodes the association rules in all log data by identifying the attribute names and attribute values ​​in the log data. It is suitable for log data of different scales and complexities, and can process high-dimensional and unstructured data. At the same time, through the global optimization capability of the genetic algorithm, the generated association rules have higher support and confidence, thereby improving the quality of the rules.

[0102] Embodiment 3

[0103] Figure 3A schematic diagram of the structure of a device for mining association rules using a genetic algorithm provided in the third embodiment of the present invention. Figure 3 As shown, the device comprises:

[0104] The log initialization module 310 is used to obtain and initialize the original log data in response to the instruction of association rule mining initiated by the user end to obtain the initialized log data;

[0105] The association rule calculation module 320 is used to associate the initialization data according to the predefined association strategy to obtain multiple association rules, and calculate the support and confidence of each association rule according to the pre-built support function and confidence function;

[0106] The initial population construction module 330 is used to encode each association rule according to a preset encoding rule, and use the support and confidence values ​​calculated under each association rule as real values, and combine them with each encoded rule to obtain multiple initial individuals in the genetic algorithm to form an initial population;

[0107] The population iteration module 340 is used to perform multiple rounds of iterations on the initial population in three ways: selection, crossover and mutation, until convergence;

[0108] The association rule mining module 350 is used to sort all new individuals participating in the last round of iteration in descending order of fitness value, and extract the association rule corresponding to the individual with the highest fitness according to the sorting result, and output it to the user end as the optimal association rule.

[0109] In the embodiment of the present invention, the global search feature of the genetic algorithm is used to optimize the mining of log association rules. First, the support and confidence are encoded in real numbers and stored in the gene sequence of the individual. The automatic optimization of parameters is achieved through iteration and evaluation, avoiding the tedious process of manually setting thresholds, which not only improves the efficiency of rule mining, but also enhances the accuracy of mining results. Secondly, the selection, crossover and mutation operations in the genetic algorithm increase the diversity of rules, help the algorithm to search for the optimal solution in a wider solution space, and improve the global search capability.

[0110] Optionally, based on the above embodiments, the association rule calculation module 320 may include:

[0111] An attribute information pair acquisition unit is used to read attribute information pairs in each initialization log data, and perform deduplication processing on each attribute information pair to obtain deduplication attribute information pairs, wherein the attribute information pairs include attribute names and attribute values;

[0112] The attribute value set construction unit is used to extract all attribute names from each pair of deduplicated attribute information, cluster all attribute values ​​under the same attribute name, and obtain an attribute value set corresponding to each attribute name;

[0113] A preceding and following attribute name relationship pair obtaining unit is used to query the pre-built preceding and following attribute mapping relationship, and obtain the preceding and following attribute name relationship pairs from all the extracted attribute names, wherein the attribute value of the following attribute name depends on the attribute value of the preceding attribute name;

[0114] The association rule combination unit is used to combine and obtain multiple association rules according to the attribute value sets corresponding to the preceding attribute name and the following attribute name in each preceding and following attribute name relationship pair.

[0115] Optionally, based on the above embodiments, the association rule combination unit specifically includes:

[0116] Extract the target antecedent attribute name and the target subsequent attribute name from the currently processed antecedent attribute name relationship pair;

[0117] Obtain a first attribute value set corresponding to the target antecedent attribute name, and a second attribute value set corresponding to the target consequent attribute name;

[0118] Combining the target antecedent attribute name with each first attribute value in the first attribute value set to obtain a plurality of antecedent attribute information pairs, and combining the target posterior attribute name with each second attribute value in the second attribute value set to obtain a plurality of posterior attribute information pairs;

[0119] Each preceding attribute information pair is combined with each succeeding attribute information pair to obtain an association rule that matches the preceding and succeeding attribute name relationship pair currently being processed.

[0120] Optionally, based on the above embodiments, the association rule calculation module 320 may further include:

[0121] An initialization log entry unit, used to traverse all initialization log entries in the initialization log data and record the total number of initialization log entries;

[0122] A first log entry number counting unit, used to count the number of log entries satisfying the current association rule among all the initialized log entries as the first log entry number;

[0123] A second log entry number counting unit is used to count the number of log entries that contain the preceding attribute information pair of the current association rule in all the initialized log entries as the second log entry number;

[0124] The support calculation unit is used to calculate the support according to the formula: Calculate and obtain the support S corresponding to the current association rule;

[0125] The confidence calculation unit is used to calculate the confidence according to the formula: The confidence C corresponding to the current association rule is calculated.

[0126] Optionally, based on the above embodiments, it further includes an association rule encoding unit, which is used to construct all attribute values ​​in each attribute information pair read as an attribute value set after reading the attribute information pairs in each initialization log data;

[0127] The initial population construction module 330 may also include:

[0128] Each association rule is encoded according to the constructed attribute value set and the pre-set encoding rules.

[0129] Optionally, based on the above embodiments, the initial population construction module 330 may also be used to:

[0130] Respectively identify the preceding attribute name and the succeeding attribute name in the current association rule, and extract the first characters of the preceding attribute name and the succeeding attribute name as the preceding first character code value and the succeeding first character code value;

[0131] Respectively identify the preceding attribute value and the succeeding attribute value in the current association rule, and respectively obtain the positions of the preceding attribute value and the succeeding attribute value in the attribute value set as the preceding last digit code value and the succeeding last digit code value;

[0132] Combine the first digit code value of the previous item and the last digit code value of the previous item to obtain the previous item code value of the current association rule;

[0133] Combine the first digit code value of the subsequent item and the last digit code value of the subsequent item to obtain the subsequent item code value of the current association rule;

[0134] Combine the preceding encoding value with the succeeding encoding value to obtain the encoded current association rule;

[0135] The position of each attribute value in the attribute value set is counted starting from 1.

[0136] Optionally, based on the above embodiments, the population iteration module 340 may further include:

[0137] An individual selection unit, used to calculate the fitness of each initial individual in the initial population according to a predefined fitness function, and determine the initial individual with the highest fitness;

[0138] The individual crossover unit is used to take each initial individual in the initial population as a parent, and exchange genes between two parents according to a preset crossover probability and crossover method to obtain multiple offspring;

[0139] The individual mutation unit is used to perform a mutation operation on each initial individual in the initial population according to a preset mutation probability and mutation mode to obtain a new mutation individual corresponding to each initial individual;

[0140] A new population generation unit is used to take the initial individual with the highest fitness, all offspring, and each new mutant individual as a new population to enter the next round of iteration;

[0141] The fitness of each initial individual in the initial population is calculated according to a predefined fitness function, specifically including:

[0142] According to the formula, F = α·S + β·C, calculate the fitness F of each initial individual;

[0143] Wherein, α and β are preset weight coefficients, and α+β=1, S is the support of the association rule corresponding to the initial individual, and C is the confidence of the association rule corresponding to the initial individual.

[0144] The device for performing association rule mining using a genetic algorithm provided by an embodiment of the present invention can execute a method for performing association rule mining using a genetic algorithm provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0145] Embodiment 4

[0146] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0147] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0148] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0149] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a method for mining association rules using a genetic algorithm.

[0150] That is: in response to the instruction of association rule mining initiated by the user end, the original log data is obtained and initialized to obtain the initialized log data;

[0151] The initialization data is associated according to a predefined association strategy to obtain multiple association rules, and the support and confidence of each association rule are calculated according to the pre-built support function and confidence function;

[0152] Encode each association rule according to a preset encoding rule, and use the support and confidence values ​​calculated under each association rule as real values, and combine them with each encoded rule to obtain multiple initial individuals in the genetic algorithm to form an initial population;

[0153] The initial population is iterated multiple times in three ways: selection, crossover, and mutation until convergence;

[0154] All new individuals participating in the last round of iteration are sorted in descending order according to the fitness value, and the association rules corresponding to the individuals with the highest fitness are extracted according to the sorting results and output to the user end as the optimal association rules.

[0155] In some embodiments, a method for mining association rules using a genetic algorithm may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for mining association rules using a genetic algorithm described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute a method for mining association rules using a genetic algorithm in any other appropriate manner (e.g., by means of firmware).

[0156] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0157] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0158] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0160] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0161] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0162] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0163] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for mining association rules using a genetic algorithm, characterized in that: include: In response to an instruction of association rule mining initiated by a user terminal, original log data is acquired and initialized to obtain initialized log data; The initialization data is associated according to a predefined association strategy to obtain multiple association rules, and the support and confidence of each association rule are calculated according to the pre-built support function and confidence function; Encode each association rule according to a preset encoding rule, and use the support and confidence values ​​calculated under each association rule as real values, and combine them with each encoded rule to obtain multiple initial individuals in the genetic algorithm to form an initial population; The initial population is iterated multiple times in three ways: selection, crossover, and mutation until convergence; All new individuals participating in the last round of iteration are sorted in descending order according to the fitness value, and the association rules corresponding to the individuals with the highest fitness are extracted according to the sorting results and output to the user end as the optimal association rules.

2. The method according to claim 1, characterized in that The initialization data is associated according to the predefined association strategy to obtain multiple association rules, including: Reading attribute information pairs in each initialization log data respectively, and performing deduplication processing on each attribute information pair to obtain deduplication attribute information pairs, wherein the attribute information pairs include attribute names and attribute values; Extract all attribute names from each pair of deduplicated attribute information, cluster all attribute values ​​under the same attribute name, and obtain attribute value sets corresponding to each attribute name; Query the pre-built mapping relationship between the previous and next items, and obtain the relationship pairs of the previous and next item attribute names from all the extracted attribute names, where the attribute value of the next item attribute name depends on the attribute value of the previous item attribute name; According to the attribute value sets corresponding to the antecedent attribute name and the consequent attribute name in each antecedent attribute name relationship pair, a plurality of association rules are obtained by combination.

3. The method according to claim 2, characterized in that According to the attribute value sets corresponding to the antecedent attribute name and the consequent attribute name in each antecedent attribute name relationship pair, multiple association rules are obtained by combination, including: Extract the target antecedent attribute name and the target subsequent attribute name from the currently processed antecedent attribute name relationship pair; Obtain a first attribute value set corresponding to the target antecedent attribute name, and a second attribute value set corresponding to the target consequent attribute name; Combining the target antecedent attribute name with each first attribute value in the first attribute value set to obtain a plurality of antecedent attribute information pairs, and combining the target posterior attribute name with each second attribute value in the second attribute value set to obtain a plurality of posterior attribute information pairs; Each preceding attribute information pair is combined with each succeeding attribute information pair to obtain an association rule that matches the preceding and succeeding attribute name relationship pair currently being processed.

4. The method according to claim 2 or 3, characterized in that: The support and confidence of each association rule are calculated according to the pre-built support function and confidence function, including: In the initialization log data, traverse all initialization log entries and record the total number of initialization log entries; Among all the initialized log entries, the number of log entries that meet the current association rule is counted as the first number of log entries; Among all the initialized log entries, the number of all log entries containing the preceding attribute information pair of the current association rule is counted as the number of second log entries; According to the formula: Calculate and obtain the support S corresponding to the current association rule; According to the formula: The confidence C corresponding to the current association rule is calculated.

5. The method according to claim 2, characterized in that After reading the attribute information pairs in each initialization log data, it also includes: Construct all attribute values ​​in each attribute information pair read into an attribute value set; Encode each association rule according to the pre-set encoding rules, including: Each association rule is encoded according to the constructed attribute value set and the pre-set encoding rules.

6. The method according to claim 5, characterized in that Each association rule is encoded according to the constructed attribute value set and the pre-set encoding rules, including: Respectively identify the preceding attribute name and the succeeding attribute name in the current association rule, and extract the first characters of the preceding attribute name and the succeeding attribute name as the preceding first character code value and the succeeding first character code value; Respectively identify the preceding attribute value and the succeeding attribute value in the current association rule, and respectively obtain the positions of the preceding attribute value and the succeeding attribute value in the attribute value set as the preceding last digit code value and the succeeding last digit code value; Combine the first digit code value of the previous item and the last digit code value of the previous item to obtain the previous item code value of the current association rule; Combine the first digit code value of the subsequent item and the last digit code value of the subsequent item to obtain the subsequent item code value of the current association rule; Combine the preceding encoding value with the succeeding encoding value to obtain the encoded current association rule; The position of each attribute value in the attribute value set is counted starting from 1.

7. The method according to claim 1, characterized in that The initial population is iterated in multiple rounds by selection, crossover and mutation until convergence, including: Calculate the fitness of each initial individual in the initial population according to the predefined fitness function, and determine the initial individual with the highest fitness; Each initial individual in the initial population is used as a parent, and genes are exchanged between two parents according to the preset crossover probability and crossover method to obtain multiple offspring; Perform mutation operation on each initial individual in the initial population according to the preset mutation probability and mutation mode to obtain new mutation individuals corresponding to each initial individual; The initial individual with the highest fitness, all offspring, and each new mutant individual are used as a new population to enter the next round of iteration; The fitness of each initial individual in the initial population is calculated according to a predefined fitness function, specifically including: According to the formula, F = α·S + β·C, calculate the fitness F of each initial individual; Wherein, α and β are preset weight coefficients, and α+β=1, S is the support of the association rule corresponding to the initial individual, and C is the confidence of the association rule corresponding to the initial individual.

8. A device for mining association rules using a genetic algorithm, characterized in that: include: The log initialization module is used to respond to the instruction of association rule mining initiated by the user end, obtain the original log data and initialize it to obtain the initialized log data; An association rule calculation module is used to associate the initialization data according to a predefined association strategy to obtain multiple association rules, and calculate the support and confidence of each association rule according to a pre-built support function and confidence function; The initial population construction module is used to encode each association rule according to a preset encoding rule, and use the support and confidence values ​​calculated under each association rule as real values, and combine them with each encoded rule to obtain multiple initial individuals in the genetic algorithm to form an initial population; The population iteration module is used to perform multiple rounds of iterations on the initial population in three ways: selection, crossover, and mutation until convergence; The association rule mining module is used to sort all new individuals participating in the last round of iteration in descending order of fitness value, and extract the association rules corresponding to the individuals with the highest fitness according to the sorting results, and output them to the user end as the optimal association rules.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a method for mining association rules using a genetic algorithm according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a method for mining association rules using a genetic algorithm according to any one of claims 1 to 7 when executed.

Images