Data mining system and method based on artificial intelligence

Abstract

The invention relates to the technical field of data mining analysis, in particular to a data mining system and method based on artificial intelligence, and the method comprises the following steps: carrying out the association rule scanning of a sparse transaction data set, calculating a frequent item set with the support degree exceeding a support degree threshold value as a mode embryo, and outputting a mode embryo set E; the method comprises the following steps: constructing an inter-dimension correlation channel through a neuromorphic calculation unit, enabling an embryo to diffuse and grow in an adjacent dimension space, outputting a mature mode set M, and distributing an initial confidence coefficient weight for each mature mode; and inputting the mature mode set M into a time sliding window controller, detecting a data distribution offset, adjusting a mode confidence coefficient weight according to the distribution offset, and outputting a stable mode set S. According to the method, high-precision screening of mode adaptability in a dynamic environment is achieved, interference of abnormal data on mining results is reduced, and good reliability is shown in high-dynamic scenes such as behavior monitoring, financial anti-fraud and medical diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data mining analysis, and in particular to a data mining system and method based on artificial intelligence. BACKGROUND

[0002] A tender service platform accumulates a large amount of heterogeneous transaction data about project types, bidding enterprises, qualification responses, bid price structures, and bid evaluation results during operation. These data have significant characteristics such as high dimensionality, sparse structure, and strong time sequence dynamics.

[0003] At present, the tender service platform urgently needs to deeply mine the above-mentioned massive transaction data based on artificial intelligence technology to identify enterprise bidding behavior patterns, monitor changes in competitive strategies, discover potential bid-rigging risks, or perceive market preference trends in advance, and provide decision support for platform operators. However, existing data mining methods are mainly based on traditional frequent item set analysis or static rule modeling, and have the following technical bottlenecks:

[0004] Firstly, in the face of highly sparse transaction structures in the tendering scene (for example, a small number of tenders participated by an enterprise, and a low repetition rate of the same type of project), traditional horizontal scanning type frequent item set algorithms perform poorly in support statistical efficiency and effective pattern discovery ability, and it is difficult to extract basic patterns with business correlation value.

[0005] Secondly, existing pattern expansion methods generally rely on fixed item set combinations or artificial rule paths, and cannot adaptively identify potential dynamic evolution relationships between different project dimensions (such as industry classification, qualification level, and regional distribution), resulting in fragmented mining results and poor generalization ability.

[0006] In addition, tender data has obvious periodic fluctuations and strategic adjustment characteristics, and enterprise bidding patterns may shift with policy changes, market saturation, or competitive structure. Traditional data mining models generally ignore the "concept drift" phenomenon of transaction distribution over time, lack dynamic monitoring mechanisms for pattern stability and timeliness, and are prone to misjudgment or strategy failure. SUMMARY

[0007] The present application provides a data mining system and method based on artificial intelligence, which is suitable for the tender service platform scene, has sparse transaction modeling capability, dynamic association learning mechanism, and time sequence stability screening function, and realizes a complete pattern evolution closed loop from basic behavior extraction, semantic structure expansion to high credible pattern screening.

[0008] A data mining method based on artificial intelligence, comprising the following steps:

[0009] S1, perform association rule scanning on the sparse transaction dataset, calculate the frequent itemsets whose support exceeds the support threshold as pattern embryos, and output the pattern embryo set E;

[0010] S2 takes the set of embryos E output by S1 as input, constructs inter-dimensional correlation channels through the neuromorphic computing unit, enables the embryos to spread and grow in adjacent dimensional spaces, outputs a set of mature patterns M, and assigns an initial confidence weight to each mature pattern.

[0011] S3 inputs the mature mode set M into the time sliding window controller, detects the data distribution offset, adjusts the mode confidence weights according to the distribution offset, and outputs the stable mode set S.

[0012] Optionally, the support calculation includes converting the sparse transaction dataset into a bitmap matrix, where rows represent transactions, columns represent items, non-zero elements are marked as 1, performing a bitwise AND operation on the bitmap matrix by column, and calculating the support for each frequent itemset.

[0013] Optionally, S1 further includes setting a support threshold, which is used to filter frequent itemsets. When the support of a frequent itemset exceeds the support threshold, the current frequent itemset is marked as a pattern embryo.

[0014] Optionally, S1 aggregates all frequent itemsets that meet the conditions and outputs a set of pattern embryos E, wherein the support threshold is adjusted as the data sparsity changes.

[0015] Optionally, S2 specifically includes:

[0016] S21. For each frequent itemset in the pattern embryo set E, calculate its dimensional similarity with other frequent itemsets in the pattern embryo set, and generate a dimensional similarity matrix D.

[0017] S22, based on the neuromorphic computing unit, synaptic connections are established between dimensions that satisfy the condition sim(Xi,Xj) is greater than θ. sim(Xi,Xj) represents the dimensional similarity value between frequent itemsets Xi and frequent itemsets Xj, which is calculated using cosine similarity. θ is the dimensional similarity threshold.

[0018] S23, using synaptic connections as the path, the energy value of the model embryo is diffused to adjacent dimensions according to the connection weight. When the accumulated energy of the target dimension exceeds the preset energy threshold, a new extended itemset is generated.

[0019] S24, perform confidence verification on the extended itemset, retain itemsets with a support improvement rate greater than a set percentage, and output the mature pattern set M.

[0020] Optionally, the synaptic connections are dynamically adjusted by the neuromorphic computing unit according to the pulse timing-dependent plasticity rule to form an itemset synaptic topology graph including connection weights. The changes in synaptic strength depend on the order of frequent itemset activation, thus realizing a learnable dimensional association structure.

[0021] Optionally, the pulse timing-dependent plasticity rule is a rule for updating synaptic strength based on time sequence, specifically including:

[0022] S231, Each itemset records its activation timestamp when it is triggered or diffused;

[0023] S232, for any two similar frequent itemsets X i X j Calculate its activation time difference Δt = t j -t i :

[0024] If Δt>0: it means X i It is activated first, strengthening the connection.

[0025] If Δt < 0: it means X j It is activated first, weakening the connection strength;

[0026] Positive reinforcement of connection strength indicates that downstream itemsets are more easily activated by energy diffusion;

[0027] Reducing connection strength in the opposite direction limits the probability of low-relevance paths being activated.

[0028] S233, maintain the itemset synaptic topology graph, record the synaptic weights between each pair of connected itemsets, and adjust the connection weights in each iteration based on the latest activation sequence.

[0029] Optionally, the time-sliding window controller is used to control the sliding process of the historical window and the detection window, extract the support distribution of mature patterns in different time periods, and monitor and judge the consistency of the weight sequence to achieve data distribution offset detection and pattern stability verification, specifically including:

[0030] S31, set a fixed-length historical window and a detection window, calculate the support distribution of the mature mode set M in the two windows respectively, and output the historical window distribution vector and the detection window distribution vector respectively;

[0031] S32, calculate the Wasserstein distance between the historical window distribution vector and the detection window distribution vector as the distribution offset; this method does not directly compare probability values, but measures the intensity of change by simulating the quality transfer process between distributions, thereby obtaining the data distribution offset that reflects the concept drift or the degree of expression fluctuation.

[0032] S33, update the initial confidence weights of the mature mode according to the distribution offset and the exponential decay rule, and output the updated weighted confidence.

[0033] S34 performs a sliding window consistency check on the mature pattern set after the weight reset and confidence update, retains the patterns whose weight fluctuation is less than the fluctuation threshold within multiple consecutive windows, and outputs a stable pattern set S.

[0034] Optionally, the weight reset confidence level updated in S33 specifically includes:

[0035] Based on the calculated distribution offset, the current confidence weight of each mature model is adjusted using an exponential decay strategy. The larger the offset, the faster the weight is updated; the smaller the offset, the more the weight tends to remain the same. At the same time, a support decay factor is introduced to adjust the decay rate, ensuring that high-support models have strong anti-decay capabilities.

[0036] An artificial intelligence-based data mining system, used to implement the aforementioned artificial intelligence-based data mining method, includes the following modules:

[0037] Transaction preprocessing module: used to perform association rule scanning on sparse transaction datasets, extract frequent itemsets with support exceeding the support threshold as pattern embryos and output a set of pattern embryos;

[0038] Neuromorphic extension module: Based on the pattern embryo, construct the interdimensional association channel, so that it spreads in the adjacent dimensional space to generate mature patterns, output the mature pattern set M, and assign an initial confidence weight to each mature pattern;

[0039] Time-based sliding window control module: Used to monitor the mature pattern set through a sliding window, calculate the support distribution offset under different time windows, and adjust the initial confidence weight of the patterns according to the offset;

[0040] Stable mode output module: Used to output the final stable mode set with stable confidence weight changes and high expression consistency.

[0041] The beneficial effects of this invention are:

[0042] This invention introduces bitmap matrix compression and vertical projection counting mechanisms in the initial pattern extraction stage, transforming high-dimensional sparse transaction datasets into low-redundancy, bitwise-operable sparse matrices. This effectively solves the computational bottleneck encountered by traditional horizontal scanning when processing million-dimensional transactions. By replacing set intersection judgment with bit-level compression encoding and Boolean AND operations, the complexity of frequent itemset extraction is reduced from quadratic to a linear approximation. Especially in data environments with sparsity exceeding 95%, the accompanying adaptive support threshold dynamic adjustment mechanism can extract a sufficient number of business-meaning pattern embryos while ensuring accuracy, providing a high-quality starting point for subsequent neuromorphic expansion.

[0043] This invention introduces impulse temporal dependent plasticity (STDP) into the synaptic connection construction of frequent pattern networks during pattern expansion, enabling dynamic learning of associations between itemsets. This avoids the low adaptability problem caused by manually setting expansion paths based on heuristic rules in traditional methods. Simultaneously, a cross-dimensional propagation path is established through a biomimetic energy diffusion equation, guiding patterns to naturally expand in high-dimensional space based on synaptic weights and dimensional distance, effectively overcoming the structural breakage problem caused by the dimensional island effect in traditional frequent pattern mining. At the level of expanded itemsets, a screening mechanism based on support improvement rate is introduced to ensure that only combinations with significant semantic growth potential are identified as mature patterns, thereby constructing a set of highly expressive mature patterns with strong semantics and continuous structure.

[0044] This invention constructs a time-sliding window controller in the final stage of pattern evolution, integrating dual-window monitoring and Wasserstein distance calculation to dynamically quantify the performance differences of mature patterns in historical and current distributions. An exponential decay function maps the distribution shift to a confidence weight adjustment ratio, and a decay factor is introduced to protect high-value patterns from excessive weakening, enhancing the model's tolerance to noise and abrupt changes. Furthermore, combined with sliding window consistency detection, unstable patterns with large weight fluctuations are eliminated, retaining only stable patterns whose variance within multiple windows is below a set threshold. This achieves high-precision screening of patterns adaptability in dynamic environments, reduces the interference of abnormal data on mining results, and demonstrates good reliability in highly dynamic scenarios such as behavioral monitoring, financial anti-fraud, and medical diagnosis. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Fig. 1This is a schematic diagram of the data mining method according to an embodiment of the present invention;

[0047] Fig. 2 This is a logic diagram of the time sliding window controller according to an embodiment of the present invention. Detailed Implementation

[0048] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. For some well-known technologies, those skilled in the art may also use other alternative methods to implement the invention. Moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0049] like Figs. 1-2 As shown, an artificial intelligence-based data mining method includes the following steps:

[0050] S1 performs association rule scanning on the sparse transaction dataset, calculates frequent itemsets with support exceeding the support threshold as pattern embryos, and outputs the pattern embryo set E.

[0051] S1 specifically includes:

[0052] S11, Transaction Matrix Compression: Converting a sparse transaction dataset into a bitmap matrix B∈{0,1} n×m ,in:

[0053] Line i∈{1,2,…,n} represents the i-th transaction;

[0054] The column j∈{1,2,…,m} represents the j-th item;

[0055] B ij =1 indicates that the i-th transaction contains the j-th item, otherwise it is 0, n is the number of transactions, and m is the total number of items.

[0056] This bitmap matrix employs bit-level compressed encoding for storage to reduce the memory overhead of high-dimensional sparse data. For example, recording a million-dimensional transaction as a bit string of length m, such as 0100110, effectively improves storage efficiency. This is demonstrated in the Amazon product review dataset.

[0057] S12, Vertical Projection Counting: Perform a vertical bitwise AND operation on the column-wise columns of the compressed bitmap matrix to calculate any candidate selection set. Support level:

[0058]

[0059] in, This indicates that itemset X is the universal set. A subset of (i.e., the set of all possible items), Let X represent the complete set of all unique itemsets in the transaction dataset, and let X represent a specific candidate itemset consisting of several items used for frequent itemset mining. sup(X) represents the support of itemset X, i.e., the number of transactions that simultaneously contain all items x∈X. ix =1 indicates transaction T i The intersection of the included term x is achieved through bitwise AND operations in actual calculations: Among them, B x The term x represents the bitmap column vector across all transactions, ∧ represents the bitwise AND operation, and nonzero represents the bitwise AND operation. count (·) indicates the number of 1s in the result.

[0060] The above method employs an efficient vertical projection approach, combining bitmap matrices with bit operations to quickly calculate the support of candidate options.

[0061] Bitmap representation of transaction data: First, the original transaction data is transformed into a two-dimensional 0-1 matrix, where each row represents a transaction and each column represents an item. If an item is contained in a transaction, the corresponding position is 1; otherwise, it is 0. In this way, each column forms a vector showing the occurrence of a particular item in all transactions.

[0062] Support calculation is transformed into column vector intersection: For any candidate itemset, to calculate its support, i.e., how many transactions simultaneously contain all items in the itemset, this process can be transformed into: performing a bitwise AND operation on the column vectors corresponding to these items to obtain a new column vector representing which transactions simultaneously contain these items.

[0063] Statistical support: For the result vector after a bitwise AND operation, count the number of positions where the value is 1. This represents the number of transactions supporting the itemset, which is the support of the itemset. Because it uses bitwise operations (i.e., bitwise AND), this method is more efficient than traditional methods such as scanning each transaction and performing set intersection operations. The computational complexity is significantly reduced, especially when dealing with high-dimensional sparse data.

[0064] S13, Support threshold screening: Global support threshold δ∈[0.01,0.04]. When the support of an itemset X satisfies: sup(X)≥δ·n, the itemset is determined to be a frequent itemset and marked as a pattern embryo.

[0065] Where n is the total number of transactions and δ is the support threshold. This rule can be dynamically adjusted. If the sparsity of the detected data exceeds 90%, δ will be automatically lowered appropriately to avoid missing embryo patterns.

[0066] When the data sparsity is ≤90%, the value is 0.04; since the items co-occur frequently, it is suitable to set a high threshold to control noise.

[0067] When the data sparsity is 90% to 95%, the value is 0.02; when the co-occurrence of items is moderate, the threshold is appropriately lowered to retain meaningful low-frequency combinations.

[0068] When the data sparsity is >95%, the value is 0.01; for extremely sparse data, the threshold needs to be significantly reduced to ensure the generation of model embryos.

[0069] The sparser the data, the fewer the non-zero elements, and the lower the co-occurrence probability between items. Therefore, in highly sparse scenarios, if the support threshold is set too high, most candidate itemsets will be filtered out, ultimately failing to extract effective frequent pattern embryos. Unlike complex adaptive functions, the "partition setting" strategy is simple and intuitive, suitable for practical engineering deployments, especially in edge AI devices or systems with limited memory resources. It can achieve threshold switching without relying on complex calculations, improving system robustness and real-time performance.

[0070] In this invention, the set of pattern embryos E serves as the input source for subsequent neuromorphic computation channel expansion. Insufficient embryo numbers will directly lead to the absence or excessive homogeneity of mature patterns, affecting the effectiveness and adaptability of the final stable pattern set S. Setting a lower δ ensures that potentially meaningful initial patterns can still be extracted from highly sparse data.

[0071] S14, Embryo Set Generation: Summarize all frequent itemsets that meet the support requirement to generate a pattern embryo set E: This set serves as subsequent input.

[0072] S2 takes the set of embryos E output by S1 as input, constructs inter-dimensional correlation channels through neuromorphic computing units, enables the embryos to spread and grow in adjacent dimensional spaces, outputs a set of mature patterns M, and assigns an initial confidence weight to each mature pattern.

[0073] S2 specifically includes the following steps:

[0074] S21, Dimensional similarity calculation:

[0075] Input: Set of pattern embryos E;

[0076] Output: Dimensional similarity matrix D.

[0077] For the model embryo set E = {X1, X2, ..., X...} i Each frequent itemset X in} i Calculate the cosine similarity with each of the itemsets in the entire set, and generate a dimensional similarity matrix D, where:

[0078]

[0079] Among them, X i ·X jRepresents itemset X i With X j The vector dot product is ||X i || represents the itemset vector X i The vector norm of ||X| is defined as the square root of the sum of squares of the itemset. j ||Itemset vector X j The vector norm, D ij ∈[0,1] represents X i With X j dimensional similarity, cos(θ) ij ) represents itemset X i With X j The cosine of the angle between two itemsets measures the similarity between them in a dimensional space. ij Represents itemset X i With X j In multidimensional space, the smaller the angle, the higher the similarity.

[0080] S22, Connection Channel Construction:

[0081] Input: Dimensional similarity matrix D;

[0082] Output: Associated channel topology T.

[0083] Synaptic connections are established between itemsets that satisfy similarity conditions using the impulse temporal dependence plasticity rule (STDP) of neuromorphic computational units. If D ij >θ, then in itemset X i With X j Synaptic connections are established between the dimensions, where θ is the dimensional similarity threshold, set to 0.7. A cosine similarity exceeding 0.7 between two itemsets indicates they share a large number of identical items (i.e., strong co-occurrence), suggesting a higher likelihood of potential semantic associations or behavioral patterns. The weight update rule is as follows:

[0084]

[0085] Where Δw represents the change in synaptic weight, Δt represents the trigger time difference, and A + A - τ represents the positive or negative learning rate constant. + ,τ - This represents the time constant.

[0086] The constructed synaptic connection topology is denoted as T = (V, E) T ), where the node set V is the item set set, and the edge set E T This indicates the connection relationships and weights.

[0087] In S22, based on the dimensional similarity matrix output from S21, item sets in the pattern embryo set are compared pairwise, and item set pairs with similarity exceeding a preset threshold are selected. These item set pairs are considered to be highly correlated in the semantic space or transaction dimension. To simulate the learning connection mechanism between biological neurons, the synaptic plasticity principle from neuromorphic computing, namely the impulse temporal dependent plasticity rule (STDP), is introduced. This mechanism allows the system to dynamically adjust the connection weights between item set pairs based on their relative "triggering time difference." Specifically, when the similarity between two item sets exceeds the threshold, a synaptic connection is established between them, and the connection strength is calculated using the STDP rule based on their activation time difference. This ultimately forms a topological graph where nodes are item sets, edges are synaptic connections, and edge weights represent their correlation strength. This synaptic topology will serve as the path basis for the next step of cross-dimensional diffusion.

[0088] The implementation mechanism of the Impulse Timing Dependency Plasticity Rule (STDP) is as follows:

[0089] STDP is a rule based on the time sequence of synaptic strength updates, simulating the effect of the activation order of two neurons on the connection strength between them in a biological nervous system.

[0090] If the predecessor itemset X i The activation occurs earlier than the successor itemset X. j , represents X i Possibly for X j The activation of these elements produces a "causal effect," thus strengthening the connections between them;

[0091] If X j Activation earlier than X i This suggests that such activation order has no biological reinforcing value and may even represent a false association, weakening the connection.

[0092] Rule implementation logic:

[0093] 1. Activation Time Recording: Each itemset records its activation timestamp when it is triggered or spreads;

[0094] 2. Time difference calculation: For any two similar itemsets X i X j Calculate its activation time difference Δt = t j -t i ;

[0095] 3. Decision-making based on weight changes:

[0096] If Δt>0: it means X i It is activated first, which strengthens the connection (indicating that its activation is pioneering).

[0097] If Δt < 0: it means X j It is activated first, weakening the connection strength (considered non-causal or interference).

[0098] 4. Weight update effect:

[0099] Positive reinforcement connectivity means that downstream itemsets are more easily activated by energy diffusion;

[0100] Reverse weakening of connections limits the likelihood of low-relevance paths being activated.

[0101] 5. Synaptic structure renewal:

[0102] Maintain an itemset synaptic topology graph to record the synaptic weights between each pair of connected itemsets;

[0103] Each iteration adjusts these edge weights based on the latest activation sequence, enabling dynamic learning and evolution.

[0104] Introducing the STDP model can replace the traditional fixed threshold screening mechanism, so that the connection between itemsets is no longer based on static rules, but evolves dynamically with the actual activation process; it supports the generation of self-organizing synaptic networks, avoids manually preset association paths, and improves the plasticity and adaptability of the pattern expansion process. Compared with the manually set rule itemset combination method, STDP learning shows higher efficiency and recall quality in real transaction data. This mechanism also provides a reasonable and convergent structural path for subsequent energy diffusion and pattern maturation, avoiding the uncontrolled generalization of the diffusion graph.

[0105] S23, transdimensional diffusion of the embryo:

[0106] Input: Associated channel topology T, pattern embryo set E;

[0107] Output: The set of extended itemsets, C.

[0108] Using the synaptic connection topology T as the diffusion path, the energy of the original model embryo is diffused to the associated dimension through an energy propagation mechanism. The energy diffusion equation is defined as follows:

[0109] Among them, E j This represents the cumulative energy of node j. w represents the set of adjacent nodes connected to node j. ij d represents the synaptic weight between nodes i and j. ij Represents itemset X i With X j In dimensional space, λ represents the energy diffusion attenuation coefficient.

[0110] When the energy E of a certain target itemset dimension j When the energy exceeds the preset energy threshold η, a new extended itemset Y is generated. jAnd add it to the extended set C. The value η = 0.1, meaning it only applies when the cumulative energy E in the target dimension is... j A value ≥0.1 is considered sufficient for that dimension to receive adequate neural activation, thus generating the extended itemset Y. j In energy diffusion, the initial energy E of the source embryo... i It is usually normalized to 1.0, thus diffusing to other dimensions of E. j Most of them are small values ​​between 0 and 1. η = 0.1 is an empirical critical point. <0.05 is easy to trigger noise diffusion and generate meaningless itemsets. >0.15 is difficult to activate diffusion pathways and easy to lead to expansion failure. 0.1 is an empirical balance point that takes into account both activation rate and accuracy.

[0111] The energy diffusion model draws on the fundamental principles of weighted propagation mechanisms and physical diffusion processes in neural networks. Its design aims to construct a controllable, gradual, and decaying pattern expansion process. A synaptic connection exists between each pair of connected itemsets, and the weight w of this connection... ij This represents the strength of their semantic or structural association. During propagation, the energy of the source itemsets is weighted according to this weight, ensuring that highly associated paths receive more energy than weakly associated paths. The activation of a target itemset does not come from a single path, but rather from the "resultant force" of multiple activated itemsets. Therefore, this model uses a set of adjacent nodes. The energy value of the target itemset is calculated by summing the total contributions, which aligns with the cumulative effect logic of network diffusion. To avoid disordered energy diffusion in the dimensional space, leading to the generation of a large number of noisy itemsets, the equation introduces a variable 'd' depending on the itemset distance. ij Increased exponential decay factor Its function is to reduce the energy received by itemsets that are farther apart, suppress unrelated expansion across long distances, and simulate the decay of biological synaptic connections with spatial distance. The decay coefficient λ provides adjustability to the diffusion range; a larger value indicates that the system allows energy to be transmitted further, while a smaller value limits expansion to local high-density regions. This mechanism supports flexible adjustment in data with different sparsity and types.

[0112] S24, Model Maturity Screening:

[0113] Input: Extended itemset set C, pattern embryo set E;

[0114] Output: M sets of mature patterns.

[0115] For each itemset Y in the extended itemset set C j Calculate its support improvement rate and compare it with its source embryo itemset X. i Compare:

[0116] Δsup represents the increase in support, that is, the increase in support for a certain itemset during the expansion process. When this increase rate exceeds the set support increase rate threshold of 0.2 (i.e., the support increase exceeds 20%), X is considered to have increased support. i ∪Y j For effective mature models, include them in the mature model set M:

[0117]

[0118] Where Δt is the number of diffusion iterations or the diffusion time window.

[0119] After the mature pattern set M is generated, an initial confidence weight w0 is assigned to each pattern, which can be set to a fixed value of 1.0.

[0120] S3 inputs the mature mode set M into the time sliding window controller, detects the data distribution offset, adjusts the mode confidence weights according to the distribution offset, and outputs the stable mode set S.

[0121] S3 specifically includes the following steps:

[0122] S31, Dual-Window Distributed Monitoring: Two time windows are set for parallel distributed monitoring: one is a fixed-length historical window W. h One is used to capture long-term trends; the other is a detection window W with a sliding length. d This is used to capture recent changes. The support distribution of each pattern in the mature pattern set M is calculated in two windows, and the corresponding distribution vector is output:

[0123]

[0124] in, Representing pattern X i Support levels within the historical window, Representing pattern X i Support within the detection window, P h ,P d This indicates the distribution of the pattern set across the two windows.

[0125] S32, Dynamic Calculation of Offset: Calculates the degree of offset between two support distributions, using the Wasserstein distance metric under a one-dimensional distribution, defined as:

[0126]

[0127] In practical calculations, the cumulative distribution function F is used. h ,F d Expressed as:

[0128]

[0129] Where x and y represent the real coordinates corresponding to the support values, respectively from the pattern support distributions in the historical window and the detection window, and Δd represents the minimum transformation cost between the two distributions in the historical window and the detection window, i.e., the distribution offset, Γ(P h ,P d Let ) denote the joint distribution space, γ denote the joint probability distribution that maps or "transfers" one distribution to another, and γ(x,y) represent the probability quality of transporting from historical support x to detection support y, which is a term in the joint distribution. The inverse cumulative distribution function representing the distribution of the history and the detection window; P represents the historical window support distribution. h The inverse cumulative distribution function represents the support value corresponding to a certain percentile u in the distribution. P represents the support distribution of the detection window. d The inverse cumulative distribution function, meaning and The same but from the current data, u is the cumulative probability value in the interval [0,1], used to read the corresponding support position from the cumulative distribution.

[0130] S33, based on the calculated distribution offset Δd, the confidence weights of the model are updated exponentially: w new =w old ·e -δ·Δd The attenuation factor δ is adaptively adjusted with the model maturity and is defined as follows: Among them, w old The current confidence weights of the pattern are represented by w0 and w1. new The adjusted weights are represented by δ0, which is the initial decay coefficient, set to 2.0. A value that is too small will make the distribution shift insensitive to weight changes; a value that is too large may cause even normal fluctuations to trigger a sharp drop in weights. In tasks such as behavior detection, financial fraud, and real-time recommendation, a value around 2.0 can balance response speed and pattern retention rate, avoiding the false decay of high-support patterns. sup(X) represents the support of the current pattern. max This represents the highest support among all patterns. This mechanism effectively suppresses excessive weighting of high-value patterns while accelerating the elimination of ineffective or temporarily active patterns.

[0131] S34, perform stability checks on all updated patterns and verify consistency within a sliding time window. Define a sliding window sequence with g time windows and calculate the variance of the weight fluctuation for each pattern within this sequence:

[0132] w iLet μ represent the confidence weight value of a pattern in the i-th sliding window, μ represent the average weight of that pattern over g consecutive sliding windows, and σ represent the confidence weight value of that pattern. 2 This represents the variance of the weighted fluctuations of the pattern across g windows.

[0133] When the weight fluctuation of a certain pattern is lower than the set fluctuation threshold y over a consecutive g windows. p When this happens, it is included in the final stable mode set S: S = {X} i ∈M∣Var(w i ) <y p}, Var(w i The ) represents the variance of the pattern confidence weights within the sliding window sequence. This mechanism ensures that the output patterns have long-term consistency and can adapt to dynamic data environments under the background of distribution evolution. p The value is 0.1. Since the confidence weight is generally normalized between 0 and 1, a fluctuation threshold of 0.1 means that fluctuations within a window of about ±10% are allowed, which is sufficient to accommodate natural jitter and exclude severely unstable patterns.

[0134] An artificial intelligence-based data mining system for implementing the above-mentioned data mining methods includes the following modules:

[0135] Transaction preprocessing module: used to perform association rule scanning on sparse transaction datasets, extract frequent itemsets with support exceeding the support threshold as pattern embryos and output a set of pattern embryos;

[0136] Neuromorphic extension module: Based on the pattern embryo, construct the interdimensional association channel, so that it spreads in the adjacent dimensional space to generate mature patterns, output the mature pattern set M, and assign an initial confidence weight to each mature pattern;

[0137] Time-based sliding window control module: Used to monitor the mature pattern set through a sliding window, calculate the support distribution offset under different time windows, and adjust the initial confidence weight of the patterns according to the offset;

[0138] Stable mode output module: Used to output the final stable mode set with stable confidence weight changes and high expression consistency.

[0139] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0140] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An artificial intelligence-based data mining method, characterized by, The method comprises the following steps: S1, performing an association rule scan on the sparse transaction dataset to calculate frequent item sets with support exceeding a support threshold as pattern embryos, and outputting a pattern embryo set E; S2, taking the pattern embryo set E output by S1 as input, constructing a dimension correlation channel by a neuromorphic computing unit, and enabling the embryos to spread and grow in adjacent dimension spaces to output a mature pattern set M, and assigning an initial confidence weight to each mature pattern; S3, inputting the mature pattern set M into a time sliding window controller, detecting a data distribution offset, adjusting the pattern confidence weight according to the distribution offset, and outputting a stable pattern set S. 2.The data mining method based on artificial intelligence according to claim 1, characterized in that, The calculation of the support comprises converting the sparse transaction dataset into a bitmap matrix, wherein rows represent transactions and columns represent items, non-zero elements are marked as 1, and the bitmap matrix is subjected to a bitwise AND operation to calculate the support of each frequent item set. 3.The data mining method based on artificial intelligence of claim 1, wherein, S1 further comprises setting a support threshold, which is used to screen frequent item sets, and when the support of a frequent item set exceeds the support threshold, the current frequent item set is marked as a pattern embryo. 4.The data mining method based on artificial intelligence according to claim 3, characterized in that, S1 aggregates all frequent item sets meeting the condition to output the pattern embryo set E, and the support threshold is adjusted according to the change of data sparsity.

5. The data mining method based on artificial intelligence according to claim 1, characterized in that, S2 specifically comprises: S21, for each frequent item set in the pattern embryo set E, calculating the dimension similarity between the frequent item set and other dimension frequent item sets in the pattern embryo set to generate a dimension similarity matrix D; S22, based on the neuromorphic computing unit, establishing a synaptic connection between dimensions meeting the condition that sim(Xi,Xj) is greater than θ, wherein sim(Xi,Xj) represents the dimension similarity value between the frequent item set Xi and the frequent item set Xj, θ is a dimension similarity threshold, and the dimension similarity is calculated by using cosine similarity; S23, taking the synaptic connection as a path, diffusing the energy value of the pattern embryo to adjacent dimensions according to the connection weight, and generating a new extended item set when the cumulative energy of the target dimension exceeds a preset energy threshold; S24, performing confidence verification on the extended item set, retaining item sets with a support improvement rate greater than a set percentage, and outputting a mature pattern set M.

6. The data mining method based on artificial intelligence according to claim 5, characterized in that, The synaptic connection is dynamically adjusted by the neuromorphic computing unit according to the spike-timing-dependent plasticity rule to form an item set synaptic topology graph including connection weights, and the strength of the synapse changes depending on the order of activation of the frequent item sets, thereby realizing a learnable dimension correlation structure.

7. The data mining method based on artificial intelligence according to claim 6, characterized in that, The spike-timing-dependent plasticity rule is a rule for updating the synaptic strength based on the time sequence, and specifically comprises: S231, recording the activation time stamp of each item set when it is triggered or diffused; S232, for any two similar frequent itemsets X i , X j , compute their activation time difference Δt = t j - t i : If Δt > 0: indicates X i First activated, enhanced connection strength; If Δt < 0: indicates X j First activated, weaken the connection strength; A forward enhanced connection strength indicates that downstream item sets are more likely to be activated by energy diffusion; A reverse weakened connection strength limits the probability of activation of low-correlation paths; S233, maintaining an item set synaptic topology graph to record the synaptic weight between each pair of connected item sets, and adjusting the connection weight according to the latest activation time sequence in each iteration. 8.The data mining method based on artificial intelligence according to claim 1, characterized in that, The time sliding window controller is used to control the sliding process of the historical window and the detection window, extract the support distribution of the mature patterns in different time periods, and monitor and judge the weight sequence for consistency, so as to realize data distribution offset detection and pattern stability verification, and specifically comprises: S31, setting a fixed length of the historical window and the detection window, calculating the support distribution of the mature pattern set M in the two windows respectively, and outputting the historical window distribution vector and the detection window distribution vector respectively; S32, calculating the Wasserstein distance between the historical window distribution vector and the detection window distribution vector as the distribution offset; S33, updating the initial confidence weight of the mature pattern according to the exponential decay rule of the distribution offset, and outputting the updated weight confidence; S34, sliding window consistency detection is performed on the mature pattern set after updating the weight confidence, the pattern with a weight fluctuation less than a fluctuation threshold in continuous multiple windows is retained, and a stable pattern set S is output. 9.The data mining method based on artificial intelligence of claim 8, wherein, The S33 outputs the updated weight confidence, which specifically comprises: Based on the calculated distribution offset, the current confidence weight of each mature pattern is adjusted, the adjustment mode adopts an exponential decay strategy, the greater the offset, the faster the weight update, the smaller the offset, the weight tends to remain, and at the same time, a support decay factor is introduced to adjust the decay rate, so as to ensure that the high support mode has strong anti-decay ability.

10. An artificial intelligence based data mining system for implementing an artificial intelligence based data mining method as claimed in any one of claims 1 to 9, characterized by Comprise the following modules: Transaction preprocessing module: used for association rule scanning on sparse transaction data set, extracting frequent item set with support exceeding support threshold as pattern embryo and outputting pattern embryo set; Neuromorphic expansion module: based on the pattern embryo, an association channel between dimensions is constructed to diffuse and generate mature patterns in adjacent dimensional space, output a mature pattern set M, and assign an initial confidence weight to each mature pattern; Time sliding window control module: used for sliding window monitoring of the mature pattern set, calculating the support distribution offset in different time windows, and adjusting the initial confidence weight of the pattern according to the offset; Stable pattern output module: used for outputting the final stable pattern set with stable confidence weight change and high expression consistency.

Images