Real-time anomaly detection and intelligent early warning system and method based on artificial intelligence

Abstract

The invention discloses a real-time anomaly detection and intelligent early warning system and method based on artificial intelligence, and relates to the technical field of artificial intelligence and data security, and the method comprises the steps: obtaining historical data assets, and constructing a tensor potential energy function; guiding the generated detector particles to migrate through the tensor gradient of the tensor potential energy function; negative selection is carried out on the migrated detector particles, an initial detector is generated, cloning is carried out on the initial detector, a clone detector is generated, and the detector is migrated to a potential abnormal region based on the tensor gradient; when the clone detector stably resides in the potential abnormal region, recording an evolution trajectory of the clone detector, and constructing a trend memory unit; according to a trend memory unit, generating a trend detector and migrating to a detector particle insufficient coverage area based on a tensor gradient; and performing response matching on the to-be-detected data assets, the clone detector and the trend detector to complete anomaly detection early warning. And high detection precision and early warning response rate are kept in a complex data environment.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and data security technology, specifically to a real-time anomaly detection and intelligent early warning system and method based on artificial intelligence. Background Technology

[0002] In the context of the rapid development of data-driven businesses, various information systems have accumulated a large amount of structured and unstructured data assets, covering multiple dimensions such as field content, scheduling records, and tags. These data assets support key aspects such as intelligent analysis, predictive modeling, and decision execution, and have become core resources supporting organizational operations and business optimization. However, with the continuous growth of data volume and the increasing complexity of processing chains, data anomalies are becoming increasingly prominent, often manifesting as disordered data structures, abnormal processing paths, and sovereign compliance risks. If these anomalies are not identified and handled in a timely manner, they can easily lead to system performance degradation, business logic errors, and even information security incidents.

[0003] Existing anomaly detection methods generally rely on static rules or fixed threshold models, making it difficult to adapt to the dynamic changes in comprehensive factors such as heterogeneous data structures, multi-source data lineage, and sovereignty attributes. They suffer from significant problems such as weak generalization ability, slow response, and insufficient coverage. Furthermore, most traditional methods employ point-to-point detection mechanisms, lacking modeling of data processing evolution trends, resulting in low accuracy in identifying potential risk areas and making it difficult to achieve real-time, accurate, and scalable anomaly identification. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is the weak ability to identify potential abnormal regions in existing multi-source heterogeneous data, insufficient detector coverage, and delayed early warning response.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a real-time anomaly detection and intelligent early warning method based on artificial intelligence, comprising:

[0007] Acquire historical data assets and construct a tensor potential function;

[0008] The tensor gradient of the tensor potential function guides the migration of the generated detector particles.

[0009] The negative selection filters the migrated detector particles to generate an initial detector. The initial detector is cloned to generate a cloned detector, which is then migrated to the potential anomalous region based on the tensor gradient.

[0010] When the clone detector stably resides in the potentially anomalous region, the evolution trajectory of the clone detector is recorded and a trend memory unit is constructed; based on the trend memory unit, a trend detector is generated and migrated to the region with insufficient detector particle coverage based on the tensor gradient;

[0011] The data assets to be detected are matched with the responses of the clone detector and the trend detector to complete the anomaly detection and early warning.

[0012] As a preferred embodiment of the real-time anomaly detection and intelligent early warning method based on artificial intelligence described in this invention, the historical data assets include structured and unstructured fields, task scheduling records, and data tag information.

[0013] Feature vectors are extracted from structured and unstructured fields and standardized into historical feature vectors; historical data processing paths are reconstructed based on task scheduling records to construct data lineage chains; sensitive attributes and compliance identifiers of historical data assets are parsed from data tag information and mapped to generate sovereign information tensors.

[0014] Using the recording order of historical data assets as an index, multiple sample points are generated sequentially to form a sample point set; each sample point is formed by concatenating historical feature vectors, data lineage chains, and sovereignty information tensors; each sample point is compressed into a vector of fixed dimension d using MLP, and the feature space R is obtained based on the vector. d The location of the sample points in the data.

[0015] As a preferred embodiment of the real-time anomaly detection and intelligent early warning method based on artificial intelligence described in this invention, the tensor potential function is constructed as follows: the local density anomaly is calculated using the local outlier method through the historical feature vector of each sample point; the structural entropy is calculated as the path complexity through the data lineage chain of each sample point; and the sovereign risk intensity is generated by weighted fusion of the sovereign information tensor attributes of each sample point.

[0016] The local density anomaly degree, path complexity, and sovereign risk intensity of each sample point are combined according to a preset weighting factor to form a single-point anomaly potential value. The distance between any position in the feature space and the sample point is calculated using a Gaussian kernel function to construct a tensor potential energy function. The potential energy function values ​​of all sample points are sorted in order to form a set of potential energy function values. The sovereign information tensor attributes include data sensitivity, supervision strictness, and cross-border flow level.

[0017] As a preferred embodiment of the real-time anomaly detection and intelligent early warning method based on artificial intelligence described in this invention, the generation of the detector particles includes: when the potential energy function value is greater than the dynamic quantile threshold θ min Uniform sampling of N in the region initθ is used as the initial position of the detector particle, and the detector particle is generated; where θ min N is the q% quantile of the potential energy function set for all sample points; init Indicates the number of particles generated by the detector;

[0018] The detector particle migration includes calculating the gradient of the tensor potential function based on the current position of the detector particle, multiplying the gradient by a damping coefficient, adding it to the current particle position, and then superimposing the result with a mean of 0 and a variance of σ. 2 The Gaussian distribution perturbation term is used to obtain the updated position; when the difference in the potential function value of any detector particle in two adjacent iterations is less than the average fluctuation amplitude of the potential function value, convergence is determined and the iteration is stopped.

[0019] As a preferred embodiment of the real-time anomaly detection and intelligent early warning method based on artificial intelligence described in this invention, the generation of the initial detector includes: selecting normal sample points from the sample point set to form a normal sample point set; calculating the mean center point and feature space standard deviation of the normal sample point set, and setting the normal region judgment radius to k times the feature space standard deviation; calculating the Euclidean distance between each detector particle and the mean center point, and retaining the detector particle as the initial detector when the Euclidean distance is greater than the normal region judgment radius; otherwise, discarding the detector particle.

[0020] The generated clone detector includes generating N from each initial detector. c There are 1 clone detector, and the position of each clone detector is determined by adding a sum to the position of the initial detector, which follows a mean of 0 and a variance of . The cloned Gaussian perturbation term is used; the gradient of the tensor potential function is calculated based on the current position of the clone detector, and migration is performed based on the cloned Gaussian perturbation term; when the tensor potential function value corresponding to the position of the clone detector is ≥ the g% quantile of the potential function set, the clone detector is considered to be in a potential anomaly region; where N c This indicates the number of clone detectors generated.

[0021] In a preferred embodiment of the artificial intelligence-based real-time anomaly detection and intelligent early warning method described in this invention: when the clone detector is in the potential anomaly region for L consecutive iterations, it is considered that the clone detector is stably stationary; the position iteration trajectory of the clone detector is recorded, and the trajectory length does not exceed T. max , among which, T max This represents the upper limit of the trajectory length; the trend memory unit is constructed by summing the position iteration trajectories of all stably resident clone detectors.

[0022] As a preferred embodiment of the real-time anomaly detection and intelligent early warning method based on artificial intelligence described in this invention, the following steps are included: generating a trend detector involves extracting trend hotspot regions based on the position iteration trajectory of a stably residing clone detector; randomly sampling and generating an initial position for the trend detector within the trend hotspot region; updating the position based on the gradient of the tensor potential function to obtain the final trend detector position; constructing a spatial partition map based on the position; sorting the area of ​​each spatial partition map in ascending order; defining the top p% of the areas as insufficient coverage regions; if the distance between the trend detector position and any clone detector within an insufficient coverage region is less than a preset spatial radius threshold, then the trend detector is determined to fall into an insufficient coverage region; extracting trend hotspot regions involves extracting the trajectory points of the position iteration trajectory of the stably residing clone detector and performing density clustering; constructing a spherical region with a radius equal to a multiple of the cluster variance v at the center of the cluster center point; and combining all spherical regions as the trend hotspot region.

[0023] The anomaly detection and early warning system includes calculating the response of the data asset to be detected to each clone detector and each trend detector using the inverse ratio of Euclidean distance, and summing all the calculated response values ​​as the total response intensity; when the total response intensity exceeds the response threshold, it is determined to be an anomaly and an intelligent early warning is triggered; the intelligent early warning system includes recording abnormal events, pushing alarm information, and triggering data asset control in conjunction with the system.

[0024] As a preferred embodiment of the real-time anomaly detection and intelligent early warning system based on artificial intelligence described in this invention, the tensor potential energy function module acquires historical data assets and constructs a tensor potential energy function.

[0025] The detector module guides the migration of generated detector particles through the tensor gradient of the tensor potential energy function; negative selection filters the migrated detector particles to generate an initial detector; cloning is performed on the initial detector to generate a clone detector, which then migrates to the potential anomaly region based on the tensor gradient.

[0026] The trend memory module records the evolution trajectory of the clone detector and constructs trend memory units when the clone detector stably resides in a potentially anomalous region.

[0027] The coverage module generates a trend detector based on the trend memory unit and migrates to the region where the detector particles are not covered based on the tensor gradient.

[0028] The anomaly detection module matches the data assets to be detected with the responses of the clone detector and trend detector to complete the anomaly detection and early warning.

[0029] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement a method for real-time anomaly detection and intelligent early warning based on artificial intelligence.

[0030] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a real-time anomaly detection and intelligent early warning method based on artificial intelligence.

[0031] The beneficial effects of this invention are as follows: The real-time anomaly detection and intelligent early warning method based on artificial intelligence provided by this invention achieves accurate characterization of potential anomaly regions in historical data by constructing a tensor potential energy function that integrates local density anomaly degree, data lineage structure complexity, and sovereign risk intensity; it uses tensor gradients to guide detector particle migration, enabling the detector to adaptively focus on high anomaly potential regions, and enhances the coverage and spatial flexibility of anomaly detection through negative selection strategies and dynamic cloning mechanisms; after the cloned detectors have stabilized, their evolution trajectories are recorded, and trend memory units are constructed to extract anomaly hotspots, further supplementing particle coverage blind spots, ensuring high detection accuracy and early warning response rate in complex data environments. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 The flowchart illustrates the overall process of the real-time anomaly detection and intelligent early warning method based on artificial intelligence provided in the first embodiment of the present invention. Detailed Implementation

[0034] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0035] Example 1, referring to Figure 1 As one embodiment of the present invention, a real-time anomaly detection and intelligent early warning method based on artificial intelligence is provided, comprising:

[0036] S1: Acquire historical data assets and construct a tensor potential function; guide the migration of generated detector particles through the tensor gradient of the tensor potential function.

[0037] Historical data assets include: structured data fields: numerical and categorical data fields; unstructured data fields: text, image, audio and video fields, etc.; task scheduling records: task scheduling logs of data processing flow (including task ID, execution time, and task type); data tag information: fields such as sensitivity level, regulatory compliance requirements, and cross-border circulation level.

[0038] Structured and unstructured fields are preprocessed separately (e.g., text embedding, image feature extraction) and then unified into a fixed-dimensional historical feature vector f. i Based on the task ID and execution timestamp recorded in the task scheduling log, the task execution order is restored to generate a clear data lineage structure L. line,i ; Clearly defined sensitive attributes, regulatory stringency, and cross-border circulation levels are extracted from data tag information and unified into a three-dimensional numerical sovereignty information tensor L. son,i .

[0039] Based on the recording order of historical data assets, and taking each record as a sample point, a corresponding single sample structure is generated:

[0040] x i =[f i ,L line,i ,L son,i ]

[0041] The above splicing structure x i The input is uniformly fed into a pre-trained multilayer perceptron (MLP), which maps it to a vector of uniform dimension.

[0042] x i ′ =MLP(x i ),x i ′ ∈R d

[0043] The position vector X′ of each sample point in the feature space = {x i ′}.

[0044] Where, x i f represents the concatenated feature vector of the i-th sample point; i L represents the historical feature vector of the i-th sample point; line,i L represents the data lineage vector of the i-th sample point; son,i The vector represents the sovereign information tensor of the i-th sample point; MLP(·) represents the Multi-Layer Perceptron model; xi ′ Let d represent the embedding vector of the i-th sample point after MLP mapping; d represents the dimension of the uniform embedding vector; X′={x i '} represents the set of position vectors of all sample points in the feature space.

[0045] Each sample point x i ′、f i L line,i L son,i Processing procedure: Calculate the three potential energies respectively: local density anomaly S(f) i ): The local density outlier of each sample point is calculated using the Local Outlier Factor (LOF): S(f i ) = LOF(f i ).

[0046] Data lineage chain structure entropy Ey(L) line,i ): Treating the data lineage chain as a sequence of node types, the Shannon entropy of the type is calculated to represent the path complexity.

[0047]

[0048] Sovereign risk intensity Rk(L) son,i The following factors are considered using a weighted approach: sensitivity, regulatory stringency, and cross-border eligibility.

[0049] Rk(L son,i )=w1·Sy+w2·RS+w3·CBL

[0050] The above three indicators are combined to form the single-point anomaly potential value for each sample point:

[0051] Q i =α1S(f i )+α2Ey(L line,i )+α3Rk(L son,i )

[0052] Wherein, S(f i ) represents the local density anomaly of the i-th sample point; LOF(·) represents the local outlier factor calculation function; Ey(L) represents the entropy of the data lineage structure. line,i ) represents the entropy of the data lineage structure of the i-th sample point. Let p(c) represent the universal set of all kinship node types, and let logp(c) represent the probability of a node of type c appearing in the kinship chain. Let c represent the index; Rk(L son,i) represents the sovereign risk intensity of the i-th sample point. w1, w2, w3 represent the weight coefficients of sensitivity, regulatory strictness, and cross-border level, respectively. Sy represents the data sensitivity score. RS represents the regulatory strictness score. CBL represents the cross-border level score. Q i α1, α2, and α3 represent the comprehensive anomaly potential value of the i-th sample point; α1, α2, and α3 represent the fusion weight coefficients of the three indicators, which are used to adjust their influence ratio in the potential value.

[0053] Using a Gaussian kernel, the potential value of a single-point anomaly is diffused to any point in the feature space. Tensor potential energy function:

[0054]

[0055] The Gaussian kernel function is defined as follows:

[0056]

[0057] Where Φ(x) represents the potential energy value of the potential energy function at position x in the characteristic space; x represents any value to be evaluated. (Originally obtained); K(x,x′) i ) represents the Gaussian kernel similarity between position x and the i-th sample point.

[0058] The potential energy function values ​​of all sample points are sorted in order to form a set of potential energy function values.

[0059] Generating the detector particles includes, when the potential energy function value is greater than the dynamic quantile threshold θ min Uniform sampling of N in the region init θ is a feature space location point used as the initial position of the detector particle, and points are used to generate detector particles; where θ min N is the q% quantile of the potential energy function set for all sample points; init This indicates the number of particles generated by the detector.

[0060] Gradient based on tensor potential function The formula for calculating the particle update path and the specific update rule is expressed as follows:

[0061]

[0062] Represents the position vector of the j-th detector particle in the feature space at the (t+1)-th iteration; β represents the damping coefficient; This represents the tensor potential function Φ(x) at position. gradient at ε; tThis represents the random perturbation term added in the t-th iteration; This indicates that the mean is 0 and the variance is σ. 2 Gaussian distribution; σ 2 The variance of the Gaussian perturbation is represented.

[0063] The detector particle migration is complete when the difference in potential function values ​​between two adjacent iterations for any detector particle is less than or equal to the average fluctuation range of the potential energy value.

[0064] In processing structured and unstructured data assets, this method extracts and embeds features from numerical and categorical fields, as well as unstructured fields such as text and images, into vectorized forms. These features are then combined with a three-dimensional sovereign information tensor constructed from the data processing path structure reconstructed from task scheduling logs and sensitive attribute fields to create a unified-dimensional feature-stitched sample. Compared to traditional methods that only use structured fields or partial metadata to construct samples, this method comprehensively integrates static data features, dynamic task behavior chains, and sovereign regulatory dimensions, thereby improving the completeness and discriminability of the sample representation.

[0065] In terms of anomaly potential modeling, this approach calculates local density anomalies by introducing a Local Outlier Factor (LOF), assesses the complexity of task paths using path structure entropy, and employs a weighted method to fuse sensitivity, regulatory stringency, and cross-border level to generate a single-point anomaly potential value for each sample point. Subsequently, a Gaussian kernel is used to diffuse the single-point potential across the entire feature space, constructing a continuously differentiable tensor potential function. Compared to methods based on fixed thresholds or category labels for discrete judgment, this scheme can accurately characterize data distribution trends and local perturbation risks in a high-dimensional continuous space, thereby improving the accuracy and interpretability of potential anomaly region modeling.

[0066] In the detector generation and migration stages, the gradient information of the tensor potential function is calculated to guide the force field migration of detector particles sampled in high potential regions. Stable convergence of particle positions is achieved under a dynamic perturbation and damping balance mechanism. This scheme not only avoids the generalization deficiency caused by the random distribution of initial detectors but also adaptively adjusts the initial position based on the spatial distribution patterns of historical samples, making subsequent detection coverage more concentrated in potential anomaly regions, thereby effectively improving initial detection capability and subsequent response efficiency.

[0067] S2: Negative selection filters the migrated detector particles, generates an initial detector, clones the initial detector, generates a clone detector, and migrates it to the potential anomaly region based on the tensor gradient.

[0068] In historical data assets, a normal sample set is constructed by screening record samples that conform to normal behavioral characteristics. These characteristics include: data tag information indicating compliance; stable data lineage path structure with no historical abnormal processing records; and sovereign information tensor exhibiting low sensitivity, high regulatory intensity, and no involvement in cross-border circulation.

[0069] Sample points that meet the above conditions are defined as normal sample points, forming a set of normal sample points.

[0070] Calculate the set of normal sample points The feature space mean center point μ and standard deviation vector σ; set the normal region judgment radius r = k·||σ||, where k is a preset multiplier coefficient; for each migrated detector particle z j Calculate its Euclidean distance from the center point μ. j -μ||; when the distance is / / z j If -μ||>r, retain the particle as the initial detector; otherwise, discard the particle. After completing the above filtering, the initial detector set is formed.

[0071] This represents the set normal region radius multiplier; r = k·σ / / represents the normal region radius used for judgment; z j / / z represents the position vector of the j-th detector particle; j -μ / / represents the Euclidean distance between the detector particle and the center point; This represents the initial set of detectors retained after negative selection filtering.

[0072] For each initial detector Generate m clone detectors 2. The initial position of each clone detector is generated by superimposing the original detector position with the clone perturbation term:

[0073] in

[0074] Perform tensor gradient-guided clonal migration for each clonal detector Perform the following update iterations:

[0075]

[0076] Where β represents the damping coefficient in the gradient direction; Represents the gradient of the tensor potential function; ∈ (j,t) For each round of superimposed clonal Gaussian perturbation.

[0077] Among them, z iThis represents the position vector of the i-th initial detector; m represents the number of detectors to be cloned for each initial detector; Indicates the initial position of the j-th clone detector (0th iteration); ∈ (j) This represents the perturbation term for clone migration location; This indicates that the disturbance term follows a pattern with a mean of 0 and a variance of . Gaussian distribution; This represents the variance (clonal perturbation strength) of the clonal perturbation term; This indicates the position of the j-th clone detector of the i-th initial detector in the t-th iteration; This indicates its position in the next iteration; β represents the damping coefficient of the gradient of the tensor potential function; Represents the gradient of the tensor potential function; ∈ (j,t) T represents the cloning perturbation term in round t; clone Indicates the maximum number of iterations for clone migration; ∈ conv This represents the convergence threshold for the clone migration process. This indicates the magnitude of potential energy changes over two consecutive rounds; This indicates the final stable position of the cloning detector (the position after convergence).

[0078] Calculate the potential energy change for two consecutive rounds after each iteration:

[0079]

[0080] If the convergence criterion is met or the maximum number of steps is reached, the clone detector is considered to have completed the migration.

[0081] Let the converged position be denoted as .

[0082] The potential energy values ​​of all sample points are set. After sorting, obtain the potential energy value Φ of the g% quantile. thresh ;

[0083] like If the clone detector is found to be in a potentially abnormal region, it will be retained; otherwise, it will be discarded.

[0084] in, Φ represents the set of potential energy values ​​of all sample points in the feature space. thresh This represents the g% quantile (i.e., the dynamic threshold) of the set; The potential function value represents the final location of the clone detector; g represents the set threshold for determining the abnormal region.

[0085] In terms of initial detector screening, by constructing a "normal sample point set" under structured conditions and jointly utilizing three types of information—data labels, lineage paths, and sovereignty tensors—data records without abnormal historical behavior can be accurately extracted across multiple sources. Furthermore, based on this set, the center point and standard deviation vector are calculated in the feature space. By setting a Euclidean distance limit (defining the decision radius as a multiple of the standard deviation), negative selection filtering of detector particles is achieved. Compared to existing methods based on rule hard coding or random migration strategies, this approach effectively constrains the spatial distribution of the initial detectors, preventing subsequent detection from focusing on obviously normal regions and reducing false alarm rates and resource waste.

[0086] In the clone detector generation and migration stage, multiple clones with Gaussian perturbations are generated for each initial detector, overcoming the limited particle coverage capability of single-point detectors. Simultaneously, a tensor potential function gradient is introduced during the cloning process to guide migration, and random perturbations are superimposed to form a migration trajectory. The stable residence position of the clone detector is determined by a convergence criterion (such as the potential energy change amplitude being less than a set threshold). Compared to traditional static models or fixed window movement strategies, this approach can dynamically adapt to complex distribution patterns in the feature space, improving the exploration efficiency and flexibility of the detection area.

[0087] In the potential anomaly region identification stage, the tensor potential function value corresponding to the convergence position is compared with a preset dynamic quantile threshold to ensure that the clone detector ultimately focuses on relatively high-risk regions. This method upgrades the static threshold determination method to a mechanism based on dynamic adjustment of data distribution, and combines continuous convergence behavior to control the risk of false migration, significantly enhancing the effectiveness of the detector and the discriminative power of the convergence region, providing a high-quality initial foundation for subsequent trend tracking and response matching.

[0088] S3: When the clone detector is stably residing in the potential anomalous region, record the evolution trajectory of the clone detector and construct a trend memory unit; based on the trend memory unit, generate a trend detector and migrate to the region with insufficient detector particle coverage based on the tensor gradient.

[0089] If the clone detector remains in the potential anomalous region for L consecutive iterations, it is considered to be stably residing as a clone detector; the position iteration trajectory of the clone detector is recorded, and the trajectory length does not exceed T. max , among which, T max This indicates the upper limit of the trajectory length; the trajectories of all stably residing clone detectors are uniformly aggregated to construct the trend memory unit.

[0090] The evolutionary trajectories of all converged clone detectors are extracted from the trend memory unit. Trajectory points throughout the migration process of each trajectory are collected to form a trajectory point set. A density-based clustering algorithm (such as DBSCAN) is performed on this trajectory point set to extract high-density clustering regions of all trajectory points, and the variance between the center point of each cluster and the corresponding cluster is calculated.

[0091] A spherical region is constructed with the center point of each cluster as the center and a radius of v times the cluster variance. The union of all spherical regions is defined as the trend hotspot region, which is used to reflect the migration focus direction of historical anomalous behavior.

[0092] Random sampling is performed within trend hotspot regions to generate initial positions for several trend detectors. For each trend detector, a gradient-driven mechanism using the tensor potential function is employed to perform a finite-step migration process, updating the trend detector's position vector. The migration and update process is as follows:

[0093]

[0094] Where, x t x represents the position vector of the trend detector at the t-th iteration; t+1 This represents the position vector of the trend detector after the (t+1)th iteration; The tensor potential function in x t The gradient at the point; γ represents the damping coefficient during the migration process, which controls the gradient guidance strength; t This represents the Gaussian perturbation term in the t-th iteration; express t Follows a mean of 0 and a variance of The normal distribution; This represents the intensity of the disturbance during the t-th migration.

[0095] The migration process for each trend detector is set with a maximum number of steps T. trend The process terminates when the change in the tensor potential energy function value is less than the average convergence fluctuation threshold.

[0096] Finally, it is determined whether the convergence position of the trend detector is located in a feature space region not covered by the current detector particle set. If the condition is met, the trend detector is included in the trend detector set for subsequent supplementation of anomaly detection coverage.

[0097] Let set D be the convergence location of all existing, stably resident clone detectors. det A Voronoi diagram is constructed based on this set; the area of ​​each Voronoi region is calculated, and all region area values ​​are sorted in ascending order; the regions with the highest area ranking (p%) are defined as "under-coverage regions," and their corresponding detection...

[0098] Whether the potential detector enters the radius threshold of the insufficiently covered region; x′ trend This represents the position vector after the trend detector converges; p% represents the percentage of area sorting selected for under-covered regions.

[0099] Most existing anomaly detection technologies rely on static features at the current moment or one-time sampling to construct a detector set, lacking utilization of historical detector behavior paths and making it difficult to identify the evolutionary trend of anomalous activity. The process described above is based on the stable residence behavior of clone detectors. When a detector is located in a potential anomalous region L times, its complete evolutionary trajectory is recorded, and a trend memory unit is constructed. By aggregating multiple trajectory points and performing density clustering analysis, the system can automatically extract the evolutionary focus direction of multiple historical anomaly detection paths, forming trend hotspot regions. This reflects the concentrated distribution of historical anomalous behavior migration within the feature space, effectively enhancing the system's ability to model and understand non-static anomaly morphologies.

[0100] Existing cloning strategies typically use global Gaussian perturbations to generate new detectors, but lack directional guidance, easily leading to detector drift, ineffectiveness, or overlapping coverage. This approach involves uniform sampling within trend hotspot regions to generate initial positions for trend detectors, and employing a gradient-driven mechanism based on the tensor potential function for directional force migration. During migration, Gaussian perturbations are introduced to maintain exploratory behavior, enabling the detector to efficiently migrate to potentially low-coverage areas in the feature space guided by the historical evolution path of abnormal trends. This method significantly reduces the probability of new detectors falling into already high-density regions, improving the uniformity of detector coverage and response sensitivity in the abnormal trend space.

[0101] Traditional detector distribution often lacks methods for assessing coverage rationality, leading to redundancy in local areas and omission of critical areas. This mechanism introduces Voronoi diagram analysis to construct a spatial distribution map of all converged detector locations. Based on this map, the sensing area of ​​each detector is calculated, and the locations of under-covered areas are extracted by sorting them in ascending order of area. After the trend detectors have migrated, their Euclidean distance from the under-covered areas is used to determine whether they fill the gaps. If so, they are included in the detector set. This approach enables trend-guided structural detector layout optimization, ensuring a more reasonable breadth and depth of detector coverage in space, and improving the robustness and coverage stability of overall anomaly detection.

[0102] S4: Match the data assets to be detected with the responses of the clone detector and trend detector to complete the anomaly detection and early warning.

[0103] Receive the current data asset to be detected, where each record is processed using the same feature processing method as the historical data asset to generate a set of sample points to be detected with a unified dimension:

[0104]

[0105] The system loads the current set of detectors, including:

[0106] Cloning detector collection: Each detector position is

[0107] Trend detector set: Each detector position is

[0108] For each sample point to be tested The system calculates its response value in relation to all clone detectors and trend detectors:

[0109] (1) Cloning detector response:

[0110]

[0111] (2) Trend detector response:

[0112]

[0113] The total response intensity is obtained by summing the response values ​​of each sample point.

[0114]

[0115] Set response threshold θ r ,when When, determine X test Abnormal data triggers an intelligent early warning mechanism. The response threshold θ r The value can be set based on the statistical quantiles of the total response intensity distribution of the normal sample point set. For example, the p% quantile of the normal response distribution can be taken as θ. r To achieve dynamic and sensitive judgment.

[0116] Among them, X test This represents a sample set consisting of the data assets to be tested. Let represent the i-th sample point to be detected, with dimension d, belonging to R. d N represents the total number of sample points to be tested; This represents the set of clone detectors, containing n items. c One clone detector; Let represent the position vector of the i-th clone detector in the feature space, belonging to R. d ; This represents a set of trend detectors, containing n items. t A trend detector; This represents the position vector of the k-th trend detector in the feature space, belonging to R. d ; This represents the response value between the i-th sample point and the j-th clone detector; This represents the square of the Euclidean distance between the sample point and the clone detector; The variance parameter represents the response function of the clone detector, which controls the sensitivity of the response function to distance; exp(·) represents the exponential function, which is used to map the distance to a similarity value of [0,1] (the smaller the distance, the higher the response). This represents the response value between the i-th sample point and the k-th trend detector; This represents the square of the Euclidean distance between the sample point and the trend detector; The variance parameter represents the response function of the trend detector; This represents the total response intensity of the i-th sample point across all detectors (clonal + trend). This represents the sum of the response values ​​of all clone detectors; θ represents the sum of the response values ​​of all trend detectors; r The warning threshold representing the total response intensity; if The i-th sample is considered an outlier, triggering an early warning mechanism; p% represents the statistical quantile setting, for example, set to the 95th quantile of the response intensity distribution in normal samples.

[0117] When the intelligent warning is triggered, the system automatically performs the following linked operations:

[0118] 1. Record the current abnormal data asset identifier, abnormal timestamp, and corresponding clone detector response record.

[0119] 2. Push alarm information to the visual monitoring interface or the early warning subscription terminal.

[0120] 3. Trigger data asset control mechanisms in conjunction with the system, including suspending downstream processing, reporting to the compliance system, or performing isolation operations.

[0121] A detector ensemble consisting of a clone detector and a trend detector is used for response matching of the data assets to be detected, which can effectively cover different types of potential anomaly patterns. The clone detector, derived from local replication and migration of the initial detector, has a high spatial distribution density and is suitable for detecting concentrated anomalies; the trend detector, derived from a trend memory feedback mechanism, has trend directionality and regional repair capabilities, making it suitable for covering boundary areas and response gaps. By fusing the two detector ensembles in the response function and uniformly calculating the response intensity, the detection coverage and response sensitivity can be significantly improved, thereby enhancing the overall accuracy of anomaly detection.

[0122] By applying a Gaussian function to the detector response values, the Euclidean distance is mapped to a standardized response value. A weighted sum of all detector response values ​​is then performed to obtain the total response intensity for each test sample. Simultaneously, the response threshold is dynamically set using the statistical quantiles of the response intensity distribution of the normal sample point set, avoiding misjudgments caused by fixed thresholds under different data distributions. When the data asset feature structure exhibits periodicity, heterogeneity, or data drift, this dynamic judgment method can automatically adjust the judgment boundary based on historical normal distributions, ensuring the stability and applicability of anomaly detection.

[0123] When the total response intensity exceeds the judgment threshold, the system automatically triggers a multi-level intelligent early warning response mechanism, including recording anomaly detector and data identification information, pushing visual alarm events, and triggering asset control operations in conjunction with the alarm. The response value-driven mechanism enables alarm events to be associated with specific detectors, thereby improving subsequent fault location capabilities and response interpretability. Simultaneously, the early warning linkage mechanism supports automated processing flows, such as data isolation, access control freezing, or early warning notification, reducing manual response costs and improving risk control efficiency. This facilitates a rapid closed-loop system for anomaly detection and response in actual production systems.

[0124] Example 2 is an embodiment of the present invention, which provides a real-time anomaly detection and intelligent early warning method based on artificial intelligence. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiment.

[0125] This experiment uses historical data assets from a city's data governance platform in the fourth quarter of 2023 as the experimental object. The data includes multiple dimensions such as user access logs, ETL scheduling records, API call trajectories, and field compliance tags, collecting a total of 1000 historical data records. The experiment first extracts 10-dimensional standardized features from structured fields to construct a unified feature vector set. Then, a scheduling path reconstruction tool is used to reconstruct the data lineage chain, and its path complexity is measured using information entropy. Simultaneously, data tags are parsed to generate a sovereign information tensor containing "data sensitivity," "regulatory intensity," and "cross-border level." The above features are concatenated to form the complete input vector for each sample point, and then compressed into a fixed-dimensional representation in the feature space using an MLP model.

[0126] Based on the aforementioned sample point set, the system constructs a tensor potential energy function, integrating local outlier factors, path entropy, and sovereign risk intensity to calculate the potential value of a single point. The experiment uses a Gaussian kernel function to estimate the potential field distribution and uniformly samples regions with potential energy above the 90th quantile to generate particle positions. All particles migrate in a directed manner according to the potential energy gradient, with the convergence condition being that continuous potential energy changes are lower than the average fluctuation amplitude. After particle convergence, a negative selection algorithm is applied to remove particles falling into the normal region, retaining only detectors with high offsets to construct the initial detector set.

[0127] Subsequently, each initial detector spawns multiple clone detectors, which are then guided to migrate to regions with high potential function values. The system establishes continuous and stable dwell conditions, records dwell trajectories, and performs density clustering to construct trend hotspots, forming trend detectors. Finally, the system loads the current clone detector set and the trend detector set, calculating anomaly responses for new data batches. The response uses an inverse Euclidean distance mapping function; if the response value exceeds a threshold set based on the 95th percentile of normal samples, an alert is triggered, and the system automatically records the anomaly, reports the alarm, and suspends downstream data calls.

[0128] Historical sample count: 1000; Sample dimensions: 10-dimensional structured + 1-dimensional path entropy + 3-dimensional sovereign tensor; Initial detector count: 80; Total number of clone detectors: 800 (10 each); Trend detector count: 45; New batch of samples to be tested: 300.

[0129] The response results are as follows:

[0130] Number of data entries identified as abnormal: 42, accounting for 14.00%.

[0131] 11 anomalies were independently identified by the trend detector, accounting for 26.19%.

[0132] Samples hit by multiple detectors: 21, accounting for 50.00%.

[0133] Response value threshold (95th percentile of normal): 0.1467.

[0134] Maximum response intensity: 0.2784, minimum: 0.0243, mean: 0.1082.

[0135] Experimental data demonstrates that this invention exhibits significant advantages in anomaly detection accuracy and response efficiency. Firstly, in the sample point generation stage, by employing a triple fusion model of structured features, path entropy, and sovereignty tensor, this method more accurately captures implicit high-risk features related to compliance and sensitivity compared to traditional methods based solely on feature similarity or isolated forests. In this experiment, 12 anomalous data points, highly sensitive due to their sovereignty attributes but lacking obvious anomalies in conventional features, were effectively identified by this method.

[0136] The detector particle migration strategy guided by the tensor potential function enables rapid focusing of the detector in high-risk fields and avoids redundancy and misjudgment caused by traditional grid sampling or manual position setting. Through the combination mechanism of potential gradient migration and Gaussian perturbation, the convergence of 80 initial detectors requires an average of only 8 steps, far less than the baseline method (average 13 steps), effectively improving the model initialization efficiency.

[0137] Furthermore, the introduction of trend memory mechanisms and trend detectors significantly enhances the system's ability to identify "structural evolution anomalies." For example, a certain type of field may have moderate sensitivity in the early stages but has recently clustered frequently, which traditional methods often overlook due to insufficient window size. However, this invention identifies this trend change through trajectory evolution clustering, successfully capturing multiple trend risk points and improving the overall detection coverage by approximately 18%.

[0138] Finally, in the response matching and early warning mechanism, a dynamic response threshold is adopted instead of a fixed standard. The lower limit of the response is dynamically adjusted based on the current detector coverage capability, which effectively controls the false alarm rate (the false alarm rate in this experiment is 3.2%, far lower than the 7.9% of a certain commercial model). The linkage control mechanism also realizes the automated response after the early warning, which greatly improves the practicality of the system.

[0139] In summary, this experiment fully verifies that the proposed method exhibits good reliability, sensitivity, and adaptability in anomaly detection and intelligent early warning tasks in high-dimensional data environments, demonstrating broad engineering feasibility and practical application value.

[0140] Example 3, an embodiment of the present invention, provides a real-time anomaly detection and intelligent early warning system based on artificial intelligence, comprising:

[0141] The detector module guides the migration of generated detector particles through the tensor gradient of the tensor potential energy function; it filters the migrated detector particles with negative selection to generate an initial detector; it performs cloning on the initial detector to generate a clone detector, and then migrates to the potential anomaly region based on the tensor gradient.

[0142] The trend memory module records the evolution trajectory of the clone detector and constructs trend memory units when the clone detector stably resides in a potentially anomalous region.

[0143] The coverage module generates a trend detector based on the trend memory unit and migrates it to the region where the detector particles are under-covered based on the tensor gradient.

[0144] The anomaly detection module matches the data assets to be detected with the responses of the clone detector and trend detector to complete the anomaly detection and early warning.

[0145] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0146] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0147] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0148] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc. It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0149] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A real-time anomaly detection and intelligent early warning method based on artificial intelligence, characterized in that, include: Acquire historical data assets and construct a tensor potential function; The tensor gradient of the tensor potential function guides the migration of the generated detector particles. The negative selection filters the migrated detector particles to generate an initial detector. The initial detector is cloned to generate a cloned detector, which is then migrated to the potential anomalous region based on the tensor gradient. When the clone detector stably resides in a potentially anomalous region, the evolutionary trajectory of the clone detector is recorded, and a trend memory unit is constructed. Based on the trend memory unit, a trend detector is generated and migrated to the region where the detector particles are under-covered based on the tensor gradient. The data assets to be detected are matched with the responses of the clone detector and the trend detector to complete the anomaly detection and early warning.

2. The real-time anomaly detection and intelligent early warning method based on artificial intelligence as described in claim 1, characterized in that: The historical data assets include structured and unstructured fields, task scheduling records, and data tag information; Feature vectors are extracted from structured and unstructured fields and standardized into historical feature vectors; historical data processing paths are reconstructed based on task scheduling records to construct data lineage chains; sensitive attributes and compliance identifiers of historical data assets are parsed from data tag information and mapped to generate sovereign information tensors. Using the recording order of historical data assets as an index, multiple sample points are generated sequentially to form a sample point set; each sample point is formed by concatenating historical feature vectors, data lineage chains, and sovereignty information tensors; each sample point is compressed into a vector of fixed dimension d using MLP, and the feature space R is obtained based on the vector. d The location of the sample points in the data.

3. The real-time anomaly detection and intelligent early warning method based on artificial intelligence as described in claim 2, characterized in that: The construction process of the tensor potential function is as follows: the local density anomaly is calculated using the local outlier method through the historical feature vector of each sample point; the structural entropy is calculated as the path complexity through the data lineage chain of each sample point; and the sovereign risk intensity is generated by weighted fusion of the sovereign information tensor attributes of each sample point. The local density anomaly degree, path complexity, and sovereign risk intensity of each sample point are combined according to a preset weighting factor to form a single-point anomaly potential value. The distance between any position in the feature space and the sample point is calculated using a Gaussian kernel function to construct a tensor potential energy function. The potential energy function values ​​of all sample points are sorted in order to form a set of potential energy function values. The sovereign information tensor attributes include data sensitivity, supervision strictness, and cross-border flow level.

4. The real-time anomaly detection and intelligent early warning method based on artificial intelligence as described in claim 3, characterized in that: Generating the detector particles includes, when the potential energy function value is greater than the dynamic quantile threshold θ min Uniform sampling of N in the region init θ is used as the initial position of the detector particle, and the detector particle is generated; where θ min N is the q% quantile of the potential energy function set for all sample points; init Indicates the number of particles generated by the detector; The detector particle migration includes calculating the gradient of the tensor potential function based on the current position of the detector particle, multiplying the gradient by a damping coefficient, adding it to the current particle position, and then superimposing the result with a mean of 0 and a variance of σ. 2 The perturbation term of the Gaussian distribution is used to obtain the updated position; when the difference in potential function value of any detector particle in two adjacent iterations is less than the average fluctuation amplitude of the potential function value, it is determined to be converged and the iteration is stopped.

5. The real-time anomaly detection and intelligent early warning method based on artificial intelligence as described in claim 4, characterized in that: The generation of the initial detector includes selecting normal sample points from the sample point set to form a normal sample point set; Calculate the mean center point and feature space standard deviation of the normal sample point set, and set the normal region judgment radius to be k times the feature space standard deviation; Calculate the Euclidean distance between each detector particle and the mean center point. When the Euclidean distance is greater than the normal region determination radius, retain the detector particle as the initial detector. Otherwise, remove the detector particles; The generated clone detector includes generating N from each initial detector. c There are 1 clone detector, and the position of each clone detector is determined by adding a sum to the position of the initial detector, which follows a mean of 0 and a variance of . The cloned Gaussian perturbation term is used; the gradient of the tensor potential function is calculated based on the current position of the clone detector, and migration is performed based on the cloned Gaussian perturbation term; when the tensor potential function value corresponding to the position of the clone detector is ≥ the g% quantile of the potential function set, the clone detector is considered to be in a potential anomaly region; where N c This indicates the number of clone detectors generated.

6. The real-time anomaly detection and intelligent early warning method based on artificial intelligence as described in claim 5, characterized in that: The clone detector is considered to be stably residing when it is in the potential anomalous region for L consecutive iterations. Record the position iteration trajectory of the clone detector, the trajectory length of which does not exceed T. max , among which, T max Indicates the upper limit of the trajectory length; The trend memory unit is constructed by summarizing the position iteration trajectories of all stably residing clone detectors.

7. The real-time anomaly detection and intelligent early warning method based on artificial intelligence as described in claim 6, characterized in that: The generation of the trend detector includes: extracting trend hotspot regions based on the position iteration trajectory of the stably resident clone detector; randomly sampling and generating the initial position of the trend detector within the trend hotspot region; and performing migration update based on the gradient of the tensor potential function to obtain the final trend detector position; and constructing a spatial partition map based on the position, sorting the area of ​​each spatial partition map in ascending order, and defining the area ranking in the top p% as the under-coverage area. If the distance between the trend detector location and any clone detector within an insufficient coverage area is less than a preset spatial radius threshold, then the trend detector is determined to have fallen into an insufficient coverage area. The extraction of trend hotspot regions includes extracting trajectory points of the position iteration trajectory of the stably resident clone detector and performing density clustering to construct a spherical region with a radius equal to a multiple of the cluster variance v, centered on the cluster center point; and combining all spherical regions as the trend hotspot region. The anomaly detection and early warning includes calculating the response of the data asset to be detected to each clone detector and each trend detector using the inverse ratio of Euclidean distance, and summing all the calculated response values ​​as the total response intensity. When the total response intensity exceeds the response threshold, it is determined to be abnormal and an intelligent early warning is triggered; the intelligent early warning includes recording abnormal events, pushing alarm information, and triggering data asset control in conjunction with the event.

8. A system employing the real-time anomaly detection and intelligent early warning method based on artificial intelligence as described in any one of claims 1 to 7, characterized in that: The tensor potential energy function module acquires historical data assets and constructs tensor potential energy functions. The detector module guides the migration of the generated detector particles through the tensor gradient of the tensor potential energy function. Negative selection filters the migrated detector particles to generate the initial detector; The initial detector is cloned to generate a clone detector, which is then migrated to the potential anomaly region based on the tensor gradient. The trend memory module records the evolution trajectory of the clone detector and constructs trend memory units when the clone detector stably resides in a potentially anomalous region. The coverage module generates a trend detector based on the trend memory unit and migrates to the region where the detector particles are not covered based on the tensor gradient. The anomaly detection module matches the data assets to be detected with the responses of the clone detector and trend detector to complete the anomaly detection and early warning.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the real-time anomaly detection and intelligent early warning method based on artificial intelligence as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the real-time anomaly detection and intelligent early warning method based on artificial intelligence as described in any one of claims 1 to 7.

Images