A machine learning-based data anomaly real-time monitoring and repairing method

Abstract

This invention discloses a real-time data anomaly monitoring and repair method based on machine learning, comprising the following steps: S1, constructing data fragments and generating a consistent constraint set; S2, performing self-consistent representation construction; S3, performing anomaly detection and generating anomaly label set; S4, performing masking processing to generate masked data fragments, inputting them into an improved UniTS model, introducing a structural perturbation suppression mechanism, and generating a repair candidate set; S5, performing structural consistency evaluation and screening target repair data that meets the minimum structural perturbation condition; S6, performing data repair and generating repaired data fragments; S7, feeding back the updated consistent constraint set to the improved UniTS model. This invention achieves highly sensitive real-time identification of anomalous data items, effectively reducing the anomaly false judgment rate and repair distortion rate, and improving the data security, stability, continuity, and reliability in high-concurrency data stream environments.

Description

Technical Field

[0001] This invention relates to the field of data security technology, and in particular to a method for real-time monitoring and repair of data anomalies based on machine learning. Background Technology

[0002] As information systems continue to expand and data-driven business scenarios evolve, data flows exhibit high frequency, high dimensionality, and strong correlation during collection, transmission, storage, and analysis. This has led to increasingly prominent technical demands for data anomaly monitoring and data quality assurance. Existing data anomaly detection and repair technologies primarily rely on rule-based threshold determination, statistical distribution assumptions, or single models trained on historical samples. However, these technologies generally suffer from the following problems in practical applications:

[0003] On the one hand, data streams are easily affected by network fluctuations, device jitter, interface latency, and unstable acquisition links during operation, leading to instantaneous anomalies, missing values, or drifts in local time locations or variable dimensions. Traditional methods based on fixed thresholds or static rules struggle to consider the inherent consistency between different time periods and variables, easily resulting in misjudgments or omissions. On the other hand, multivariate data often have complex temporal correlations and structural constraints. Existing anomaly detection methods often focus on single-point numerical deviations or local statistical features, lacking the ability to jointly characterize temporal continuity and variable structural consistency, leading to deviations between anomaly identification results and actual business semantics. Furthermore, in the anomaly data repair stage, existing methods typically use simple interpolation, mean imputation, or model prediction to directly replace outliers, lacking an evaluation mechanism for the consistency between the repair results and the original data structure constraints. This easily introduces new structural disturbances, thereby affecting the reliability of data analysis, decision modeling, and security assessment.

[0004] Therefore, how to provide a real-time monitoring and repair method for data anomalies based on machine learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a real-time data anomaly monitoring and repair method based on machine learning. This invention focuses on the inherent consistency relationship of data flow across time and variable dimensions, constructing an internal consistency constraint set and a self-consistent representation set. Data item anomaly detection driven by the self-consistent representation enables real-time identification of anomalous data. Furthermore, a masking mechanism is introduced based on anomaly labeling, inputting masked data fragments and the consistency constraint set into an improved UniTS model. A structural perturbation gating module is used to suppress data structure shifts that do not conform to consistency constraints, generating a repair candidate set. Then, a structural consistency evaluation is used to filter target repair data that meets the minimum structural perturbation condition, completing the iterative update of anomalous data repair and consistency constraints. This invention avoids dependence on a single threshold or static rules in the data anomaly identification stage and avoids introducing external estimation results that disrupt the original data structure in the data repair stage. It possesses advantages such as high stability in anomaly detection, strong consistency in repair results, controllable structural perturbations, and adaptability to complex data flow scenarios.

[0006] A method for real-time monitoring and repair of data anomalies based on machine learning according to an embodiment of the present invention includes the following steps:

[0007] S1. Collect data streams and construct data segments according to preset time granularity, extract internal consistency constraints from the data segments, and generate a set of consistency constraints;

[0008] S2. Based on the consistent constraint set, perform self-consistent representation construction on the data segment to generate a self-consistent representation set;

[0009] S3. Based on the self-consistent representation set, perform anomaly determination on the data items in the data segment to generate an anomaly marker set;

[0010] S4. Perform masking processing on the corresponding data items in the data segment based on the anomaly marker set to generate a masked data segment, and input the masked data segment and the consistency constraint set into the improved UniTS model. Introduce a structural perturbation suppression mechanism in the structural perturbation gating module of the improved UniTS model to generate a repair candidate set.

[0011] S5. Perform a structural consistency evaluation on the repair candidate set based on the consistency constraint set, and screen the target repair data that meets the minimum structural disturbance condition;

[0012] S6. Based on the anomaly marker set, perform data repair on the corresponding data items using the target repair data to generate repaired data fragments;

[0013] S7. Based on the repaired data fragment, re-extract the internal consistent constraints, update the consistent constraint set, and feed the updated consistent constraint set back to the improved UniTS model.

[0014] Optionally, S1 specifically includes:

[0015] The collected data stream is divided into continuous data windows according to a preset time granularity, and each data window constitutes a data segment.

[0016] Establish temporal relationships for data items in the data segment according to chronological order, and establish variable relationships according to variable dimensions;

[0017] The change magnitude between adjacent time position data items is calculated based on the aforementioned time correlation relationship, and the change magnitude is obtained by the numerical difference between adjacent time position data items;

[0018] The correlation strength between different variable data items at the same time position is calculated based on the aforementioned variable correlation relationship. The correlation strength is obtained by the sum of squares of the numerical deviations of each variable data item.

[0019] The change magnitude and the correlation strength are combined to construct an internal consistency constraint, which consists of a time dimension constraint term and a variable dimension constraint term.

[0020] All internal consistency constraints in the data segment are summarized to form a consistency constraint set.

[0021] Optionally, S2 specifically includes:

[0022] The set of consistent constraints is parsed to obtain the set of time-dimension constraints and the set of variable-dimension constraints.

[0023] For each data item in the data segment, calculate the time consistency degree corresponding to the time dimension constraint set and the variable consistency degree corresponding to the variable dimension constraint set.

[0024] The time consistency is obtained by summing the absolute values ​​of the changes in the data items at adjacent time positions, and the variable consistency is obtained by summing the squares of the numerical deviations of the data items at the same time position from the remaining variable data items.

[0025] Combining time consistency with variable consistency yields data item consistency.

[0026] The consistency of data items is normalized to obtain the self-consistency value of the data items.

[0027] Arrange all self-consistent values ​​of data items in the data segment according to the data item index order to form a self-consistent representation set.

[0028] Optionally, S3 specifically includes:

[0029] Sort the self-consistency values ​​of the data items in the self-consistency representation set to generate a self-consistency sequence;

[0030] The degree of dispersion is calculated for the self-consistent sequence, and the degree of dispersion is obtained by the sum of squares of the numerical differences between the self-consistent values ​​and the self-consistent mean in the self-consistent sequence;

[0031] The anomaly detection threshold is calculated based on the degree of dispersion, and the anomaly detection threshold is obtained by summing the products of the self-consistent mean and the degree of dispersion.

[0032] Each self-consistent value in the self-consistent sequence is compared with the anomaly determination threshold. If the self-consistent value is less than the anomaly determination threshold, the corresponding data item is marked as an anomaly.

[0033] The labeling results of all abnormal data are summarized to form an abnormal label set.

[0034] Optionally, the step of performing masking processing on the corresponding data items in the data segment based on the anomaly marker set to generate a masked data segment specifically involves:

[0035] The anomaly marker set is indexed and parsed to obtain the anomaly location set, which includes the time position index and variable dimension index of the anomaly data item in the data segment;

[0036] Based on the set of abnormal locations, locate abnormal data items in the data segment and generate abnormal location results;

[0037] Perform a value clearing operation on the data items corresponding to the anomaly location results, and set the values ​​of the anomaly data items to the preset mask values. The preset mask values ​​are obtained by the negative of the mean of the self-consistent values ​​of the data items.

[0038] Perform a mask identifier writing operation on the data items corresponding to the anomaly location results, record the mask identifier in the data fragment, and represent the mask identifier according to the ordered pair of time position index and variable dimension index in the anomaly location set;

[0039] The data segment that has completed the numerical clearing and mask identifier writing is output as a mask data segment.

[0040] Optionally, the improved UniTS model specifically includes a time-series representation module, a structural perturbation gating module, and a repair candidate generation module;

[0041] The timing representation module receives a masked data segment and a consistent constraint set. It extracts the data item values, time position indices, and variable dimension indices from the data items not identified by the mask in the masked data segment to form an input vector. The input vector undergoes a linear transformation to obtain channel feature vectors, which are then arranged in order of time position indices to form a timing feature sequence. The consistent constraint set is parsed to obtain a set of time-dimensional constraint items and a set of variable-dimensional constraint items. Numerical aggregation is performed on both the time-dimensional and variable-dimensional constraint sets to obtain constraint vectors. These constraint vectors undergo a linear transformation to obtain constraint feature vectors, which are then arranged in order of time position indices to form a constraint feature sequence.

[0042] The structural disturbance gating module receives a temporal feature sequence and a constraint feature sequence. A structural disturbance suppression mechanism is introduced within the module. This mechanism calculates channel correlation for corresponding channels at the same time position index. The channel correlation is obtained by summing the absolute values ​​of the differences between the channel feature values ​​of the temporal feature sequence and the constraint feature sequence. The structural disturbance suppression mechanism also calculates a gating coefficient based on the channel correlation. This gating coefficient is obtained by dividing one by the sum of the channel correlation values. Finally, the structural disturbance gating module performs channel weighting on the temporal feature sequence based on the gating coefficient. The gated feature sequence is obtained by multiplying the gating coefficient by the corresponding channel feature value of the temporal feature sequence.

[0043] The repair candidate generation module receives a gating feature sequence and a mask identifier. It extracts the gating feature vectors of adjacent time positions corresponding to the mask identifier and performs weighted synthesis to obtain a reconstructed vector. The weighting coefficients are obtained by normalizing the gating coefficients corresponding to the adjacent time positions. The repair candidate generation module outputs candidate values ​​at the positions corresponding to the mask identifiers based on the reconstructed vectors, and generates multiple sets of gating coefficients by superimposing a preset offset on the gating coefficients. Based on the multiple sets of gating coefficients, it outputs multiple sets of candidate values ​​to form a repair candidate set.

[0044] Optionally, S5 specifically includes:

[0045] Write each repair candidate data in the repair candidate set to the position corresponding to the mask identifier of the mask data segment to obtain the candidate repair segment;

[0046] The time dimension constraint terms in the consistency constraint set are called for the candidate repair fragments, and the time constraint residuals are calculated. The time constraint residuals are obtained by summing the absolute values ​​of the differences between the data terms at adjacent time positions.

[0047] For candidate repair fragments, invoke the variable dimension constraint terms in the consistency constraint set, calculate the variable constraint residuals, and obtain the variable constraint residuals by summing the squares of the numerical deviations of variable data terms at the same time position;

[0048] The structural disturbance value is obtained by combining the time constraint residual and the variable constraint residual. The structural disturbance value is obtained by summing the values ​​of the time constraint residual and the variable constraint residual.

[0049] The structural disturbance values ​​corresponding to the repair candidate set are compared, and the repair candidate data with the smallest structural disturbance value is selected to form the target repair data.

[0050] Optionally, S6 specifically includes:

[0051] The anomaly marker set is indexed and parsed to obtain the repair location set, which includes the time position index and variable dimension index of the anomaly data item in the data segment;

[0052] The target repair data is indexed and parsed to obtain a target value set, which includes target candidate values ​​that correspond one-to-one with the repair location set.

[0053] Based on the set of repair locations, abnormal data items are located in the data segment to obtain the set of locations to be repaired;

[0054] Write the target candidate values ​​from the target value set into the set of locations to be repaired, complete the replacement of abnormal data items, and obtain the repaired data fragment;

[0055] Perform a consistent constraint set recalculation on the repaired data fragment to obtain the recalculated structure perturbation value. The recalculated structure perturbation value is obtained by summing the values ​​of the time constraint residual and the variable constraint residual.

[0056] If the recalculated structural disturbance value is greater than the sum of the minimum structural disturbance value and the preset tolerance, then the target value set rollback operation is performed, restoring the abnormal data item values ​​of the set to be repaired to the preset mask value, and outputting the rollback repair data segment; if the recalculated structural disturbance value is less than the sum of the minimum structural disturbance value and the preset tolerance, then the repair data segment is output.

[0057] Optionally, the step of re-extracting internal consistency constraints based on the repaired data fragment and updating the consistency constraint set specifically involves:

[0058] The repaired data fragments are re-divided into continuous data windows according to a preset time granularity to obtain updated data fragments;

[0059] Establish temporal relationships for data items in the updated data fragment according to chronological order, and establish variable relationships according to variable dimensions;

[0060] The change range between adjacent time position data items is calculated based on the time correlation relationship, and the change range is obtained according to the numerical difference between adjacent time position data items.

[0061] The correlation strength between different variable data items at the same time location is calculated based on the variable correlation relationship. The correlation strength is obtained by the sum of squared deviations of the variable data items.

[0062] The magnitude of change and the strength of correlation are combined to construct an update internal consistency constraint, which consists of time dimension constraint terms and variable dimension constraint terms.

[0063] All internal consistency constraints in the updated data fragment are summarized to form an update consistency constraint set.

[0064] Optionally, feeding the updated set of consistent constraints back to the improved UniTS model specifically involves:

[0065] The updated consistent constraint set is input into the time series representation module of the improved UniTS model, and constraint term parsing is performed on the updated consistent constraint set to obtain the time dimension constraint term set and the variable dimension constraint term set.

[0066] Numerical aggregation is performed on the time dimension constraint term set and the variable dimension constraint term set respectively to obtain the updated constraint vector. The updated constraint vector is obtained by combining the sum of the values ​​of the time dimension constraint terms and the sum of the values ​​of the variable dimension constraint terms.

[0067] Perform a linear transformation on the updated constraint vector to obtain the updated constraint feature vector, and arrange them in time position index order to form the updated constraint feature sequence;

[0068] The updated monkey constraint feature sequence is input into the structural perturbation gating module to replace the constraint feature sequence.

[0069] The channel correlation is recalculated based on the updated constraint feature sequence and time-series feature sequence;

[0070] The gating coefficients are recalculated based on the channel correlation, and channel weighting is performed on the time-series feature sequences based on the gating coefficients to generate updated gating feature sequences.

[0071] The beneficial effects of this invention are:

[0072] (1) By extracting internal consistency constraints at the data fragment level and constructing a set of consistency constraints, and combining the self-consistent representation construction method driven by consistency constraints, the consistency of data items in terms of time change magnitude and variable correlation strength is quantitatively expressed, avoiding reliance on a single statistical feature or fixed rule for anomaly judgment, improving the stability and interpretability of anomaly identification in complex data flow scenarios, and reducing the interference of noise fluctuations and local mutations on anomaly judgment results.

[0073] (2) By constructing an anomaly judgment threshold by combining the self-consistent representation set and the degree of dispersion, the generation of the anomaly label set is constrained by both the overall distribution characteristics and the degree of local deviation. This avoids the false detection and false negative problems caused by the static setting of the threshold in traditional methods, and realizes the adaptive judgment of the abnormal state of data, thereby enhancing the applicability of anomaly detection under different data distribution conditions.

[0074] (3) In the abnormal data repair stage, a masked data fragment and improved UniTS model collaborative processing mechanism is introduced. Through the structural perturbation gating module, the consistency constraint information is directly involved in the temporal feature modulation to suppress the structural shift that may be introduced during the repair process. The target repair data with the minimum structural perturbation is screened through the repair candidate set and structural consistency evaluation, which effectively avoids the repair result from destroying the temporal correlation and variable correlation of the original data fragment.

[0075] (4) By re-extracting the internal consistent constraints and updating the consistent constraint set after the repair is completed, and then feeding the updated consistent constraint set back to the improved UniTS model, a closed-loop processing flow of anomaly identification, repair and constraint update is formed, so that the consistent constraints continue to evolve with the changes in data state, thereby improving the adaptability of the real-time monitoring and repair method of data anomalies to long-term running data streams and the overall consistency of the repair results. Attached Figure Description

[0076] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0077] Figure 1 This is a flowchart of a real-time data anomaly monitoring and repair method based on machine learning proposed in this invention;

[0078] Figure 2 This is a schematic diagram of the structure of the improved UniTS model proposed in this invention;

[0079] Figure 3 This is a data flow diagram of a real-time data anomaly monitoring and repair method based on machine learning proposed in this invention. Detailed Implementation

[0080] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0081] refer to Figure 1-3 A real-time data anomaly monitoring and repair method based on machine learning includes the following steps:

[0082] S1. Collect data streams and construct data segments according to preset time granularity, extract internal consistency constraints from the data segments, and generate a set of consistency constraints;

[0083] S2. Based on the consistent constraint set, perform self-consistent representation construction on the data segment to generate a self-consistent representation set;

[0084] S3. Based on the self-consistent representation set, perform anomaly determination on the data items in the data segment to generate an anomaly marker set;

[0085] S4. Perform masking processing on the corresponding data items in the data segment based on the anomaly marker set to generate a masked data segment, and input the masked data segment and the consistency constraint set into the improved UniTS model. Introduce a structural perturbation suppression mechanism in the structural perturbation gating module of the improved UniTS model to generate a repair candidate set.

[0086] S5. Perform a structural consistency evaluation on the repair candidate set based on the consistency constraint set, and screen the target repair data that meets the minimum structural disturbance condition;

[0087] S6. Based on the anomaly marker set, perform data repair on the corresponding data items using the target repair data to generate repaired data fragments;

[0088] S7. Based on the repaired data fragment, re-extract the internal consistent constraints, update the consistent constraint set, and feed the updated consistent constraint set back to the improved UniTS model.

[0089] In this embodiment, S1 specifically refers to:

[0090] The collected data stream is divided into continuous data windows according to a preset time granularity, and each data window constitutes a data segment.

[0091] Establish temporal relationships for data items in the data segment according to chronological order, and establish variable relationships according to variable dimensions;

[0092] The change magnitude between adjacent time position data items is calculated based on the aforementioned time correlation relationship, and the change magnitude is obtained by the numerical difference between adjacent time position data items;

[0093] The correlation strength between different variable data items at the same time position is calculated based on the aforementioned variable correlation relationship. The correlation strength is obtained by the sum of squares of the numerical deviations of each variable data item.

[0094] The change magnitude and the correlation strength are combined to construct an internal consistency constraint, which consists of a time dimension constraint term and a variable dimension constraint term.

[0095] All internal consistency constraints in the data segment are summarized to form a consistency constraint set.

[0096] In this embodiment, S2 specifically refers to:

[0097] Constraint terms are parsed on the consistency constraint set. The internal consistency constraints contained in the consistency constraint set are split according to the source dimension, and the constraint terms composed of time-related relationships and the constraint terms composed of variable-related relationships are distinguished. Time-dimensional constraint term set and variable-dimensional constraint term set are formed respectively. The time-dimensional constraint term set records the numerical change constraints between data terms at adjacent time positions, and the variable-dimensional constraint term set records the numerical association constraints between different variable data terms at the same time position.

[0098] For each data item in the data segment, the time consistency is calculated based on the set of time dimension constraints. The time consistency is obtained by iterating through the changes in the values ​​of the data items at the previous time position and the next time position, taking the absolute value of the difference between the values ​​of the corresponding data items at adjacent time positions and summing them.

[0099] For each data item in the data segment, the variable consistency is calculated based on the set of variable dimension constraints. The variable consistency is obtained by fixing the time position of the data item, comparing the values ​​of the remaining variable data items at that time position with the value of the target data item one by one, calculating the numerical deviation, taking the square, and summing them.

[0100] The consistency of time and the consistency of variables are combined by weighted summation, with a weight of 0.5 for both consistency of time and consistency of variables, to obtain the consistency of data items.

[0101] Normalization is performed on the consistency of data items. Normalization is performed by obtaining the maximum and minimum values ​​of the consistency of data items within the data segment range, subtracting the minimum value from the consistency of a single data item, and then dividing by the difference between the maximum and minimum values ​​to obtain the consistency value of the data item between 0 and 1.

[0102] The self-consistent values ​​of all data items in the data segment are arranged according to the combination of the time position index and the variable dimension index of the data item in the data segment. The arrangement rule is that the time position index takes priority and the variable dimension index takes second place, forming a self-consistent representation set. The self-consistent representation set is used to characterize the overall consistency state of the data items in the data segment under the dual constraints of time and variables.

[0103] In this embodiment, S3 specifically refers to:

[0104] The self-consistent values ​​of the data items contained in the self-consistent representation set are sorted. The sorting rule is to arrange the values ​​in descending order of self-consistent values. The sorting result forms a one-dimensional ordered numerical structure. The self-defined one-dimensional ordered numerical structure is defined as a self-consistent sequence. Each position in the self-consistent sequence maintains a mapping relationship with the time position index and variable dimension index in the data segment.

[0105] The degree of dispersion is calculated for the self-consistent sequence. The degree of dispersion is obtained by taking the arithmetic mean of all self-consistent values ​​in the self-consistent sequence as the self-consistent mean, and then summing the squared differences between each self-consistent value and the self-consistent mean. The degree of dispersion is used to characterize the overall fluctuation level of the consistency state of data items in the self-consistent sequence.

[0106] The anomaly detection threshold is calculated based on the degree of dispersion. The anomaly detection threshold is obtained by linearly combining the self-consistent mean and the degree of dispersion. The linear combination method is to add the product of the self-consistent mean and the degree of dispersion to the self-consistent mean. Preferably, the amplification factor of the degree of dispersion in the anomaly detection threshold is 0.8 to adapt to the changes in the abnormal fluctuation amplitude in the real-time data stream scenario.

[0107] Each self-consistent value in the self-consistent sequence is compared with the anomaly judgment threshold. When the self-consistent value is less than the anomaly judgment threshold, the data item corresponding to the self-consistent value is marked as abnormal data. The abnormal data is marked in binary form, with the abnormal state mark value being 1 and the non-abnormal state mark value being 0.

[0108] All data items that have completed the comparison are marked with anomalies. The anomaly marker set is formed by organizing the data items according to the combination order of their time position index and variable dimension index in the data segment. The anomaly marker set is used to characterize the spatial and temporal distribution of abnormal data items in the data segment.

[0109] In this embodiment, the step of performing masking processing on the corresponding data items in the data segment based on the anomaly marker set to generate a masked data segment specifically involves:

[0110] The anomaly marker set is indexed and parsed. The index parsing process is carried out according to the index relationship of the data item corresponding to each anomaly marker in the anomaly marker set. The index relationship consists of time position index and variable dimension index. The time position index represents the time order position of the anomaly data item in the data segment, and the variable dimension index represents the variable channel position to which the anomaly data item belongs. The index parsing result forms an anomaly position set, which is an ordered set of index pairs consisting of time position index and variable dimension index.

[0111] Based on the set of abnormal locations, the data fragment is matched and located item by item. The matching rule is that when the time location index and the variable dimension index are consistent, it is determined to be the target abnormal data item. The location result is saved in the form of an index mapping table to form the abnormal location result. The abnormal location result is used to identify the location of the data item that needs to be processed in the data fragment.

[0112] The data items corresponding to the anomaly location results are cleared. The clearing method is to replace the original values ​​of the anomaly data items with preset mask values. The preset mask values ​​are obtained by taking the arithmetic mean of the self-consistency values ​​of all data items in the data segment and taking the opposite of the self-consistency value. Preferably, the self-consistency value is calculated using the set of self-consistency values ​​in the complete data segment before the anomaly determination, and the number of samples for the self-consistency value is no less than 100 data items.

[0113] The mask identifier is written to the corresponding position of the anomaly location result after the numerical replacement is completed. The mask identifier is recorded in the form of binary index encoding. The encoding content is formed by combining the time position index and variable dimension index in the anomaly position set in a fixed order. The mask identifier is written as additional marker information to the corresponding index position of the data segment to distinguish between the cleared data items and the data items that have not been cleared.

[0114] The data segments that have completed the numerical clearing and mask identifier writing processes are subjected to integrity verification. The verification includes data segment length consistency verification and index continuity verification. After the verification passes, the processing result is output, and the processing result is defined as a mask data segment.

[0115] In this embodiment, the improved UniTS model specifically includes a time-series representation module, a structural perturbation gating module, and a repair candidate generation module;

[0116] The time series representation module receives a masked data segment and a set of consistent constraints. It reads the data item value, time position index, and variable dimension index for each data item in the masked data segment that is not identified by the mask. The data item value represents the original observation value, the time position index represents the sequential position of the data item in the data segment, and the variable dimension index represents the variable channel number to which the data item belongs. The three types of information are concatenated in a fixed order to form an input vector. The concatenation method is to first arrange the data item values, then arrange the time position index, and finally arrange the variable dimension index.

[0117] The input vector participates in the linear transformation calculation. The linear transformation obtains the channel feature vector by weighted summation of the input vector using the weight matrix and the bias vector. The dimension of the weight matrix is ​​consistent with the dimension of the input vector. Preferably, the initial value of the weight matrix is ​​set to be between -1 and 1, and the initial value of the bias vector is set to zero.

[0118] The channel feature vectors are arranged in ascending order of time position index, forming a time-series feature sequence that maintains the time structure of the data segment.

[0119] The consistent constraint set is processed by constraint term parsing. The constraint term parsing process splits the constraint terms into time dimension constraints and variable dimension constraints recorded in the consistent constraint set. The time dimension constraints represent the change constraints between adjacent time positions, and the variable dimension constraints represent the association constraints between variables at the same time position.

[0120] The time dimension constraint set and the variable dimension constraint set participate in numerical aggregation processing respectively. The numerical aggregation method is to sum the constraint values ​​and divide by the number of constraint terms to obtain the constraint vector. The constraint vector dimension is consistent with the number of time position indices.

[0121] The constraint vector is used to calculate the constraint feature vector through linear transformation. The linear transformation method is the same as the channel feature vector generation method. The constraint feature vectors are arranged in time position index order to form a constraint feature sequence.

[0122] The structural disturbance gating module receives the temporal feature sequence and the constraint feature sequence, and performs channel-by-channel calculation on the corresponding channels of the temporal feature sequence and the constraint feature sequence under the same time position index. The calculation content is the channel correlation. The channel correlation is obtained by summing the absolute values ​​of the differences between the channel feature values ​​of the temporal feature sequence and the channel feature values ​​of the constraint feature sequence.

[0123] Channel correlation is used in the calculation of the gating coefficient. The gating coefficient is obtained by dividing one by one and summing the channel correlation value. Preferably, to avoid numerical instability, 1 is added as a smoothing term to the channel correlation calculation result.

[0124] The structural perturbation suppression mechanism adjusts the channel weights of the temporal feature sequence through a gating coefficient. The adjustment method is to multiply the gating coefficient with the corresponding channel feature value, and the product result constitutes the gating feature sequence. The gating feature sequence maintains the time position index order unchanged.

[0125] The repair candidate generation module receives a gated feature sequence and a mask identifier, and locates the position to be reconstructed based on the time position index and variable dimension index recorded in the mask identifier.

[0126] Extract gated feature vectors from the adjacent time positions before and after the location. The adjacent time range is preferably set to the previous time position and the next time position. When the time position is at the boundary of the data segment, only the existing adjacent time positions are selected.

[0127] The extracted adjacency-gated feature vectors participate in the weighted synthesis process. The weighting coefficients are obtained by normalizing the gating coefficients corresponding to the adjacency time positions. The normalization method is to divide the adjacency gating coefficients by the sum of the adjacency gating coefficients. The weighted synthesis result forms the reconstructed vector.

[0128] The reconstructed vector is converted into candidate values ​​through a linear mapping, and the linear mapping parameters maintain the same dimensional structure as the linear transformation parameters in the time series representation module.

[0129] Multiple sets of gate coefficients are generated by superimposing a preset offset on the gate coefficient. The preset offset is preferably set to multiple discrete values ​​in the range of -0.05 to 0.05. The superposition method is to directly add the gate coefficient and the offset.

[0130] Based on the process of repeatedly reconstructing vectors using multiple sets of gating coefficients and generating candidate values, multiple sets of candidate values ​​are obtained, and all candidate values ​​are summarized to form a repair candidate set.

[0131] In this embodiment, the improved UniTS model is built upon the original UniTS model, addressing issues such as unstable anomaly mask ratios, easy introduction of structural shifts in repair results, and difficulty in effectively incorporating constraint information into modeling in data anomaly repair scenarios. The improvement process does not alter the original unified temporal modeling framework of the UniTS model; instead, it introduces modeling paths directly related to consistent constraints within the data segment, transforming the consistent constraint set from external constraints into computationally commensurate structural features within the model. By synchronously introducing consistent constraint features during the temporal representation stage and aligning them with the same time index as the temporal features, the model gains structural awareness at the representation level. Furthermore, structural perturbation gating is implemented. The module constructs a gating adjustment method based on channel correlation to suppress feature channels that deviate from the consistency constraints and retain the weights of feature channels that conform to the consistency constraints, thereby limiting the propagation of structural disturbances during the repair process. By introducing gating coefficient offsets in the repair candidate generation stage to generate multiple sets of candidate results, the repair process does not rely on a single prediction path and forms a candidate space with controllable structural disturbances. The improved UniTS model, while maintaining the original unified temporal modeling capability, achieves deep coupling between consistency constraints and temporal features, forming a constraint-driven anomaly repair modeling method. It has technical features that distinguish it from the existing UniTS model in terms of anomaly repair stability, structural consistency control, and feedback closed-loop update.

[0132] In this embodiment, S5 specifically refers to:

[0133] Each candidate data in the candidate data set is written to the corresponding position of the mask identifier of the mask data segment according to the index order of the candidate data in the candidate data set. The corresponding position of the mask identifier is uniquely determined by the ordered pair of time position index and variable dimension index. During the writing process, the values ​​of data items not identified by the mask identifier in the mask data segment remain unchanged, thus obtaining the candidate repair segment.

[0134] The time dimension constraint terms in the consistency constraint set are called for the candidate repair segments. For each data item in the candidate repair segment that has undergone numerical replacement, the numerical difference between the data item at the previous time position and the data item at the next time position are calculated respectively. The absolute values ​​of the numerical differences are taken and summed to obtain the time constraint residual corresponding to the candidate repair segment. The time constraint residual is used to characterize the degree of deviation of the candidate repair segment in time continuity.

[0135] The variable dimension constraint terms in the consistency constraint set are called for the candidate repair fragment. At the same time position, the values ​​of the data items that have undergone numerical replacement in the candidate repair fragment are compared one by one with the values ​​of the remaining variable data items. The numerical deviation is calculated, and the square of the numerical deviation is summed to obtain the variable constraint residual corresponding to the candidate repair fragment. The variable constraint residual is used to characterize the degree of offset of the candidate repair fragment in the variable association structure.

[0136] The time constraint residual and the variable constraint residual are combined by direct addition to obtain the structural perturbation value, which is used to characterize the overall structural offset of the candidate repair segment relative to the consistent constraint set.

[0137] The structural perturbation values ​​corresponding to all candidate repair segments in the repair candidate set are compared one by one, and sorted in ascending order of structural perturbation values. The repair candidate data with the smallest structural perturbation value is selected, and the candidate repair segment corresponding to the repair candidate data is determined as the target repair data. Preferably, when the difference in structural perturbation values ​​corresponding to multiple repair candidate data is less than 0.01, the repair candidate data with the smaller time constraint residual is selected as the target repair data to reduce the risk of time continuity disruption.

[0138] In this embodiment, S6 specifically refers to:

[0139] The anomaly marker set is indexed and parsed. Based on the time location index and variable dimension index recorded in the anomaly marker set, a repair location set is constructed. Each element in the repair location set consists of a time location index and a variable dimension index, which are used to uniquely indicate the location of the data item in the data segment where the anomaly occurred.

[0140] The target repair data is indexed and parsed. According to the order of the target repair data in the repair candidate set screening results, the target candidate values ​​that correspond one-to-one with the repair location set are extracted to form a target value set. The target candidate values ​​in the target value set are consistent with the repair location set in terms of index order.

[0141] Based on the set of repair locations, abnormal data items are located in the data segment. According to the matching relationship between the time location index and the variable dimension index, the set of locations to be repaired is determined. The set of locations to be repaired only includes the locations of data items identified by the abnormal marker set, and does not include the locations of data items that have not experienced abnormalities.

[0142] The target candidate values ​​in the target value set are written into the location to be repaired one by one in the index order. The data item values ​​at the corresponding positions are replaced by the target candidate values. The data item values ​​not included in the location to be repaired remain unchanged, thus obtaining the repaired data fragment.

[0143] Perform consistent constraint set recalculation on the repaired data fragment. Based on the time dimension constraint terms, recalculate the absolute value of the difference between the numerical values ​​of data items at adjacent time positions and sum them to obtain the recalculated time constraint residual. Based on the variable dimension constraint terms, recalculate the sum of squares of the numerical deviations of variable data items at the same time position to obtain the recalculated variable constraint residual. Then, directly add the recalculated time constraint residual and the recalculated variable constraint residual to obtain the recalculated structural perturbation value.

[0144] The recalculated structural disturbance value is compared with the minimum structural disturbance value, which is derived from the minimum structural disturbance value obtained during the candidate set screening stage. Preferably, the preset tolerance value is 0.05.

[0145] When the recalculated structural disturbance value is greater than the sum of the minimum structural disturbance value and the preset tolerance, the target value set is rolled back. The data item values ​​in the set to be repaired are restored to the preset mask value according to the set of repair locations, and the rollback repair data fragment is obtained.

[0146] When the recalculated structural disturbance value is less than or equal to the sum of the minimum structural disturbance value and the preset tolerance, the target candidate value is confirmed to meet the consistency constraint requirements, and the repaired data segment is output.

[0147] In this embodiment, the step of re-extracting internal consistency constraints based on the repaired data fragment and updating the consistency constraint set specifically involves:

[0148] The repaired data fragments are re-divided into continuous data windows according to a preset time granularity to obtain updated data fragments;

[0149] Establish temporal relationships for data items in the updated data fragment according to chronological order, and establish variable relationships according to variable dimensions;

[0150] The change range between adjacent time position data items is calculated based on the time correlation relationship, and the change range is obtained according to the numerical difference between adjacent time position data items.

[0151] The correlation strength between different variable data items at the same time location is calculated based on the variable correlation relationship. The correlation strength is obtained by the sum of squared deviations of the variable data items.

[0152] The magnitude of change and the strength of correlation are combined to construct an update internal consistency constraint, which consists of time dimension constraint terms and variable dimension constraint terms.

[0153] All internal consistency constraints in the updated data fragment are summarized to form an update consistency constraint set.

[0154] In this embodiment, feeding the updated consistent constraint set back to the improved UniTS model specifically involves:

[0155] The updated consistent constraint set is input into the time series representation module of the improved UniTS model, and constraint term parsing is performed on the updated consistent constraint set to obtain the time dimension constraint term set and the variable dimension constraint term set.

[0156] Numerical aggregation is performed on the time dimension constraint term set and the variable dimension constraint term set respectively to obtain the updated constraint vector. The updated constraint vector is obtained by combining the sum of the values ​​of the time dimension constraint terms and the sum of the values ​​of the variable dimension constraint terms.

[0157] Perform a linear transformation on the updated constraint vector to obtain the updated constraint feature vector, and arrange them in time position index order to form the updated constraint feature sequence;

[0158] The updated monkey constraint feature sequence is input into the structural perturbation gating module to replace the constraint feature sequence.

[0159] The channel correlation is recalculated based on the updated constraint feature sequence and time-series feature sequence;

[0160] The gating coefficients are recalculated based on the channel correlation, and channel weighting is performed on the time-series feature sequences based on the gating coefficients to generate updated gating feature sequences.

[0161] Example 1: To verify the feasibility of this invention in practice, it was applied to a data monitoring and anomaly repair system in a smart manufacturing plant. This plant collects machine operating status data in real time through sensors, including parameters such as temperature, pressure, vibration, and rotational speed. Due to frequent fluctuations in data acquisition equipment and equipment aging, the system often experiences instantaneous data anomalies, such as sudden sensor failures or signal transmission delays, leading to abnormal data in some areas and affecting production decisions and equipment maintenance. Traditional methods rely on fixed thresholds for anomaly detection, which easily leads to misjudgments and missed detections. Therefore, the real-time data anomaly monitoring and repair method based on machine learning proposed in this paper can dynamically identify and repair anomalies, ensuring stable system operation.

[0162] In this factory, data streams frequently experience anomalies due to equipment malfunctions, network problems, or system updates. Factory managers deployed the data anomaly monitoring and repair system of this invention to process the real-time collected data. First, the system divides the data stream into multiple data segments according to time granularity and extracts the correlation constraints between time and variables. Next, the system calculates the consistency of data items using a self-consistent representation algorithm, uses an improved UniTS model to determine data anomalies, and repairs abnormal data through a structural perturbation gating module, ensuring the continuity and consistency of data after anomaly repair.

[0163] In practical applications, the system effectively avoids misjudgments common in traditional methods and, through a structural disturbance control mechanism, ensures that the repair results do not damage the intrinsic structure of the data. The repaired data can then be used for subsequent production decisions, equipment monitoring, and maintenance, improving production efficiency and reducing equipment failure rates.

[0164] To verify the effectiveness of the method of this invention in practical applications, we conducted experiments on the production data stream of a factory, recording abnormal and repaired data under different time windows and machine states. Experimental results show that the system using the method of this invention can significantly improve the accuracy of data anomaly detection, and the repaired data has a high degree of structural consistency with the original data, providing more reliable data support for subsequent analysis.

[0165] The following are two data tables comparing the data before and after repair using the method of this invention, demonstrating the data repair effect before and after the repair.

[0166] Table 1 Comparison of Abnormal Data Before and After Repair

[0167] Time and location value of variable A value of variable B value of variable C value of variable D Value of variable A after repair Value of variable B after repair Value of variable C after repair value of variable D after repair T1 120 88 56 25 121 89 57 26 T2 115 86 53 30 116 87 54 31 T3 121 90 58 28 121 91 59 29 T4 123 92 59 32 123 93 60 33

[0168] Table 1 shows a comparison of the data before and after the repair. Variables A, B, C, and D at time positions T1 to T4 exhibited varying degrees of anomalies before the repair (e.g., the value of variable C was 56 at T1, but 57 after the repair). The method of this invention repairs these data by introducing self-consistent representation and structural perturbation gating mechanisms, ensuring that the repaired data conforms to consistency constraints in both time and variable dimensions. Compared to the abnormal data before repair, the repaired data values ​​show smaller changes and conform to the preset time and variable relationships, indicating that this method has high data consistency.

[0169] Table 2 Comparison of structural disturbance values ​​before and after repair

[0170] Time and location Original structural disturbance value Structural disturbance value after repair Change T1 0.12 0.05 -0.07 T2 0.15 0.06 -0.09 T3 0.10 0.04 -0.06 T4 0.14 0.05 -0.09

[0171] Table 2 shows a comparison of structural perturbation values ​​before and after the repair. Before data repair, the original data had large structural perturbation values, indicating significant deviations in the temporal and variable correlations of the data. Using the repair method of this invention, the structural perturbation values ​​were significantly reduced, indicating that the structural shift between the repaired data and the original data was effectively suppressed. The repaired data maintained higher structural consistency, demonstrating that this method can preserve the intrinsic correlations of the data to the greatest extent possible and avoid introducing new structural shifts during data repair.

[0172] The above analysis demonstrates that this invention not only effectively identifies anomalous data but also preserves temporal and variable consistency during the repair process, avoiding structural shifts. Experimental results prove the superiority of this method in complex data flow scenarios, exhibiting high accuracy and operability. It significantly improves the efficiency of data monitoring and repair, providing reliable technical support for data management in intelligent manufacturing and other industries.

[0173] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for real-time monitoring and repair of data anomalies based on machine learning, characterized in that, Includes the following steps: S1. Collect data streams and construct data segments according to preset time granularity, extract internal consistency constraints from the data segments, and generate a set of consistency constraints; S2. Based on the consistent constraint set, perform self-consistent representation construction on the data segment to generate a self-consistent representation set; S3. Based on the self-consistent representation set, perform anomaly determination on the data items in the data segment to generate an anomaly marker set; S4. Perform masking processing on the corresponding data items in the data segment based on the anomaly marker set to generate a masked data segment, and input the masked data segment and the consistency constraint set into the improved UniTS model. Introduce a structural perturbation suppression mechanism in the structural perturbation gating module of the improved UniTS model to generate a repair candidate set. S5. Perform a structural consistency evaluation on the repair candidate set based on the consistency constraint set, and screen the target repair data that meets the minimum structural disturbance condition; S6. Based on the anomaly marker set, perform data repair on the corresponding data items using the target repair data to generate repaired data fragments; S7. Based on the repaired data fragment, re-extract the internal consistent constraints, update the consistent constraint set, and feed the updated consistent constraint set back to the improved UniTS model.

2. The method for real-time monitoring and repair of data anomalies based on machine learning according to claim 1, characterized in that, Specifically, S1 is: The collected data stream is divided into continuous data windows according to a preset time granularity, and each data window constitutes a data segment. Establish temporal relationships for data items in the data segment according to chronological order, and establish variable relationships according to variable dimensions; The change magnitude between adjacent time position data items is calculated based on the aforementioned time correlation relationship, and the change magnitude is obtained by the numerical difference between adjacent time position data items; The correlation strength between different variable data items at the same time position is calculated based on the aforementioned variable correlation relationship. The correlation strength is obtained by the sum of squares of the numerical deviations of each variable data item. The change magnitude and the correlation strength are combined to construct an internal consistency constraint, which consists of a time dimension constraint term and a variable dimension constraint term. All internal consistency constraints in the data segment are summarized to form a consistency constraint set.

3. The method for real-time monitoring and repair of data anomalies based on machine learning according to claim 1, characterized in that, Specifically, S2 is: The set of consistent constraints is parsed to obtain the set of time-dimension constraints and the set of variable-dimension constraints. For each data item in the data segment, calculate the time consistency degree corresponding to the time dimension constraint set and the variable consistency degree corresponding to the variable dimension constraint set. The time consistency is obtained by summing the absolute values ​​of the changes in the data items at adjacent time positions, and the variable consistency is obtained by summing the squares of the numerical deviations of the data items at the same time position from the remaining variable data items. Combining time consistency with variable consistency yields data item consistency. The consistency of data items is normalized to obtain the self-consistency value of the data items. Arrange all self-consistent values ​​of data items in the data segment according to the data item index order to form a self-consistent representation set.

4. The method for real-time monitoring and repair of data anomalies based on machine learning according to claim 1, characterized in that, Specifically, S3 is: Sort the self-consistency values ​​of the data items in the self-consistency representation set to generate a self-consistency sequence; The degree of dispersion is calculated for the self-consistent sequence, and the degree of dispersion is obtained by the sum of squares of the numerical differences between the self-consistent values ​​and the self-consistent mean in the self-consistent sequence; The anomaly detection threshold is calculated based on the degree of dispersion, and the anomaly detection threshold is obtained by summing the products of the self-consistent mean and the degree of dispersion. Each self-consistent value in the self-consistent sequence is compared with the anomaly determination threshold. If the self-consistent value is less than the anomaly determination threshold, the corresponding data item is marked as an anomaly. The labeling results of all abnormal data are summarized to form an abnormal label set.

5. The method for real-time monitoring and repair of data anomalies based on machine learning according to claim 1, characterized in that, The process of performing masking processing on the corresponding data items in the data segment based on the anomaly marker set to generate a masked data segment is specifically as follows: The anomaly marker set is indexed and parsed to obtain the anomaly location set, which includes the time position index and variable dimension index of the anomaly data item in the data segment; Based on the set of abnormal locations, locate abnormal data items in the data segment and generate abnormal location results; Perform a value clearing operation on the data items corresponding to the anomaly location results, and set the values ​​of the anomaly data items to the preset mask values. The preset mask values ​​are obtained by the negative of the mean of the self-consistent values ​​of the data items. Perform a mask identifier writing operation on the data items corresponding to the anomaly location results, record the mask identifier in the data fragment, and represent the mask identifier according to the ordered pair of time position index and variable dimension index in the anomaly location set; The data segment that has completed the numerical clearing and mask identifier writing is output as a mask data segment.

6. The method for real-time monitoring and repair of data anomalies based on machine learning according to claim 1, characterized in that, The improved UniTS model specifically includes a time-series representation module, a structural perturbation gating module, and a repair candidate generation module; The timing representation module receives a masked data segment and a consistent constraint set. It extracts the data item values, time position indices, and variable dimension indices from the data items not identified by the mask in the masked data segment to form an input vector. The input vector undergoes a linear transformation to obtain channel feature vectors, which are then arranged in order of time position indices to form a timing feature sequence. The consistent constraint set is parsed to obtain a set of time-dimensional constraint items and a set of variable-dimensional constraint items. Numerical aggregation is performed on both the time-dimensional and variable-dimensional constraint sets to obtain constraint vectors. These constraint vectors undergo a linear transformation to obtain constraint feature vectors, which are then arranged in order of time position indices to form a constraint feature sequence. The structural disturbance gating module receives a temporal feature sequence and a constraint feature sequence. A structural disturbance suppression mechanism is introduced within the module. This mechanism calculates channel correlation for corresponding channels at the same time position index. The channel correlation is obtained by summing the absolute values ​​of the differences between the channel feature values ​​of the temporal feature sequence and the constraint feature sequence. The structural disturbance suppression mechanism also calculates a gating coefficient based on the channel correlation. This gating coefficient is obtained by dividing one by the sum of the channel correlation values. Finally, the structural disturbance gating module performs channel weighting on the temporal feature sequence based on the gating coefficient. The gated feature sequence is obtained by multiplying the gating coefficient by the corresponding channel feature value of the temporal feature sequence. The repair candidate generation module receives a gated feature sequence and a mask identifier, extracts the gated feature vector of the adjacent time position corresponding to the mask identifier, and performs weighted synthesis to obtain a reconstructed vector. The weighting coefficient is obtained by normalizing the gated coefficient corresponding to the adjacent time position. The repair candidate generation module outputs a mask identifier for the corresponding position based on the reconstructed vector, and generates multiple sets of gating coefficients by superimposing a preset offset on the gating coefficients. Based on the multiple sets of gating coefficients, it outputs multiple sets of candidate values ​​to form a repair candidate set.

7. The method for real-time monitoring and repair of data anomalies based on machine learning according to claim 1, characterized in that, Specifically, S5 is: Write each repair candidate data in the repair candidate set to the position corresponding to the mask identifier of the mask data segment to obtain the candidate repair segment; The time dimension constraint terms in the consistency constraint set are called for the candidate repair fragments, and the time constraint residuals are calculated. The time constraint residuals are obtained by summing the absolute values ​​of the differences between the data terms at adjacent time positions. For candidate repair fragments, invoke the variable dimension constraint terms in the consistency constraint set, calculate the variable constraint residuals, and obtain the variable constraint residuals by summing the squares of the numerical deviations of variable data terms at the same time position; The structural disturbance value is obtained by combining the time constraint residual and the variable constraint residual. The structural disturbance value is obtained by summing the values ​​of the time constraint residual and the variable constraint residual. The structural disturbance values ​​corresponding to the repair candidate set are compared, and the repair candidate data with the smallest structural disturbance value is selected to form the target repair data.

8. The method for real-time monitoring and repair of data anomalies based on machine learning according to claim 1, characterized in that, Specifically, S6 is: The anomaly marker set is indexed and parsed to obtain the repair location set, which includes the time position index and variable dimension index of the anomaly data item in the data segment; The target repair data is indexed and parsed to obtain a target value set, which includes target candidate values ​​that correspond one-to-one with the repair location set. Based on the set of repair locations, abnormal data items are located in the data segment to obtain the set of locations to be repaired; Write the target candidate values ​​from the target value set into the set of locations to be repaired, complete the replacement of abnormal data items, and obtain the repaired data fragment; Perform a consistent constraint set recalculation on the repaired data fragment to obtain the recalculated structure perturbation value. The recalculated structure perturbation value is obtained by summing the values ​​of the time constraint residual and the variable constraint residual. If the recalculated structural disturbance value is greater than the sum of the minimum structural disturbance value and the preset tolerance, then the target value set rollback operation is performed to restore the abnormal data item values ​​of the set to be repaired to the preset mask value and output the rollback repair data fragment. If the recalculated structural disturbance value is less than the sum of the minimum structural disturbance value and the preset tolerance, then the repaired data segment will be output.

9. The method for real-time monitoring and repair of data anomalies based on machine learning according to claim 1, characterized in that, The step of re-extracting internal consistency constraints based on the repaired data fragment and updating the consistency constraint set specifically involves: The repaired data fragments are re-divided into continuous data windows according to a preset time granularity to obtain updated data fragments; Establish temporal relationships for data items in the updated data fragment according to chronological order, and establish variable relationships according to variable dimensions; The change range between adjacent time position data items is calculated based on the time correlation relationship, and the change range is obtained according to the numerical difference between adjacent time position data items. The correlation strength between different variable data items at the same time location is calculated based on the variable correlation relationship. The correlation strength is obtained by the sum of squared deviations of the variable data items. The magnitude of change and the strength of correlation are combined to construct an update internal consistency constraint, which consists of time dimension constraint terms and variable dimension constraint terms. All internal consistency constraints in the updated data fragment are summarized to form an update consistency constraint set.

10. The method for real-time monitoring and repair of data anomalies based on machine learning according to claim 1, characterized in that, The process of feeding the updated consistent constraint set back to the improved UniTS model specifically involves: The updated consistent constraint set is input into the time series representation module of the improved UniTS model, and constraint term parsing is performed on the updated consistent constraint set to obtain the time dimension constraint term set and the variable dimension constraint term set. Numerical aggregation is performed on the time dimension constraint term set and the variable dimension constraint term set respectively to obtain the updated constraint vector. The updated constraint vector is obtained by combining the sum of the values ​​of the time dimension constraint terms and the sum of the values ​​of the variable dimension constraint terms. Perform a linear transformation on the updated constraint vector to obtain the updated constraint feature vector, and arrange them in time position index order to form the updated constraint feature sequence; The updated monkey constraint feature sequence is input into the structural perturbation gating module to replace the constraint feature sequence. The channel correlation is recalculated based on the updated constraint feature sequence and time-series feature sequence; The gating coefficients are recalculated based on the channel correlation, and channel weighting is performed on the time-series feature sequences based on the gating coefficients to generate updated gating feature sequences.

Images