On-Premise Multimodal Cross-Validation System and Method for Time Series Anomaly Detection

Abstract

The objective of the present invention is to drastically reduce false positives and false negatives, which are limitations of existing single-model-based analysis, by converting time-series data into heterogeneous modalities of text descriptions and visual chart images, and by mutually cross-validating the results of analysis independently performed by a natural language processing model and a vision model. A multimodal cross-validation system for detecting anomalies in time-series data according to the present invention, for solving the aforementioned problem, is a system for detecting anomalies in time-series data in a computing environment, and is characterized by comprising: a multimodal conversion unit that receives time-series data as input and converts it into a description in text format and a chart image in visual format; a heterogeneous analysis engine that analyzes the description in text format using a natural language processing model to generate a first analysis result and analyzes the chart image in visual format using a vision model to generate a second analysis result, wherein the two analysis processes are performed as mutually independent parallel processes; a cross-validation consensus unit that collects the first analysis result and the second analysis result and cross-validates them, and derives a final anomaly score through a preset consensus algorithm when a discrepancy between the two results is detected; and a result output unit that determines whether an anomaly exists based on the final anomaly score and outputs explanatory information based on the first analysis result.

Description

Technology Field

[0001] The present invention relates to a technology for detecting anomalies in time series data, and more specifically, to a multimodal cross-validation system and method for detecting anomalies in time series data that converts time series data into heterogeneous modalities of text and images in a security-enhanced on-premise environment, and improves the accuracy and explainability of detection by cross-validating the data using a plurality of large-scale language models (LLM) and visual analysis models. Background Technology

[0003] Across modern industries, technology that monitors time-series data in real time to identify anomalies early—such as in predictive maintenance for smart factory equipment, fraud detection systems (FDS) in the financial sector, and network security monitoring for IT infrastructure—has established itself as a key element in ensuring system stability and efficiency.

[0004] Conventional time series anomaly detection techniques have mainly relied on traditional statistical methods such as ARIMA and Z-Score, or unimodal machine learning and deep learning models that learn only continuous patterns of numerical data, such as Isolation Forest and LSTM (Long Short-Term Memory) Autoencoder.

[0005] While these technologies are effective for detecting simple outliers that deviate from the statistical distribution of data, they had limitations in capturing subtle anomalies occurring within complex and non-linear data flows.

[0006] However, since the aforementioned conventional technologies utilize only single-dimensional information such as numerical data, they have a fatal disadvantage in that problems of false positives and false negatives frequently occur, such as missing high-dimensional anomalies revealed in the contextual meaning or visual chart patterns embedded in the data, or misidentifying normal patterns as abnormalities.

[0007] In addition, recent high-performance AI models require large-scale computational resources and are primarily operated in cloud environments, which raises concerns about the external leakage of sensitive data and network latency issues. Furthermore, due to the inherent black-box nature of deep learning models, there is a problem of lack of explainability, which prevents users from being informed of the specific causes and grounds for anomaly detection.

[0008] Accordingly, there is an urgent need to develop a next-generation anomaly detection system that can innovate accuracy and clearly explain the basis for judgment by fusing and mutually verifying multiple modalities, such as text and images, in a secure local environment. Prior art literature

[0010] Republic of Korea Published Patent Application No. 10-2022-0126014 (Publication Date: September 15, 2022) Republic of Korea Registered Patent Application No. 10-2413958 (Publication Date: June 28, 2022) The problem to be solved

[0011] The objective of the present invention to solve the aforementioned problem is to drastically reduce false positives and false negatives, which are limitations of existing single-model-based analysis, by converting time-series data into heterogeneous modalities such as text descriptions and visual chart images in an on-premise environment and cross-verifying the results of analysis independently performed by a natural language processing model and a vision model.

[0012] Furthermore, the objective of the present invention is to optimize AI models requiring large-scale computation for a local GPU environment through quantization and dynamic resource allocation techniques, thereby fundamentally blocking the threat of data leakage caused by external cloud transmission and securing real-time processing performance without network latency.

[0013] Furthermore, the objective of the present invention is to provide an adaptive detection environment that goes beyond simple anomaly detection results, utilizes the reasoning capabilities of a large-scale language model to explain the causes and grounds for anomaly occurrences in natural language, and automatically updates the judgment threshold based on changes in the statistical distribution of time-series data. means of solving the problem

[0015] A multimodal cross-validation system for detecting anomalies in time-series data according to the present invention, for solving the aforementioned problem, is a system for detecting anomalies in time-series data in an on-premise environment, and is characterized by comprising: a multimodal conversion unit that receives time-series data and converts it into a description in text format and a chart image in visual format; a heterogeneous analysis engine that analyzes the description in text format using a natural language processing model to generate a first analysis result and analyzes the chart image in visual format using a vision model to generate a second analysis result, wherein the two analysis processes are performed as mutually independent parallel processes; a cross-validation consensus unit that collects the first analysis result and the second analysis result and cross-validates them, and derives a final anomaly score through a preset consensus algorithm when a discrepancy between the two results is detected; and a result output unit that determines whether an anomaly exists based on the final anomaly score and outputs explanatory information based on the first analysis result.

[0016] In addition, in a multimodal cross-validation system for detecting anomalies in time series data according to the present invention, the natural language processing model is composed of a plurality of large-scale language models (LLM), and the plurality of large-scale language models include at least one of a pattern analysis model that analyzes the trend and periodicity of time series data, a statistical analysis model that interprets statistical indicators, and a domain analysis model that applies industrial domain knowledge.

[0017] In addition, in a multimodal cross-validation system for detecting anomalies in time series data according to the present invention, the vision model of the heterogeneous analysis engine includes a convolutional neural network (CNN)-based model that extracts local features of the chart image and a vision transformer (ViT)-based model that identifies global context, and is characterized by generating the second analysis result by ensembling the output values ​​of the two models.

[0018] In addition, in a multimodal cross-validation system for detecting anomalies in time series data according to the present invention, the cross-validation consensus unit is characterized by executing an arbitration logic that, when the difference between the first analysis result and the second analysis result exceeds a threshold and is determined to be a discrepancy, recalculates a reliability weight based on the past prediction success rate of each model or performs a causal analysis on the discrepancy to select or correct one of the conflicting results.

[0019] In addition, the multimodal cross-validation system for detecting anomalies in time-series data according to the present invention is characterized by further including a quantization module that operates in a local GPU environment and dynamically adjusts the batch size of the natural language processing model or vision model or changes the computational precision according to the real-time available memory capacity.

[0020] In addition, in a multimodal cross-validation system for detecting anomalies in time series data according to the present invention, the cross-validation consensus unit includes a dynamic threshold adjustment unit that updates a threshold value serving as a criterion for anomaly determination in real time by reflecting a change in distribution according to the temporal flow of the time series data, and the result output unit is characterized by generating and providing the cause and basis of the anomaly occurrence in natural language by utilizing the text description, which is the first analysis result.

[0022] Furthermore, the multimodal cross-validation method for detecting anomalies in time-series data according to the present invention for solving the aforementioned problem comprises: (a) a step of preprocessing input time-series data to simultaneously convert it into a description in text format and a chart image in visual format; (b) a heterogeneous independent analysis step of inputting the description in text format into a natural language processing model to generate a first analysis result, and in parallel inputting the chart image in visual format into a vision model to generate a second analysis result; (c) a cross-validation consensus step of collecting and comparing the first analysis result and the second analysis result, and if a discrepancy between the two results is detected, executing a pre-set consensus algorithm to derive a final anomaly score; and (d) a step of determining whether an anomaly exists based on the final anomaly score and generating and outputting natural language description information based on the first analysis result.

[0023] In addition, in a multimodal cross-validation method for detecting anomalies in time series data according to the present invention, the step of generating the first analysis result of step (b) comprises: (b1) a step of interpreting the trend and periodicity of the time series data using a language model dedicated to pattern analysis; (b2) a step of interpreting the statistical distribution characteristics of the data using a language model dedicated to statistical interpretation; and (b3) a step of inferring context using a domain-specific language model by referring to a knowledge base of the relevant industry domain; and is characterized by generating the first analysis result by synthesizing the results of steps (b1) to (b3).

[0024] In addition, in a multimodal cross-validation method for detecting anomalies in time series data according to the present invention, the step of generating the second analysis result of step (b) comprises: a step of extracting local anomaly features within the chart image using a convolutional neural network (CNN); a step of identifying the global contextual flow of the entire chart image using a vision transformer (ViT); and a step of producing the second analysis result by ensembling the local features and the global contextual flow by assigning weights to each of them.

[0025] In addition, in the multimodal cross-validation method for detecting anomalies in time series data according to the present invention, the cross-validation consensus step comprises: (c1) a step of calculating the difference between the first analysis result and the second analysis result to determine whether a threshold is exceeded; (c2) a step of calculating a reliability score by querying the past prediction history of each model when a discrepancy is determined to be exceeded by the threshold; and (c3) a step of assigning a weight to the analysis result of the model with the higher reliability score or re-evaluating the text logic structure of the first analysis result to correct the final consensus score.

[0026] In addition, the multimodal cross-validation method for detecting anomalies in time series data according to the present invention is characterized by including, prior to step (b), a step of checking the GPU memory availability of a local system, and if the GPU memory availability is less than a threshold value, a step of performing quantization to lower the computational precision of the natural language processing model or vision model, or reducing the batch size of the input data.

[0027] In addition, in a multimodal cross-validation method for detecting anomalies in time series data according to the present invention, the step of determining whether an anomaly exists in step (d) comprises: (d1) a step of calculating the moving average and standard deviation of the final anomaly scores calculated during a preset recent period; (d2) a step of updating a dynamic threshold that reflects the distribution change of the time series data based on the calculated moving average and standard deviation; and (d3) a step of determining whether an anomaly occurs by comparing whether the final anomaly score derived in step (c) exceeds the updated dynamic threshold.

[0029] In addition, the computer program according to the present invention for solving the above-described problem is characterized by being stored in a storage medium to execute a multimodal cross-verification method for detecting anomalies in time-series data described above on a computer. Effects of the invention

[0031] According to the present invention, numerical data is converted into heterogeneous modalities such as text descriptions and visual chart images, and by having a natural language processing model and a vision model independently analyze and then cross-validate them, anomalies that are easily missed by a single model can be captured, thereby lowering the failure rate and dramatically improving the accuracy of detection.

[0032] In addition, according to the present invention, all data processing and analysis are configured to be performed in an on-premise environment without passing through an external cloud server, thereby fundamentally blocking the risk of external leakage of sensitive corporate data, such as financial transaction information or manufacturing process data, and providing high security.

[0033] In addition, according to the present invention, the black-box problem in which existing deep learning-based anomaly detection models fail to provide reasons for results is resolved, and rapid follow-up measures can be supported by explaining the cause and logical basis of anomalies in natural language that is easy for users to understand, based on the reasoning ability of a large-scale language model.

[0034] In addition, according to the present invention, by simultaneously detecting rapid local changes through a convolutional neural network (CNN) and identifying global trend flows through a vision transformer (ViT), it is possible to precisely distinguish and detect various types of abnormal patterns, such as instantaneous spike noise and gradually progressing trend deviations.

[0035] In addition, according to the present invention, by using an ensemble of multiple specialized language models that perform different roles such as pattern analysis, statistical interpretation, and domain context inference, it is possible to provide a cross-validation system and method capable of high-level anomaly judgment that considers the specificity and context of the relevant industrial field beyond simple numerical anomalies.

[0036] In addition, according to the present invention, by monitoring changes in the statistical distribution of time-series data or concept drift phenomena in real time and dynamically and automatically updating threshold values ​​that serve as criteria for anomaly detection accordingly, it is possible to prevent system performance degradation due to changes in the data environment and minimize maintenance requirements.

[0037] In addition, according to the present invention, when analysis results between models conflict, the possibility of false positives caused by bias or errors in a specific model can be effectively suppressed by deriving an optimal conclusion through an intelligent consensus algorithm that includes reliability evaluation based on past prediction success rates and causal relationship analysis.

[0038] In addition, according to the present invention, by applying a quantization technique that adjusts the computational precision of a model according to the real-time available memory status or a dynamic resource allocation technique that adjusts the batch size, large-scale AI models can be stably operated even in on-premises environments or edge devices where high-performance computational resources are limited. Brief explanation of the drawing

[0040] FIG. 1 is a diagram showing the detailed block configuration of a multimodal cross-validation system for detecting anomalies in time series data according to an embodiment of the present invention. FIG. 2 is a flowchart showing the flow of a multimodal cross-validation method for detecting anomalies in time series data according to an embodiment of the present invention. FIG. 3 is a diagram illustrating the entire data flow from the input of time series data to the output of the final anomaly detection result, to which a multimodal cross-validation system and method for detecting anomalies in time series data according to an embodiment of the present invention are applied. FIG. 4 is a diagram illustrating a data integration mechanism at the fusion point of numerical data and visual chart data to which a multimodal cross-validation system and method for detecting anomalies in time series data according to an embodiment of the present invention are applied. FIGS. 5 to 7 are detailed flowcharts of a cross-validation and fusion algorithm applying a multimodal cross-validation system and method for detecting anomalies in time series data according to an embodiment of the present invention. Specific details for implementing the invention

[0041] Before describing the present invention in detail, it should be understood that the terms and words used in this specification should not be interpreted as being limited to their ordinary or dictionary meanings, and that the inventor of the present invention may appropriately define and use the concepts of various terms to best describe their invention, and furthermore, that these terms and words should be interpreted in a meaning and concept consistent with the technical spirit of the present invention.

[0042] In other words, it should be understood that the terms used in this specification are used merely to describe preferred embodiments of the present invention and are not intended to specifically limit the content of the present invention, and that these terms are defined in consideration of various possibilities of the present invention.

[0043] In addition, it should be noted that in this specification, singular expressions may include plural expressions unless the context clearly indicates a different meaning, and that even if they are expressed in a similarly plural form, they may include the meaning of the singular.

[0044] Throughout this specification, where it is stated that a component "includes" another component, unless specifically stated otherwise, this may mean that it does not exclude any other component but may include any other component.

[0045] Furthermore, it should be noted that in cases where it is stated that a component "exists inside or is installed in connection with" another component, this component may be installed in direct connection or contact with the other component, or it may be installed at a certain distance apart, and in the case where it is installed at a certain distance apart, there may be a third component or means for fixing or connecting the component to the other component, and a description of this third component or means may be omitted.

[0046] On the other hand, if it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there is no third component or means.

[0047] Likewise, other expressions describing the relationship between each component, such as “between” and “right between”, or “adjacent to” and “directly adjacent to”, should be interpreted as having the same intent.

[0048] In addition, it should be understood that in this specification, terms such as “one side,” “other side,” “one side,” “other side,” “first,” “second,” etc., are used to clearly distinguish one component from another component, and that the meaning of the component is not restricted by such terms.

[0049] In addition, position-related terms such as "up," "down," "left," and "right" used in this specification should be understood as indicating the relative position of the corresponding component in the drawing, and unless an absolute position is specified, these position-related terms should not be understood as referring to an absolute position.

[0050] Furthermore, in specifying the reference numerals for each component of each drawing in this specification, the same component has the same reference numeral even if it is shown in different drawings; that is, the same reference numeral throughout the specification indicates the same component.

[0051] In the drawings attached to this specification, the size, position, connection relationships, etc., of each component constituting the present invention may be described in a partially exaggerated, reduced, or omitted manner for the convenience of explanation or to sufficiently clearly convey the concept of the present invention, and therefore, the proportions or scale may not be strictly accurate.

[0052] In addition, in the following description of the present invention, detailed descriptions of components, such as prior art and known technology, that are deemed to unnecessarily obscure the essence of the present invention may be omitted.

[0054] Preferred embodiments of the present invention will be described in detail below with reference to the drawings.

[0055] FIG. 1 is a diagram showing the detailed block configuration of a multimodal cross-verification system (100) for detecting anomalies in time series data according to an embodiment of the present invention.

[0056] As shown in FIG. 1, a multimodal cross-verification system (100) for detecting anomalies in time series data according to an embodiment of the present invention is a system for detecting anomalies in time series data in an on-premise environment, and may be configured to include a multimodal conversion unit (110) that receives time series data and converts it into a description in text format and a chart image in visual format; a heterogeneous analysis engine that analyzes the description in text format with a natural language processing model to generate a first analysis result and analyzes the chart image in visual format with a vision model to generate a second analysis result, wherein the two analysis processes are performed as mutually independent parallel processes; a cross-verification consensus unit (150) that collects the first analysis result and the second analysis result and cross-verifies them, and if a discrepancy between the two results is detected, derives a final anomaly score through a pre-set consensus algorithm; and a result output unit (170) that determines whether there is an anomaly based on the final anomaly score and outputs explanatory information based on the first analysis result.

[0057] A multimodal cross-verification system (100) for detecting anomalies in time series data according to an embodiment of the present invention is proposed to overcome the limitations of existing anomaly detection technology that relied on a single modality (unimodal).

[0058] Existing numerical data-centric analysis makes it difficult to grasp the contextual meaning of the data, and false positives are frequent due to the bias of a single model. Accordingly, the present invention can dramatically improve the accuracy and reliability of detection by converting time-series data into different dimensions, namely 'human language (Text)' and 'visual patterns (Image),' allowing heterogeneous AI models to independently analyze and mutually verify the data.

[0059] In particular, the multimodal cross-validation system (100) for detecting anomalies in time series data according to an embodiment of the present invention has technical significance in that it introduces a consensus algorithm that adjusts differences in opinion between models to suppress the hallucination phenomenon of AI and provide explainable results.

[0060] Furthermore, the 'on-premise environment' of the multimodal cross-verification system and method for detecting anomalies in time-series data according to the embodiments of the present invention described below refers to a computing infrastructure in which the entire process of data input, transformation, analysis, and verification is performed independently within a user's internal network or physical server without integration with an external cloud server or API, in order to ensure data security and minimize processing latency.

[0061] This is not limited to physical server rooms or in-house data centers but encompasses all high-performance computing environments capable of handling high-speed inference of large-scale language models (LLM) and vision models locally; specifically, it can be interpreted as a broad local computing environment that includes not only local GPUs (Graphics Processing Units) but also internal workstations, edge devices, or self-built server farms equipped with TPUs (Tensor Processing Units), NPUs (Neural Processing Units), and other AI-dedicated hardware accelerators.

[0062] More specifically, as shown in FIG. 1, the multi-modal transformation unit (110) may be a module that preprocesses and converts input time-series raw data into two different forms that can be interpreted by an artificial intelligence model, namely natural language description and chart image.

[0063] Here, the multimodal conversion unit (110) can perform the following functions.

[0064] 1) Data preprocessing: Data quality can be ensured by performing missing value interpolation, outlier filtering, and normalization.

[0065] 2) Text conversion: Numerical data can be converted into a description in the form of a text prompt that includes statistical summaries (mean, variance, etc.) and trend features (rising, falling, periodicity).

[0066] 3) Image conversion: Numerical data can be rendered into visual chart images such as line graphs, heatmaps, and bar graphs, and resized to fit the model input specifications.

[0067] In this way, the multimodal conversion unit (110) can achieve real-time performance by applying a sliding window technique to divide a continuous data stream into sections, and then operating a statistical calculation algorithm and an image rendering engine in parallel for each window.

[0068] A heterogeneous analysis engine is a hybrid analysis engine that combines natural language processing (NLP) models and computer vision models with different architectures and learning methods.

[0069] Such a heterogeneous analysis engine can have the following functions.

[0070] 1) Independent parallel processing: Text analysis and image analysis can be performed simultaneously in independent GPU memory spaces or processes without affecting each other's results.

[0071] 2) Natural Language Processing Model (LLM) Analysis: Based on the input descriptive text, contextual anomalies and logical contradictions of the time series are inferred to generate the first analysis result. Trends, seasonality, domain knowledge, etc., may be reflected in this.

[0072] 3) Vision model analysis: Analyzes pixel information of the chart image to detect morphological anomalies such as sharp spikes, dips, and trend deviations, and generates a second analysis result.

[0073] In this way, the NLP part performs semantic interpretation of time-series data by utilizing a Transformer-based Large Language Model (LLM).

[0074] In addition, the Vision part quantifies visual patterns by ensembling a Convolutional Neural Network (CNN), which is strong in extracting local features, and a Vision Transformer (ViT), which is strong in recognizing global patterns.

[0075] In addition, the above natural language processing model is composed of a plurality of large-scale language models (LLMs), and the plurality of large-scale language models may include at least one of a pattern analysis model that analyzes trends and periodicities of time-series data, a statistical analysis model that interprets statistical indicators, and a domain analysis model that applies industrial domain knowledge.

[0076] Here, the Pattern Analysis Model is a language model specialized in describing and analyzing the geometric shapes and temporal flows of time-series data using natural language.

[0077] The pattern analysis model identifies upward / downward trends, seasonal patterns, and cyclicity in numerical data. For example, it generates a descriptive statement such as, "A pattern of showing a gradual upward trend over the past 3 months followed by a decline every weekend was observed."

[0078] A statistical interpretation model is a model that evaluates the numerical distribution and statistical peculiarities of data.

[0079] This model calculates and interprets statistical indicators such as mean, variance, standard deviation, skewness, and kurtosis. Beyond simple numerical calculations, it can make statistical judgments such as, "The current volatility is statistically significant because it exceeds 3 sigma (3σ) relative to the historical average."

[0080] A Domain Analysis Model is a model that fine-tunes the knowledge base of a specific industry sector where the data was generated, such as finance, manufacturing, or healthcare, or receives it as a prompt context.

[0081] This model interprets the significance of discovered patterns or statistical figures within the respective industrial context. For instance, it plays a role in interpreting the same 'sudden drop' pattern differently: as a 'crash' in finance, and as a 'power cut' in manufacturing sensors.

[0082] More specifically, as shown in FIG. 1, the multimodal conversion unit (110) serves as an input to the system and performs the function of converting received time-series raw data into a processable form and distributing it to the subsequent end.

[0083] In addition, the multimodal conversion unit (110) can simultaneously operate a text generation algorithm and an image rendering engine for a single time series data window.

[0084] The generated 'text description' is branched and transmitted to the input of the natural language processing part (131), and the 'chart image' is branched and transmitted to the input of the vision processing part (133). This means that heterogeneous data enters the parallel analysis stage while maintaining temporal synchronization.

[0085] And, as shown in FIG. 1, the vision model of the heterogeneous analysis engine (130) may include a convolutional neural network (CNN)-based model that extracts local features of a chart image and a vision transformer (ViT)-based model that grasps global context, and may generate the second analysis result by ensembling the output values ​​of the two models.

[0086] When converting time-series data into images for analysis, it is necessary to simultaneously identify local spikes / dips and overall trends. However, while CNNs are strong at extracting local features, they tend to miss long-term correlations; conversely, Transformers are strong at identifying global relationships but tend to be weak at local details.

[0087] Accordingly, in the embodiment of the present invention, a hybrid architecture combining the advantages of these two models is applied to seamlessly capture subtle anomaly signs and macroscopic anomaly flows.

[0088] Specifically, a Convolutional Neural Network (CNN)-based pattern classifier is a model that generates a feature map by applying a filter to the local receptive field of an image.

[0089] Through this, it is possible to precisely detect high-frequency anomaly patterns occurring within a short period of time, such as sudden pixel-level changes within a chart image, such as sudden spikes, dips, and noise.

[0090] A Vision Transformer (ViT)-based complex pattern detector can be a model that divides an image into patches and learns the relationships between patches by applying a self-attention mechanism.

[0091] This allows for the identification of long-term dependencies extending from the beginning to the end of the chart, as well as trend drifts that gradually change or abnormal signs that appear normal locally but do not align with the overall cycle.

[0092] Then, the process of generating the ensemble and second analysis results may be a process of combining the feature vectors or outlier probability values ​​produced by CNN and ViT, respectively.

[0093] Through this, the output values ​​of the two models can be concatenated or weighted averaged to calculate a final Visual Anomaly Score that compensates for blind spots that a single model might miss.

[0094] And, as shown in FIG. 1, the heterogeneous analysis engine (130) may be structured such that the natural language processing part (131) and the vision processing part (133) process data in independent paths, while internally combining the results of the detailed modules to generate a single representative value.

[0095] The text description received from the multimodal conversion unit (110) can be simultaneously input into the LLM pattern interpretation module (131a), the LLM statistical interpretation module (131b), and the domain context inference module (131c).

[0096] Each module independently infers trends, statistics, and domain fit, and the results can be aggregated into the 'first analysis result generation module (131d)'.

[0097] The first analysis result generation module (131d) can aggregate the inference results (text and probability values) of each module to generate a text-based anomaly score, which is the 'first analysis result', and transmit it to the cross-verification consensus unit (150).

[0098] And, as shown in FIG. 1, the chart image received from the multimodal conversion unit (110) can be input in parallel to the local anomaly extraction module (133a) (CNN-based) and the global context interpretation module (133b) (ViT-based).

[0099] The local anomaly extraction module (133a) analyzes abrupt changes (Edge / Spike) in the image, and the global context interpretation module (133b) analyzes the overall shape (Shape / Trend), and these results can be transmitted to the 'second analysis result generation module (133c)'.

[0100] The second analysis result generation module (133c) can ensemble the feature maps or scores of the two modules to generate an image-based abnormal score, which is the ‘second analysis result,’ and transmit it to the cross-validation consensus unit (150).

[0101] The Cross-Validation Consensus Unit (150) is a decision module that collects analysis results from different perspectives produced by the heterogeneous analysis engine (130) and mediates the results when conflicting results occur to derive an optimal conclusion.

[0102] The cross-validation consensus unit (150) performs a disagreement detection function by first calculating an abnormal score variance or Euclidean distance between a first analysis result (text-based) and a second analysis result (image-based) to determine whether there is a discrepancy.

[0103] Secondly, if a discrepancy is detected, a consensus algorithm is executed to calculate a final consensus score by applying confidence weights based on the historical prediction accuracy of each model or through contribution evaluation methods such as the Shapley Value.

[0104] In this way, the cross-validation consensus unit (150) can apply an adaptive consensus mechanism that incorporates more of the opinions of the model showing more robust performance in specific situations (e.g., noisy data) by using a weighted voting or Bayesian model averaging technique rather than a simple averaging method.

[0105] Additionally, the cross-validation consensus unit (150) may execute an arbitration logic to select or correct one of the conflicting results by recalculating the reliability weight based on the past prediction success rate of each model or performing a causal analysis on the discrepancy when the difference between the first analysis result and the second analysis result exceeds a threshold and is determined to be a discrepancy.

[0106] In this way, the cross-validation consensus unit (150) applied to the system according to one embodiment of the present invention can be equipped with 'Intelligent Arbitration Logic' that actively resolves differences in opinion between models, going beyond simply arithmetically averaging multiple model results.

[0107] Specifically, if the difference between the text-based first analysis result and the image-based second analysis result exceeds a preset threshold and is determined to be in a 'Disagreement' state, the system seeks a point of agreement through two tracks.

[0108] First, as a statistical approach, the database of historical time series prediction success rates for each model is consulted, and weights are recalculated and assigned to models that demonstrated higher accuracy in the current data pattern.

[0109] Second, 'Causal Analysis' can be performed as an analytical approach.

[0110] This is a process of verifying whether logical reasoning included in text descriptions (e.g., "increase in costs due to rising raw material prices") causally matches specific sections of chart images (e.g., changes in the slope of the cost curve). Through this, the reliability of the system can be maximized by excluding false positives caused by simple noise and selecting or correcting results with solid logical grounds as the final abnormal score.

[0111] In addition, the multimodal cross-validation system (100) for detecting anomalies in time series data according to an embodiment of the present invention is operated in a local GPU environment and may further include a quantization module that dynamically adjusts the batch size of the natural language processing model or vision model or changes the computational precision according to the real-time available memory capacity.

[0112] As such, embodiments of the present invention may include a 'Resource Optimization Module' to run high-performance AI in a secure on-premise environment without relying on the cloud.

[0113] Since large-scale language models (LLMs) and vision models require high computational power, stable operation within limited local hardware resources is essential.

[0114] Accordingly, in this embodiment, the available capacity of the GPU memory (VRAM) is monitored in real time, and if signs of memory shortage are detected, the 'quantization' process is immediately triggered.

[0115] This is a technique that dramatically reduces memory usage by lowering the model's parameter precision from FP32 (32-bit floating-point) to FP16 or INT8 (8-bit integer).

[0116] In addition, when data inflow surges, the 'batch size,' which is the unit of data to be processed, is dynamically adjusted to balance throughput and memory usage, thereby providing technical stability that enables the continuous operation of real-time anomaly detection services without system crashes.

[0117] More specifically, as shown in FIG. 1, the cross-validation consensus unit (150) is a junction where two output values ​​(first and second analysis results) of the heterogeneous analysis engine (130) meet, and can derive a final consensus through sequential data processing.

[0118] The difference value judgment module (151) receives the first and second analysis results from the natural language processing part (131) and the vision processing part (133), respectively. It can calculate the Euclidean distance or deviation between the two result values ​​and determine whether this deviates from a preset allowable range. The judgment result (signal of discrepancy) is transmitted to the reliability calculation module (153).

[0119] The reliability calculation module (153) is activated when a mismatch signal is received, and by referring to the past performance metadata of each analysis model, it can generate a 'reliability vector' that assigns a higher weight to the model that is more reliable in the current situation and transmit it to the consensus score correction module (155).

[0120] The consensus score correction module (155) performs a weighted average of the first analysis result and the second analysis result based on the received confidence vector to derive a 'final consensus score' with conflicts resolved. This value can be transmitted to the result output unit (170).

[0121] The result output unit (170) may be an interface module that determines whether there is an anomaly based on the final calculated sum score and conveys the result and cause to the user.

[0122] The result output unit (170) compares the final abnormal score with a dynamic threshold to determine a grade such as 'normal', 'caution', 'warning', or 'danger', and, rather than simply showing the numerical result, can generate and output a human-understandable explanation (Natural Language Explanation) such as "a rapid rise occurred unlike the downward trend of the last 3 days and was detected as an abnormality" based on the analysis by the natural language processing model.

[0123] In this way, the result output unit (170) can minimize false positives by adopting a method in which the dynamic threshold is automatically updated in real time according to changes in the data distribution based on the moving average and standard deviation. In addition, the basis for analysis can be synthesized into text by utilizing the generation capabilities of the LLM.

[0124] More specifically, as shown in FIG. 1, the result output unit (170) may be a feedback loop structure that goes beyond simply displaying results and learns past data to update judgment criteria on its own.

[0125] The outlier score statistics module (171) can accumulate the final consensus score received from the cross-validation consensus unit (150) in a time series database, calculate the score distribution (moving average, standard deviation) over the most recent N time points, and transmit it to the dynamic threshold update module (173).

[0126] The dynamic threshold update module (173) can calculate an 'update threshold' optimized for data variability at the current time based on the received statistical data. This can perform an adaptive operation in which the threshold is raised when data noise becomes severe and the threshold is lowered in stable periods.

[0127] The update threshold determination module (175) compares the 'final consensus score' input in real time from the consensus score correction module (155) with the 'update threshold' calculated by the dynamic threshold update module (173), and if the final score exceeds the threshold, it can be determined as an anomaly.

[0128] When the abnormal result output module (177) receives a confirmed abnormality determination signal, it can call natural language explanation data from the first analysis result generation module of the heterogeneous analysis engine (130) and visually provide the determination result and explanation information to the user.

[0129] In addition, the cross-validation consensus unit (150) applied to the embodiment of the present invention includes a dynamic threshold adjustment unit that updates the threshold value serving as the criterion for anomaly determination in real time by reflecting the distribution change according to the temporal flow of the time series data, and the result output unit (170) can generate and provide the cause and basis of the anomaly occurrence in natural language by utilizing the text description, which is the first analysis result.

[0130] In this way, the result output unit (170) applied to the multimodal cross-verification system (100) for detecting anomalies in time series data according to an embodiment of the present invention can provide an 'adaptive explanation interface' that flexibly responds to changes in the characteristics of the time series data and presents clear grounds for judgment to the user.

[0131] Time series data is prone to 'concept drift,' a phenomenon where the distribution or trend changes over time, so using a fixed threshold can lead to false positives or missing detections.

[0132] Accordingly, the 'dynamic threshold adjustment unit' can actively update the threshold for anomaly detection by reflecting the moving average and standard deviation of recent data in real time.

[0133] Furthermore, when an anomaly is finally detected, the result output unit (170) can not only send a warning that "anomaly has occurred," but also utilize the text description (first analysis result) previously generated by the natural language processing model to generate specific "causes and grounds" in natural language form, such as "traffic patterns during the weekend decreased rapidly unlike usual and were judged to be an anomaly," and provide them to the user.

[0134] This implements Explainable AI (XAI) capabilities that allow users to intuitively understand the reasons behind the AI's decisions, enabling rapid decision-making and follow-up actions.

[0136] FIG. 2 is a flowchart showing the flow of a multimodal cross-validation method for detecting anomalies in time series data according to an embodiment of the present invention.

[0137] As shown in FIG. 2, a multimodal cross-validation method for detecting anomalies in time-series data according to an embodiment of the present invention is a multimodal cross-validation method for detecting anomalies in time-series data in an on-premise environment, comprising: (a) a step of preprocessing input time-series data to simultaneously convert it into a description in text format and a chart image in visual format (S100); (b) a heterogeneous independent analysis step (S200) in which the description in text format is input into a natural language processing model to generate a first analysis result, and in parallel with this, the chart image in visual format is input into a vision model to generate a second analysis result; (c) a cross-validation consensus step (S300) in which the first analysis result and the second analysis result are collected and compared, and if a discrepancy between the two results is detected, a preset consensus algorithm is executed to derive a final anomaly score; and (d) determining whether an anomaly exists based on the final anomaly score, and generating and outputting natural language description information based on the first analysis result. It can be configured to include step (S400).

[0138] More specifically, as shown in FIG. 2, step (a) (S100) may be a step of performing a duplication process to preprocess the input time series data stream to remove noise, and then simultaneously convert it into two different modalities: a 'text description' describing numerical features and a 'chart image' representing visual patterns.

[0139] (b) Step (S200) is a heterogeneous independent analysis step, wherein the converted text description is input into a natural language processing model (NLP) and the chart image is input into a vision model, and the two models produce a text-based first analysis result and an image-based second analysis result through a parallel computation process without interference with each other.

[0140] Here, the text format descriptive analysis process performed in step (b) (S200) does not rely on a single model, but is performed from multiple angles through multiple specialized language models with different analysis perspectives.

[0141] Specifically, in step (b1), a 'language model dedicated to pattern analysis' can be operated to interpret the unique shape features of the time series, such as the upward / downward trend or seasonality of the time series data.

[0142] Then, in step (b2), the numerical significance of statistical distribution characteristics such as the mean, variance, and skewness of the data is analyzed through a 'language model dedicated to statistical interpretation'.

[0143] At the same time or sequentially, in step (b3), a 'domain-specific language model' that references the knowledge base of the industry domain to which the data belongs, such as finance, manufacturing, and healthcare, can be used to infer in depth what context the currently occurring pattern signifies in the relevant industry.

[0144] Finally, the system can comprehensively interpret the complex meaning hidden behind the numerical data by aggregating the individual analysis results produced in steps (b1) to (b3) above to generate a first analysis result.

[0145] And, the visual format chart image analysis process of step (b) (S200) may be a step of adopting a hybrid architecture that combines two neural networks with different inductive biases to simultaneously capture local features and global context.

[0146] First, by using a convolutional neural network (CNN) that effectively processes spatial associations between pixels, local anomaly features in high-frequency bands, such as sharp spikes, dips, or noise within a chart image, can be precisely extracted.

[0147] At the same time, by utilizing a Vision Transformer (ViT) that processes images in patch units and applies a self-attention mechanism, it is possible to identify global contextual flows, such as long-term dependencies extending from the chart's start point to the end point and overall trend deviations.

[0148] Then, the system can produce a high-precision second analysis result that considers both minute fluctuations and macroscopic flows by performing an ensemble process of concatenating or weighted averaging the extracted local features and global contextual flows by assigning weights determined according to the characteristics of the data or the noise level to each, respectively.

[0149] In addition, a multimodal cross-validation method for detecting anomalies in time series data according to another embodiment of the present invention may include a step of performing a preliminary resource check and optimization step prior to the full-scale analysis process of step (b) (S200) in order to stably operate an AI model requiring high-performance computation on a local system (On-premise System) or edge device with limited resources.

[0150] Specifically, the system monitors the real-time status of GPU memory (VRAM) to check the available capacity, and if a 'resource shortage' situation is detected where this figure falls below the minimum threshold required for model operation, it can take immediate lightweighting measures.

[0151] The measures taken at this time include two strategies. First, through quantization, the precision of the model's weight operations can be reduced from the existing 32-bit floating-point (FP32) to 16-bit (FP16) or 8-bit integer (INT8), thereby dramatically reducing memory usage.

[0152] Second, instantaneous peak memory usage can be reduced by decreasing the batch size, which is the unit of data bundle processed at once in the input data queue, and through this dynamic control, the system can guarantee continuous anomaly detection services without memory overflow (OOM) errors.

[0153] And, as shown in FIG. 2, step (c) (S300) may be a step of executing an arbitration process to derive an optimal consensus score by triggering a pre-set consensus algorithm including weight adjustment based on reliability evaluation or causal inference logic when a discrepancy between two analysis results is detected where the deviation between the results exceeds an allowable range.

[0154] Here, the cross-validation consensus process of step (c) (S300) can be performed according to a step-by-step arbitration protocol to logically resolve differences in opinion between heterogeneous models.

[0155] First, in step (c1), a quantitative difference value (e.g., Euclidean distance or score deviation) between the text-based first analysis result and the image-based second analysis result is calculated, and a 'disagreement' situation can be identified by determining whether this value exceeds a preset allowable threshold.

[0156] If a discrepancy is determined, the system enters step (c2) to query the past prediction history stored in the metadata of each model and evaluates which model showed a higher hit rate in similar data patterns to calculate a confidence score for each model.

[0157] In the subsequent (c3) step, a consensus can be found by weighting the results of the model with the higher calculated reliability score, or by performing an in-depth verification to re-evaluate the causality and logical contradictions within the text description, which is the result of the first analysis, thereby correcting the final consensus score and deriving an optimal anomaly detection result based on logical validity rather than a simple numerical average.

[0158] (d) By comparing the derived final consensus score with a dynamic threshold to make a final determination of whether an anomaly exists, and by generating and outputting a natural language report that explains the cause and basis of the anomaly occurrence using text information included in the first analysis result, accurate detection and intuitive explanation can be provided simultaneously.

[0159] In addition, in the multimodal cross-verification method for detecting anomalies in time series data according to an embodiment of the present invention, the process of determining whether an anomaly exists in step (d) (S400) may follow an adaptive determination mechanism that reflects the dynamic changes in time series data in real time, rather than a fixed reference value.

[0160] First, in step (d1), the moving average and standard deviation are calculated in real time for the set of final outlier scores calculated during a recent sliding window, thereby identifying the central tendency and variability of the data at the current point in time.

[0161] In the subsequent (d2) step, the dynamic threshold, which is the baseline for anomaly detection, can be updated based on the calculated moving average and standard deviation (e.g., mean + K × standard deviation) to reflect changes in data distribution or concept drift.

[0162] This has the effect of preventing false positives by raising the threshold in noisy sections and preventing false negatives by lowering the threshold in stable sections.

[0163] Finally, in step (d3), by comparing whether the current final anomaly score derived in step (c) (S300) exceeds the updated dynamic threshold, it is possible to finally determine whether it is a true anomaly with statistical significance.

[0164] In addition, another embodiment of the present invention may feature a computer program stored in a storage medium to execute a multimodal cross-verification method for detecting anomalies in time-series data on a computer.

[0165] In addition, the program applied to the multimodal cross-validation method for detecting anomalies in time-series data according to one embodiment of the present invention may be implemented as computer-readable code on a computer-readable recording medium. The codes and code segments implementing the above program can be easily inferred by a computer programmer in the field.

[0166] Here, a computer-readable recording medium may include any type of recording device in which data that can be read by a computer system is stored. Examples of computer-readable recording media may include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, etc. Additionally, computer-readable recording media may be distributed across networked computer systems and may be written and executed as computer-readable code in a distributed manner.

[0168] Hereinafter, a multimodal cross-validation process for detecting anomalies in time series data will be described in detail with reference to a drawing illustrating an overall system architecture and algorithm for applying a multimodal cross-validation system (100) and method for detecting anomalies in time series data according to an embodiment of the present invention.

[0169] FIG. 3 is a diagram illustrating the entire data flow from the input of time series data to the final anomaly detection result output, to which the multimodal cross-verification system (100) and method for detecting anomalies in time series data according to an embodiment of the present invention are applied.

[0170] Referring to FIG. 3, a time series anomaly detection process according to an embodiment of the present invention is initiated as raw data including real-time and batch time series data and metadata is input into a 'data preprocessing module'.

[0171] The preprocessed data is converted into visual chart images and numeric data for text analysis, respectively, through the 'Chart Generation Engine' and 'Numerical Data Normalization' modules, forming a bifurcated path.

[0172] The process then enters a secure 'local GPU environment,' where the 'visual chart analysis module' and the 'LLM ensemble engine' operate in parallel.

[0173] At this time, while the 'GPU memory management' module and 'load balancer' control the computational load in real time for the efficient operation of local resources, the visual path extracts local and global patterns through a CNN pattern classifier and a Vision Transformer, and the text path performs multifaceted text inference through pattern analysis LLM, statistical analysis LLM, and domain analysis LLM to produce independent analysis results.

[0174] Subsequently, the independently generated heterogeneous analysis results mentioned above are aggregated into a 'multimodal fusion engine' and enter the integrated analysis stage.

[0175] The fusion engine applies weights based on the reliability of each model through a 'weighted voting system,' or, in the event of conflicts between results, activates a 'disagreement resolution algorithm' to derive an optimal consensus.

[0176] The results of this primary fusion are passed to a 'cross-validation module' and verified through a process of 'time-segmented cross-validation' that considers time-series characteristics and 'dynamic threshold adjustment' that adapts to changes in data distribution.

[0177] Finally, the system determines the 'final anomaly score' that has passed these multi-stage verifications and completes the anomaly detection process with high accuracy and explainability by providing the judgment results and analysis basis to the user through the 'result output' module.

[0179] FIG. 4 is a diagram illustrating a data integration mechanism at the fusion point of numerical data and visual chart data to which a multimodal cross-validation system (100) and method for detecting anomalies in time series data according to an embodiment of the present invention are applied.

[0180] That is, Figure 4 is a detailed flowchart of the 'multimodal fusion and decision-making mechanism' showing how numerical data and visual data are integrated into a single conclusion through their respective analysis paths.

[0181] Referring to FIG. 4, the data integration mechanism according to an embodiment of the present invention has a structure in which a text analysis path and a visual analysis path converge into a 'fusion engine'.

[0182] First, in the text path on the left, after the data is converted into text, it is interpreted in parallel by three specialized language models called LLM 1 (pattern analysis), LLM 2 (statistical analysis), and LLM 3 (domain analysis), producing detailed intermediate results called text-based anomalies, statistics-based anomalies, and context-based anomalies, respectively.

[0183] Simultaneously, in the visual path on the right, after features are extracted from the chart image, they pass through the CNN analysis module and the ViT (Vision Transformer) analysis module to calculate local pattern anomalies and global pattern anomalies, respectively.

[0184] The five heterogeneous outlier scores derived from different perspectives are passed as input values ​​to the 'fusion engine' and enter the integrated analysis stage.

[0185] Subsequently, the 'fusion engine' performs a 'model consistency check' procedure to verify the consistency among the five collected anomaly scores.

[0186] If the deviation between scores is small at this stage and the agreement is determined to be 'high', the system immediately performs a 'weighted average calculation' by applying weights based on the confidence level of each model.

[0187] On the other hand, if the agreement is determined to be 'low' due to large deviations between scores, the system triggers the 'disagreement resolution' module to analyze the causes of conflicts, performs a weighted average after undergoing a correction process that lowers the weights of models with low confidence or reconfirms causal relationships.

[0188] The result generated through these conditional logical operations is ultimately confirmed and output as the 'final anomaly score,' thereby providing highly reliable anomaly detection results that eliminate the possibility of misjudgment by a single model.

[0190] FIGS. 5 to 7 are detailed flowcharts of a cross-validation and fusion algorithm applying a multimodal cross-validation system (100) and a method for detecting anomalies in time series data according to an embodiment of the present invention.

[0191] Figure 5 is a flowchart showing the process of reliability evaluation and weighted average calculation of multiple model results, Figure 6 is a flowchart showing the operation method of the mediation algorithm when a mismatch occurs, and Figure 7 is a flowchart showing the feedback loop of the dynamic threshold adjustment mechanism.

[0192] Referring to FIG. 5, the multimodal fusion engine applied to an embodiment of the present invention initiates a full-scale fusion procedure while receiving LLM ensemble results, visual analysis results, and statistical analysis results calculated from each model.

[0193] The fusion engine performs 'model-specific reliability calculation' for each received result based on past prediction history or data quality indicators.

[0194] The calculated confidence values ​​undergo a verification process in which they are compared with a pre-set standard value called the 'confidence threshold.' If all models pass the threshold, they immediately enter the 'weight application' stage.

[0195] However, if a 'partial failure' situation occurs where the reliability of some models falls below a threshold, the system performs filtering to 'exclude low-reliability models' and applies weights after going through a 're-weighting calculation' process to adjust the remaining valid models so that the sum of the total weights becomes 1.

[0196] Subsequently, the system performs a 'weighted average calculation' by multiplying the confirmed weights by the analysis result scores of each model and summing them up, thereby producing a 'preliminary anomaly score' in which the influence of models with noise or a high probability of misjudgment is excluded.

[0197] The preliminary outlier scores calculated in this way are passed to the 'inter-model agreement verification' stage before final confirmation, which reconfirms whether the results between each model match at a statistically significant level.

[0198] This process goes beyond simple numerical combination and has technical features that dramatically improve the accuracy and stability of anomaly detection by implementing an adaptive fusion mechanism that evaluates the reliability of each analysis path in real time and dynamically reflects it.

[0199] And, as shown in FIG. 6, the model mismatch resolution process applied to the embodiment of the present invention is initiated as soon as a conflict is detected in the 'model-to-model consistency verification' step.

[0200] The fusion engine triggers an in-depth 'discrepancy resolution process' instead of a simple numerical average; first, through the 'model-specific evidence analysis' stage, it backtracks whether the features identified as abnormal by each model are text context or visual patterns.

[0201] Next, in the 'identifying conflicting causes' step, it is determined whether these discrepancies are caused by data noise, model bias, or information asymmetry between modalities.

[0202] Based on the identified cause, the system applies a 'mediation algorithm' to make a logical judgment that increases the weight of a highly reliable model or prioritizes evidence with a clear causal relationship, thereby completing a primary consensus by calculating an 'Adjusted Score' from which bias has been removed.

[0203] Subsequently, the system enters the 'time-division cross-validation' phase to verify the validity of the calculated score.

[0204] Due to the nature of time-series data, the 'training data splitting' process is performed in strict adherence to chronological order to prevent future information from influencing past training (preventing look-ahead bias).

[0205] 'Validation set evaluation' is performed on the partitioned data to calculate 'performance metrics' such as Precision and Recall in real time.

[0206] Then, the system completes an adaptive anomaly detection environment that flexibly responds to data pattern changes (Concept Drift) by performing a 'dynamic threshold adjustment' step that updates the threshold serving as the baseline for anomaly detection by reflecting calculated performance metrics and the current data distribution state.

[0207] And, as shown in FIG. 7, the abnormality determination process applied to the embodiment of the present invention is performed based on a dynamic threshold that varies according to the statistical flow of data rather than a fixed standard.

[0208] The system performs the process of 'calculating a moving average' to identify trend lines from time series data accumulated through 'historical data analysis' and calculating a 'standard deviation-based threshold' to measure volatility.

[0209] In addition, the 'final threshold' most suitable for the current time is determined through a 'seasonality correction application' step that reflects the periodic characteristics of the data.

[0210] Subsequently, through comparison logic, if the final anomaly score exceeds a threshold, it is classified as an 'anomaly detection' to perform 'anomaly grade classification' and 'warning level determination' based on severity; if it is below the threshold, it is judged as 'normal' and the corresponding 'normality confidence' is calculated, thereby establishing a sophisticated judgment system that minimizes false positives.

[0211] The final result generated is provided to the user along with a 'natural language description', 'visual evidence', and 'reliability index' as shown in Fig. 7, and in the embodiment of the present invention, it does not stop there but forms a 'feedback loop' that utilizes the user's input to improve the system.

[0212] When a user provides 'user feedback' (e.g., confirmation of true / false positives) regarding the judgment result, the system utilizes this as training data to perform 'model performance updates'.

[0213] This updated information is fed back to the confidence calculation step of Fig. 5 and is immediately reflected in the 'weight readjustment' of each model, thereby establishing a self-evolving structure in which detection accuracy improves as the operating period increases.

[0215] Although various preferred embodiments of the present invention have been described above with some examples, the descriptions of various embodiments described in the "Specific details for carrying out the invention" section are merely illustrative, and those skilled in the art to which the present invention pertains will understand that the present invention can be modified in various ways or equivalent embodiments can be carried out based on the above description.

[0216] In addition, since the present invention can be implemented in various other forms, the present invention is not limited by the description above. The above description is provided merely to make the disclosure of the present invention complete and to fully inform those skilled in the art of the scope of the present invention, and it should be understood that the present invention is defined only by each claim of the claims. Explanation of the symbols

[0218] 100: Multimodal Cross-Verification System 110 : Multimodal converter 130 : Heterogeneous Analysis Engine 131a: LLM Pattern Interpretation Module 131b: LLM Statistical Interpretation Module 131c: Domain Context Inference Module 131d: 1st Analysis Result Generation Module 133a: Local anomaly extraction module 133b : Global Context Resolution Module 133c : 2nd Analysis Result Generation Module 150 : Cross-validation consensus 151 : Difference value determination module 153 : Reliability Calculation Module 155 : Consensus Score Correction Module 170 : Result Output Section 171 : Outlier Statistics Module 173 : Dynamic Threshold Update Module 175 : Update threshold determination module 177 : Abnormal result output module

Claims

Claim 1 A system for detecting anomalies in time-series data in an on-premise environment, comprising: a multimodal conversion unit that receives time-series data, divides the time-series data into multiple time-series data windows according to a sliding window technique, and simultaneously operates a text generation algorithm and an image rendering engine for each time-series data window to generate a description in text format and a chart image in visual format from the same time-series data window in parallel and in a temporally synchronized state; and a heterogeneous analysis engine that analyzes the description in text format generated from each time-series data window using a natural language processing model to generate a first analysis result, and analyzes the chart image in visual format using a vision model to generate a second analysis result, wherein the two analysis processes are performed as mutually independent parallel processes so that the first analysis result and the second analysis result correspond to the same time-series data window. A cross-validation consensus unit that collects the first analysis result and the second analysis result corresponding to the same time-series data window, calculates the Euclidean distance or deviation between the first analysis result and the second analysis result to determine whether there is a discrepancy between the two results based on whether they fall outside a preset allowable range, and if a discrepancy is determined, generates a confidence vector for each analysis model by referring to past performance metadata including the past prediction history of each analysis model, corrects the first analysis result and the second analysis result by weighted averaging based on the confidence vector to obtain a final consensus score with the conflict resolved, and derives the corrected final consensus score as a final ideal score;A multimodal cross-validation system for detecting anomalies in time series data, characterized by comprising: a result output unit that accumulates the final anomaly scores sequentially received from the cross-validation consensus unit in a time series database, calculates the moving average and standard deviation of the final anomaly scores during a preset recent N time points to update a dynamic threshold reflecting the distribution change of the time series data, determines whether an anomaly exists by comparing the final anomaly score at the current time point with the updated dynamic threshold, and outputs explanatory information based on the first analysis result. Claim 2 A multimodal cross-validation system for detecting anomalies in time series data according to claim 1, wherein the natural language processing model comprises a plurality of large-scale language models (LLM), and the plurality of large-scale language models include at least one of a pattern analysis model that analyzes trends and periodicity of time series data, a statistical analysis model that interprets statistical indicators, and a domain analysis model that applies industrial domain knowledge. Claim 3 A multimodal cross-validation system for anomaly detection in time-series data according to claim 1, wherein the vision model of the heterogeneous analysis engine includes a convolutional neural network (CNN)-based model that extracts local features of the chart image and a vision transformer (ViT)-based model that identifies global context, and generates the second analysis result by ensembling the output values ​​of the two models. Claim 4 A multimodal cross-validation system for detecting anomalies in time series data according to claim 1, wherein the cross-validation consensus unit, when the difference between the first analysis result and the second analysis result exceeds a threshold and is determined to be inconsistent, queries the database of past time series prediction success rates of each analysis model to recalculate the reliability weight for the analysis model that showed a higher hit rate in the current data pattern, performs a causal analysis to verify whether the logical reasoning included in the technical statement in text format causally matches the shape change including the change in slope of the curve appearing in a specific section of the chart image, and executes an arbitration logic to correct the final anomaly score so as to exclude false positives caused by simple noise and reflect the analysis result with confirmed logical grounds, based on the results of the recalculation of the reliability weight and the verification results of the causal analysis. Claim 5 A multimodal cross-validation system for detecting anomalies in time-series data, characterized in that, in claim 1, it further includes a quantization module that operates in a local GPU environment and dynamically adjusts the batch size of the natural language processing model or vision model or changes the computational precision according to the real-time available memory capacity. Claim 6 A multimodal cross-validation system for detecting anomalies in time series data, wherein, in claim 1, the cross-validation consensus unit includes a dynamic threshold adjustment unit that updates a threshold value serving as a criterion for anomaly determination in real time by reflecting a change in distribution according to the temporal flow of the time series data, and the result output unit generates and provides the cause and basis of the anomaly occurrence in natural language by utilizing the text description, which is the first analysis result. Claim 7 A multimodal cross-validation method for detecting anomalies in time-series data in an on-premise environment comprises: (a) dividing input time-series data into multiple time-series data windows according to a sliding window technique, and simultaneously operating a text generation algorithm and an image rendering engine for each time-series data window to generate a description in text format and a chart image in visual format from the same time-series data window in parallel in a temporally synchronized state; (b) a heterogeneous independent analysis step in which the two analysis processes are performed as mutually independent parallel processes such that the description in text format generated from each time-series data window is analyzed by a natural language processing model to generate a first analysis result, and the chart image in visual format generated from each time-series data window is analyzed by a vision model to generate a second analysis result, wherein the first analysis result and the second analysis result correspond to the same time-series data window. (c) A cross-validation consensus step comprising collecting the first analysis result and the second analysis result corresponding to the same time series data window, calculating the Euclidean distance or deviation between the first analysis result and the second analysis result to determine whether there is a discrepancy between the two results based on whether they fall outside a preset allowable range, and, if a discrepancy is determined, generating a confidence vector for each analysis model by referring to past performance metadata including the past prediction history of each of the natural language processing model and the vision model, and correcting the first analysis result and the second analysis result by weighted averaging based on the confidence vector to obtain a final consensus score with the conflict resolved, and deriving the corrected final consensus score as a final ideal score;and (d) a step of accumulating the final anomaly scores sequentially derived in the cross-validation consensus step into a time series database, calculating the moving average and standard deviation of the final anomaly scores over a preset recent N time points to update a dynamic threshold reflecting the distribution change of the time series data, comparing the final anomaly score at the current time point with the updated dynamic threshold to determine whether an anomaly exists, and generating and outputting natural language explanation information based on the first analysis result, characterized by a multimodal cross-validation method for detecting anomalies in time series data. Claim 8 A multimodal cross-validation method for anomaly detection in time series data according to claim 7, wherein the step of generating the first analysis result of step (b) comprises: (b1) a step of interpreting the trend and periodicity of the time series data using a language model dedicated to pattern analysis; (b2) a step of interpreting the statistical distribution characteristics of the data using a language model dedicated to statistical interpretation; and (b3) a step of inferring context using a domain-specific language model by referring to a knowledge base of the relevant industry domain; and wherein the results of steps (b1) to (b3) are combined to generate the first analysis result. Claim 9 A multimodal cross-validation method for detecting anomalies in time series data according to claim 7, wherein the step of generating the second analysis result of step (b) comprises: a step of extracting local anomaly features within the chart image using a convolutional neural network (CNN); a step of identifying the global contextual flow of the entire chart image using a vision transformer (ViT); and a step of producing the second analysis result by ensembling the local features and the global contextual flow, respectively, by assigning weights to each. Claim 10 A multimodal cross-validation method for detecting anomalies in time series data, wherein the cross-validation consensus step of step (c) comprises: (c1) a step of calculating the difference between the first analysis result and the second analysis result to determine whether a threshold is exceeded; (c2) a step of calculating a reliability score by querying the past prediction history of each model if a discrepancy is determined to be exceeded by the threshold; and (c3) a step of recalculating and assigning a reliability weight to an analysis model that showed a higher hit rate in the current data pattern from the past time series prediction history, verifying whether the logical reasoning included in the text-formatted technical statement causally matches a shape change including a change in the slope of a curve appearing in a specific section of the chart image, and correcting the final consensus score based on the results of recalculating the reliability weight and the verification results of the causal matching to exclude false positives caused by simple noise and reflect an analysis result with confirmed logical grounds. Claim 11 A multimodal cross-validation method for detecting anomalies in time-series data, characterized in that, prior to step (b) above, the method includes a step of checking the GPU memory availability of a local system, and if the GPU memory availability is below a threshold, a step of performing quantization to lower the computational precision of the natural language processing model or vision model, or reducing the batch size of the input data. Claim 12 A multimodal cross-validation method for detecting anomalies in time series data according to claim 7, wherein the step of determining anomalies in step (d) comprises: (d1) a step of calculating a moving average and standard deviation of final anomaly scores calculated over a preset recent period; (d2) a step of updating a dynamic threshold reflecting a change in the distribution of time series data based on the calculated moving average and standard deviation; and (d3) a step of determining anomaly occurrence by comparing whether the final anomaly score derived in step (c) exceeds the updated dynamic threshold. Claim 13 A computer program characterized by being stored in a storage medium to execute a multimodal cross-validation method for detecting anomalies in time series data according to any one of claims 7 to 12 on a computer.

Images