Method and apparatus for detecting exception of memory of java virtual machine

Abstract

The application discloses an abnormality detection method and device for memory of a Java virtual machine, and relates to the technical field of abnormality detection, which comprises the following steps: acquiring index data of multi-dimensional indexes of the memory of the Java virtual machine to be detected, and reading time series data; extracting index features based on a time interval to obtain a feature vector and a scene label; matching model parameters to determine an abnormality detection model to be used, and outputting an abnormality score, a reconstruction error and an abnormality baseline of the memory by using the abnormality detection model to be used; and determining an abnormality level and an abnormality cause, which solves the technical problems of lacking scene correlation analysis, low efficiency, response lag and being unable to locate the root cause of the abnormality, in addition, lacking dynamic adaptive capacity, being difficult to balance the false alarm and the missed alarm risk, and being difficult to meet the complex operation and maintenance scene demand, and achieves the technical effects of fusing the multi-dimensional indexes and multi-modal detection, realizing scene adaptation of the detection threshold and the model parameters, accurately locating the abnormality, improving the detection precision and reducing the operation and maintenance cost.

Description

Technical Field

[0001] This application relates to the field of anomaly detection technology, and in particular to a method and apparatus for anomaly detection of Java Virtual Machine memory. Background Technology

[0002] In related technologies, memory metrics during Java Virtual Machine (JVM) runtime, such as heap usage, GC (Garbage Collection) frequency, object distribution, and heap dump data, can be actively collected via command-line tools. These metrics can then be combined with manual analysis of metric thresholds or heap object reference chains to locate abnormal issues such as memory leaks and frequent GC. Alternatively, the remote monitoring capabilities of JMX (Java Management Extensions) can be used to dynamically obtain JVM memory status data and determine whether abnormal conditions have been met through threshold comparison logic. This can then automatically trigger alarm notifications via message push or log systems.

[0003] However, in related technologies, manual monitoring relies on manual inspection of indicators one by one and lacks scenario correlation analysis, resulting in low efficiency, delayed response, and inability to locate the root cause of anomalies in large-scale clusters; while fixed threshold detection lacks dynamic adaptive capabilities, making it difficult to balance the risks of false alarms and false negatives, and failing to meet the needs of complex operation and maintenance scenarios, and urgently needs improvement. Summary of the Invention

[0004] This application provides a method and apparatus for detecting memory anomalies in a Java Virtual Machine, which at least solves the problems in related technologies, such as manual monitoring relying on manual inspection of indicators one by one and lacking scenario correlation analysis, resulting in low efficiency, delayed response and inability to locate the root cause of anomalies in large-scale clusters; and fixed threshold detection lacking dynamic adaptive capability, making it difficult to balance the risks of false alarms and false negatives, and failing to meet the needs of complex operation and maintenance scenarios.

[0005] This application provides a method for detecting memory anomalies in a Java Virtual Machine (JVM), comprising the following steps: acquiring multi-dimensional indicator data of the memory of the JVM under test, and reading time series data of the indicator data; extracting at least one indicator feature from the indicator data based on the time interval of the time series data to obtain a feature vector corresponding to the indicator data and a scene label of the feature vector; matching the model parameters of a target anomaly detection model based on the feature vector and the scene label, determining the anomaly detection model to be used based on the model parameters, and inputting the indicator data into the anomaly detection model to be used, so as to output the memory anomaly score using an isolated forest, output the memory reconstruction error using an autoencoder, and output the memory anomaly baseline using a statistical baseline; in response to at least two of the anomaly score, the reconstruction error, and the anomaly baseline satisfying a preset anomaly condition, determining that the memory is in an abnormal state, and determining the anomaly level and cause of the memory based on the anomaly score, the reconstruction error, and the anomaly baseline.

[0006] This application also provides an anomaly detection device for Java Virtual Machine memory, comprising: an acquisition module, configured to acquire indicator data of multidimensional indicators of the memory of the Java Virtual Machine under test, and read time series data of the indicator data; a generation module, configured to extract at least one indicator feature from the indicator data based on the time interval of the time series data, to obtain a feature vector corresponding to the indicator data and a scene label of the feature vector; an output module, configured to match the model parameters of a target anomaly detection model based on the feature vector and the scene label, and determine the anomaly detection model to be used based on the model parameters, and input the indicator data into the anomaly detection model to be used, so as to output the anomaly score of the memory using an isolated forest, output the reconstruction error of the memory using an autoencoder, and output the anomaly baseline of the memory using a statistical baseline; and a determination module, configured to determine that the memory is in an abnormal state in response to at least two of the anomaly score, the reconstruction error, and the anomaly baseline satisfying a preset anomaly condition, and determine the anomaly level and anomaly cause of the memory based on the anomaly score, the reconstruction error, and the anomaly baseline.

[0007] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described Java Virtual Machine memory exception detection methods when executing the computer program.

[0008] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described Java Virtual Machine memory exception detection methods.

[0009] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described Java Virtual Machine memory exception detection methods.

[0010] This application allows for the acquisition of multi-dimensional metrics data of the memory of a Java Virtual Machine under test, along with the reading of corresponding time-series data. Based on the time intervals of the time-series data, metric features are extracted to obtain corresponding feature vectors and scene labels for these feature vectors. These features are then used to match the model parameters of a target anomaly detection model, determining the anomaly detection model to be used. The application utilizes an isolated forest to output memory anomaly scores, an autoencoder to output memory reconstruction errors, and a statistical baseline to output memory anomaly baselines. If at least two of the anomaly scores, reconstruction errors, and anomaly baselines meet certain anomaly conditions, the memory is determined to be in an abnormal state, thus identifying the anomaly level and cause. Therefore, this approach addresses the problems of manual monitoring, which relies on manual inspection of metrics and lacks scene correlation analysis, leading to inefficiency, delayed response, and inability to locate the root cause of anomalies in large-scale clusters. Furthermore, fixed threshold detection lacks dynamic adaptability, making it difficult to balance false positives and false negatives and meet the needs of complex operation and maintenance scenarios. This application achieves multi-dimensional metric fusion and multi-modal detection, improving the accuracy of anomaly detection, dynamically adapting to different scenarios, enabling scene-adaptive adjustment of detection thresholds and model parameters, accurately locating the root cause of anomalies, and reducing operation and maintenance costs. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a method for detecting memory anomalies in a Java Virtual Machine, provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the working principle of a method for detecting memory anomalies in a Java Virtual Machine under a scenario management mode according to an embodiment of this application. Figure 3 A flowchart illustrating the working principle of a method for detecting memory anomalies in a Java Virtual Machine not in scene management mode according to an embodiment of this application; Figure 4 This is a block diagram of a Java Virtual Machine memory anomaly detection device provided according to an embodiment of this application.

[0013] Figure label: Among them, 10 is an anomaly detection device for the memory of the Java Virtual Machine; 100 is a first acquisition module; 200 is a first generation module; 300 is an output module; and 400 is a first determination module. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0015] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0016] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] The embodiments of this application provide a method for detecting memory anomalies in a Java Virtual Machine. The method is described in detail below, taking into account the execution flow of the Java Virtual Machine memory anomaly detection method.

[0018] Specifically, Figure 1 This is a flowchart of a method for detecting memory anomalies in a Java Virtual Machine according to an embodiment of this application.

[0019] like Figure 1 As shown, the Java Virtual Machine's memory anomaly detection method includes the following steps: In step S101, the multidimensional metrics data of the memory of the Java Virtual Machine under test are obtained, and the time series data of the metrics data is read.

[0020] It is understood that, in the embodiments of this application, time series data is a set of indicator data arranged in chronological order, used to analyze the trend of indicator changes over time. It may include, but is not limited to, timestamps, used to record the specific time of data collection, such as xx year xx month xx day xx hour xx minute xx second, etc., which are not specifically limited in this application; data points, used to represent the indicator value corresponding to each timestamp, such as xx year xx month xx day xx hour xx minute xx second, with a data size of 512MB; and sampling frequency, used to represent the interval of data collection.

[0021] The indicator data can be understood as the specific quantitative values ​​of multi-dimensional indicators, usually recorded in the form of time series. For example, heap memory usage can be expressed as HeapUsed=512MB, garbage collection time can be expressed as FullGC_Duration=200ms, that is, the time spent in a full garbage collection, and object allocation rate can be expressed as ObjectAllocationRate=10000 / s, that is, the number of new objects created per second. The specific settings can be set by those skilled in the art according to the actual situation, and this application does not impose specific limitations.

[0022] In some embodiments, for the Java Virtual Machine under test, the present application can first obtain the indicator data of the multidimensional indicators of memory, and then read the time series data formed by the indicator data in the form of a time series.

[0023] It should be noted that this application embodiment can utilize Influx DB to store time-series data. Influx DB supports efficient queries by time range and application name, and has a 30-day data retention policy. Simultaneously, it calculates historical metric data and corresponding feature vectors using a Redis cache sliding window, which is also used to store temporary files required for model hot updates, ensuring millisecond-level data access performance. Furthermore, at the data flow level, Influx DB can write metric data in real time, query historical metric data, and retrieve real-time cached data from Redis.

[0024] Optionally, in one embodiment of this application, before obtaining the multi-dimensional metrics data of the memory of the Java Virtual Machine under test, the method further includes: collecting environmental data of the memory in its current state; determining the basic metrics, performance metrics, and business metrics of the memory based on the environmental data; and constructing multi-dimensional metrics based on the basic metrics, performance metrics, and business metrics.

[0025] It is understood that the embodiments of this application adopt a non-intrusive design, providing a data foundation for subsequent detection by collecting multi-dimensional indicators related to memory. These multi-dimensional indicators can quantify the memory state from multiple different dimensions, and may include, but are not limited to, basic indicators such as maximum heap memory, used heap memory, heap utilization, used, committed, and maximum memory of the young generation, used, committed, and maximum memory of the old generation, used, committed, and maximum memory of the metaspace, etc., which are not specifically limited in this application; performance indicators such as GC counts, total execution time, maximum pause time per instance, pause percentage, object promotion rate, heap growth rate, GC frequency, object promotion rate, etc., which are not specifically limited in this application; and business indicators such as QPS, concurrent users, etc., which are not specifically limited in this application.

[0026] Environmental data refers to various data related to the Java Virtual Machine runtime environment, which may include, but is not limited to, hardware environment, such as the model, number of cores, and clock speed of the server's central processing unit, the physical capacity and frequency of memory, disk type (such as solid-state drive or hard disk drive), capacity, and read / write speed, etc., which are not specifically limited in this application; operating system environment, such as the type (such as Linux, Windows, macOS), version number, and memory management strategy of the operating system, etc., which are not specifically limited in this application; and application deployment environment, such as standalone servers, containers, etc., which are not specifically limited in this application.

[0027] In some embodiments, before obtaining the indicator data, the present application embodiments may first collect the environmental data of the memory in its current state, and then accurately determine the basic indicators, performance indicators and business indicators of the memory based on the collected environmental data, thereby using the basic indicators, performance indicators and business indicators to construct corresponding multi-dimensional indicators.

[0028] This application embodiment constructs multi-dimensional indicators by collecting current memory environment data to determine basic indicators, performance indicators, and business indicators. These indicators can comprehensively and accurately reflect the memory status and provide a reliable basis for subsequent detection and analysis.

[0029] Optionally, in one embodiment of this application, acquiring multi-dimensional metrics data of the memory of the Java Virtual Machine under test includes: determining the acquisition frequency and acquisition method of the metrics data based on environmental data; generating an executable acquisition instruction that meets preset instruction conditions based on the acquisition frequency and acquisition method; and acquiring the metrics data according to the executable acquisition instruction.

[0030] It is understood that in the embodiments of this application, the acquisition frequency refers to the time interval for collecting indicator data, that is, how often the indicator data is acquired, which can determine the time granularity and real-time nature of the indicator data; while the acquisition method refers to the specific technical means and methods used to collect indicator data, which may include, but are not limited to, JMX, Java proxy technology and operating system commands, etc., and this application does not impose specific limitations.

[0031] JMX is a management framework provided by the Java platform. Through JMX, the metrics data of the Java Virtual Machine under test can be remotely monitored and managed, and the Java Virtual Machine under test can be connected through JMX client tools, such as Visual VM (Visual Virtual Machine Monitor).

[0032] Java agents are a technique that allows code to be dynamically inserted during the runtime of a Java program. By writing custom Java agents, they can be loaded when the Java Virtual Machine under test starts, collect metric data during program execution, and then send the data to a specified storage or monitoring system.

[0033] Operating system commands can be understood as follows: in this embodiment of the application, some commands such as top, free, vmstat, etc. can be used in the Linux system to obtain indicator data.

[0034] The executable acquisition instructions can be generated as specific commands or scripts based on the acquisition frequency, acquisition method, and preset instruction conditions. These commands can be executed by a computer system to collect indicator data. The preset instruction conditions can be set by those skilled in the art according to actual circumstances, and this application does not impose specific limitations.

[0035] In some embodiments, the present application embodiments may determine the acquisition frequency and acquisition method of different indicator data based on environmental data, thereby generating an executable acquisition instruction that meets preset instruction conditions, and acquiring indicator data according to the executable acquisition instruction.

[0036] For example, in this application embodiment, the JMX API (Application Programming Interface) can be used to determine the acquisition method of indicator data based on environmental data. For instance, the MemoryMXBean interface can be used to implement a global overview of Java Virtual Machine memory, the MemoryPoolMXBean interface can be used to implement detailed monitoring of the memory pool, the GarbageCollectorMXBean interface can be used to implement GC analysis, a custom Logback log outputter can be used to parse GC logs to extract fine-grained indicators such as promotion rate and young generation collection rate, and business indicators can be collected using Spring MVC (Spring Web ModelView Controller, the web page layer model-view-controller architecture of the Spring framework) interceptors. This application does not impose specific limitations; the acquisition frequency is once every 5 seconds and supports dynamic adjustment.

[0037] The embodiments of this application can flexibly determine the acquisition frequency and method of indicator data based on environmental data, and then generate executable instructions for acquisition, which can efficiently and accurately adapt to different scenarios and ensure the accuracy and timeliness of indicator data acquisition.

[0038] Optionally, in one embodiment of this application, before reading the time series data of the indicator data, the method further includes: in response to the indicator data meeting a preset abnormal data condition, generating an abnormal data processing action for the indicator data according to the preset abnormal data condition; and processing the indicator data according to the abnormal data processing action until the processed indicator data no longer meets the preset abnormal data condition.

[0039] It is understood that, before reading the time series data of the indicator data, this application embodiment may first check whether there are missing values ​​in the indicator data, whether there are differences in the dimensions of each indicator data, etc., and this application does not impose specific limitations.

[0040] Furthermore, in embodiments of this application, when missing values ​​exist in the indicator data, it can determine that the indicator data meets preset abnormal data conditions, and determine that the abnormal handling action is to first fill in and then smooth, thereby obtaining processed indicator data that does not meet the preset abnormal data conditions. The preset abnormal data conditions can be set by those skilled in the art according to actual circumstances, and this application does not impose specific limitations.

[0041] For example, in the embodiments of this application, when there are missing values ​​in the indicator data, the missing values ​​are first filled in, that is, the indicator value of the previous sampling point is used to supplement the missing data to avoid data breakage; then, the sliding window is used to smooth the data, and the average value of three consecutive sampling points is used to replace the original value to reduce indicator noise interference.

[0042] In some embodiments, when there are differences in the dimensions of the indicator data, the present application can determine that the indicator data meets the preset abnormal data conditions, and determine that the abnormal handling action is normalization processing, so as to obtain the processed indicator data that does not meet the preset abnormal data conditions.

[0043] For example, in the embodiments of this application, when there are differences in the dimensions of the various indicator data, normalization processing can be performed. The method based on the minimum and maximum values ​​of the normal data in the past 7 days is used to scale all indicator data to the [0,1] range to eliminate the influence of dimensions.

[0044] This application embodiment automatically generates and executes anomaly handling actions in response to preset abnormal data conditions until the data is normal, thereby effectively ensuring the quality of indicator data and avoiding interference from abnormal data to subsequent analysis.

[0045] Optionally, in one embodiment of this application, reading the time series data of the indicator data includes: identifying the storage location, storage format, and storage size of the indicator data; and identifying the time interval in which the indicator data is located based on the storage location, storage format, and storage size, so as to read the time series data according to the time interval.

[0046] It is understood that, in order to configure, identify and verify sudden traffic scenarios and provide a basis for decision-making for subsequent dynamic rule adjustments of the system, the embodiments of this application configure corresponding time intervals in some typical scenarios. These may include, but are not limited to, fixed holiday scenarios, such as the 618 shopping festival and the Spring Festival, where the time interval is configured as the date of the 618 shopping festival or the Spring Festival; in periodic time scenarios, such as the daily morning rush hour, the time interval is configured as 9-11 am, and this application does not impose specific restrictions; in temporary event scenarios, such as flash sales, the time interval is configured as the time of the flash sale, and the specific settings can be made by those skilled in the art according to the actual situation, and this application does not impose specific restrictions.

[0047] It should be noted that, in the case of fixed holiday scenarios, periodic time periods, or other periodic time periods, the time series data in this application embodiment can also reflect the complete start and end timestamps of the scenario, expected traffic multiples, related business indicators, etc., and this application does not impose specific limitations.

[0048] In some embodiments, the present application can identify the time interval of the indicator data by identifying the storage location, storage format and storage size of the indicator data, and then read the time series data according to the time interval.

[0049] For example, if the indicator data in this application embodiment is stored in a file system, the time interval of the indicator data can be determined based on the file's creation time, modification time, or timestamp information in the file name; for structured data, the time interval of the indicator data can be determined by querying the time field in the table; while the storage size itself cannot directly determine the time interval of the data, it can be used as a reference factor. For example, if it is known that the amount of indicator data generated each day is roughly the same, the time interval of the indicator data can be estimated based on the change in storage size.

[0050] The embodiments of this application can determine the time interval by accurately identifying the storage location, format and size of the indicator data, and then read the time series data, which can efficiently and accurately obtain the required data and improve the efficiency and accuracy of data processing.

[0051] In step S102, based on the time interval of the time series data, at least one indicator feature is extracted from the indicator data to obtain the feature vector and the scene label of the feature vector corresponding to the indicator data.

[0052] It is understood that, in the embodiments of this application, the indicator features may include, but are not limited to, heap utilization, old generation utilization, GC pause ratio, heap growth rate, GC frequency, promotion rate, QPS (Queries Per Second) growth rate, concurrent user ratio, and the degree of linear correlation between traffic fluctuation and Java Virtual Machine heap memory fluctuation, etc., and this application does not impose specific limitations.

[0053] In actual implementation, the embodiments of this application can obtain the feature vector and scene label of the feature vector corresponding to the indicator data by extracting the indicator features from the indicator data based on time series data.

[0054] For example, in embodiments of this application, when the time interval is in a fixed holiday scenario, a periodic time scenario, or a periodic time scenario, i.e. when the memory is in scenario management mode, the indicator features are determined as heap utilization, old generation utilization, GC pause ratio, heap growth rate, GC frequency, promotion rate, QPS growth rate, concurrent user ratio, linear correlation between traffic fluctuation and Java Virtual Machine heap memory fluctuation, etc., and then the corresponding feature vector and scenario label of the feature vector are obtained.

[0055] In this application embodiment, when the time interval is not in a fixed holiday scenario, periodic time scenario, or periodic time scenario, i.e. when the memory is not in the scenario management mode, the indicator characteristics are determined as heap utilization, old generation utilization, GC pause ratio, heap growth rate, GC frequency, promotion rate, QPS growth rate, concurrent user ratio, etc., and then the corresponding feature vector and the scenario label of the feature vector are obtained.

[0056] Optionally, in one embodiment of this application, based on the time interval of the time series data, at least one indicator feature is extracted from the indicator data to obtain the feature vector and the scene label of the feature vector corresponding to the indicator data, including: based on the time interval, in response to the memory being in scene management mode, determining that the label value of the scene label is equal to the target value; otherwise, determining that the label value is not equal to the target value.

[0057] It is understood that, in the embodiments of this application, the scene management mode can be understood as a fixed holiday scene, a periodic time scene, and a temporary event scene. The specific settings can be made by those skilled in the art according to the actual situation, and this application does not impose any specific restrictions.

[0058] Furthermore, it should be noted that in this embodiment of the application, when the memory is in scene management mode, it can determine that the tag value of the scene tag is equal to the target value, such as 1; otherwise, it can determine that the tag value is not equal to the target value, such as 0. This application does not impose any specific restrictions.

[0059] In some embodiments, this application can determine whether the memory is in scene management mode by time interval, and determine that the tag value is equal to 1 when it is in scene management mode; otherwise, determine that the tag value is equal to 0.

[0060] For example, in this embodiment, the memory is initially determined to be in scene management mode by the time interval. While refreshing the memory at a frequency of once per minute, it is determined whether the time interval of all the indicator data initially in scene management mode meets the configured time range. If it does, the collected business indicator data, such as QPS, is compared with the average of the indicator data that are not in scene management mode in the past 7 days. Based on the comparison result, if the preset expected multiple is reached (such as 3 times, which is not specifically limited in this application), the memory is finally confirmed to be in scene management mode, that is, the scene is effective, and all indicator data is cached in Redis for subsequent real-time querying.

[0061] It can flexibly determine the label value of scene labels based on time intervals and whether the memory is in scene management mode, which can accurately adapt to different scenes and provide more targeted feature vectors and scene labels for subsequent analysis.

[0062] Optionally, in one embodiment of this application, before matching the model parameters of the target anomaly detection model, the method further includes: acquiring target indicator data of multidimensional indicators of the target memory, and reading target time series data of the target indicator data; extracting at least one indicator feature from the target indicator data based on the target time series data to obtain the target feature vector and the target scene label of the target feature vector corresponding to the target indicator data; determining the training data of each tree in the isolated forest of the target anomaly detection model based on the target feature vector and the target scene label, so as to construct the isolated forest based on the training data; determining the layer information of the input layer, hidden layer and output layer of the autoencoder in the target anomaly detection model and the corresponding layer parameters of each layer based on the target feature vector and the target scene label, so as to construct the autoencoder based on the layer information and the layer parameters; determining the statistical baseline of the target memory based on the isolated forest, the autoencoder, the target feature vector and the target scene label; constructing the target anomaly detection model based on the isolated forest, the autoencoder and the statistical baseline, and determining the model parameters of the target anomaly detection model based on the target feature vector and the target scene label.

[0063] It is understood that the embodiments of this application can achieve anomaly detection through a target anomaly detection model, while taking into account both detection accuracy and real-time performance. This target anomaly detection model includes an isolated forest (main model), an autoencoder (auxiliary model), and a statistical baseline (backup). Specifically, the isolated forest constructs 100 decision trees, quickly isolating outliers in the metrics to output anomaly scores between 0 and 1 (the higher the anomaly score, the greater the reconstruction error), primarily responsible for quickly identifying obvious anomalies; the autoencoder constructs a neural network with a 6-dimensional input layer, a 3-dimensional hidden layer, and a 6-dimensional output layer, calculating the reconstruction error (the larger the error, the greater the reconstruction error), used to capture complex nonlinear dependencies between features; the statistical baseline sets extreme thresholds for heap utilization and old generation utilization based on the 3σ principle, serving as a backup detection mechanism to avoid missing serious anomalies.

[0064] In some embodiments, this application can simultaneously acquire target indicator data of multidimensional indicators of target memory and read corresponding target time series data. Then, based on the indicator features in the target indicator data, the corresponding target feature vector and the target scene label of the target feature vector are determined. This leads to the acquisition of training data for each tree of the isolated forest, layer information of the input layer, hidden layer, and output layer of the autoencoder, and the corresponding layer parameters of each layer, as well as the statistical baseline of the target memory. This allows for the construction of the isolated forest and the autoencoder, thereby building a target anomaly detection model.

[0065] Furthermore, embodiments of this application can determine the model parameters of the target anomaly detection model based on the target feature vector and the target scene label.

[0066] This application embodiment can obtain multi-dimensional index data of the target memory and extract feature vectors and scene labels, and specifically determine the isolated forest training data, autoencoder layer parameters and statistical baseline, thereby constructing a target anomaly detection model and matching the model parameters. This can accurately adapt to the characteristics of the target memory and improve the accuracy and effectiveness of anomaly detection.

[0067] Optionally, in one embodiment of this application, before matching the model parameters of the target anomaly detection model based on feature vectors and scene labels, the method further includes: determining the training data of the target anomaly detection model based on label values; training the target anomaly detection model based on the training data until an anomaly detection model that meets preset training conditions is obtained, so as to construct the target anomaly detection model.

[0068] It is understood that the embodiments of this application can train and maintain two versions of the anomaly detection model. The normal scenario model is trained based on normal data in the time interval of atypical scenarios; the sudden scenario model is specifically trained based on normal data in the time interval of historical typical scenarios (without alarms and traffic in line with expectations), and its model parameters are more adaptable to the fluctuation characteristics under high traffic.

[0069] In some embodiments, this application embodiment can determine whether the memory is in scene management mode based on the label value, and if it is in scene management mode, determine that the training data is normal data of the time interval of historical typical scenes; otherwise, determine that the training data is normal data of the time interval of atypical scenes.

[0070] Therefore, embodiments of this application can train a target anomaly detection model based on training data until an anomaly detection model that meets preset training conditions is obtained, thereby constructing a corresponding target anomaly detection model. The preset training conditions can be set by those skilled in the art according to actual circumstances, and this application does not impose specific limitations.

[0071] Furthermore, this application's embodiments support dynamic updates of the anomaly detection model to be used, configuration of detection rules, and adjustment of indicator collection parameters. At a preset time, such as 3 AM every Monday, the model update process is automatically executed, filtering out normal data from the Influx DB for the past 7 days and retraining the model based on this data, completing a hot update without requiring a service restart. Simultaneously, a more flexible management interface is provided, supporting adjustments to various operating parameters such as model thresholds, alarm classification standards, and collection frequency via API or user interface.

[0072] The embodiments of this application can determine training data and train a target anomaly detection model based on label values ​​until an anomaly detection model that meets preset training conditions is obtained, and a target anomaly detection model is constructed, thereby ensuring that the model learns effectively for specific scenarios and improving the accuracy and reliability of anomaly detection.

[0073] In step S103, based on feature vectors and scene labels, the model parameters of the target anomaly detection model are matched, and the anomaly detection model to be used is determined based on the model parameters. The index data is then input into the anomaly detection model to be used, so as to use the isolated forest to output the memory anomaly score, the autoencoder to output the memory reconstruction error, and the statistical baseline to output the memory anomaly baseline.

[0074] In some embodiments, the present application embodiments may first match the model parameters of the target anomaly detection model based on feature vectors and scene labels, thereby determining the corresponding anomaly detection model to be used.

[0075] For example, in embodiments of this application, when not in scene management mode, the threshold for the anomaly score of the isolated forest is determined to be 0.6, that is, if the anomaly score is greater than 0.6, an "anomaly vote" is cast, and the memory is determined to be in an abnormal state; the threshold for the reconstruction error of the autoencoder is 0.1, that is, if the reconstruction error is greater than 0.1, an "anomaly vote" is cast, and the memory is determined to be in an abnormal state; the threshold for the heap or old generation utilization rate in the statistical baseline is 90%, that is, if the heap or old generation utilization rate is greater than 90%, an "anomaly vote" is cast, and the memory is determined to be in an abnormal state.

[0076] In the scenario management mode of this application embodiment, the threshold for the anomaly score of the isolated forest is 0.8, the threshold for the reconstruction error of the autoencoder is 0.2, and the threshold for the heap or old generation utilization rate in the statistical baseline is 94.5%. These can be set by those skilled in the art according to the actual situation, and this application does not impose any specific limitations.

[0077] Furthermore, in this embodiment of the application, index data can be input into the anomaly detection model to be used, thereby utilizing the isolated forest to output the anomaly score of the memory, utilizing the autoencoder to output the reconstruction error of the memory, and utilizing the statistical baseline to output the anomaly baseline of the memory.

[0078] Optionally, in one embodiment of this application, the abnormal score of the memory is output using an isolated forest, the reconstruction error of the memory is output using an autoencoder, and the abnormal baseline of the memory is output using a statistical baseline. This includes: obtaining the fluctuation characteristics of the abnormal score, reconstruction error, and abnormal baseline, respectively; generating adjustment instructions for the isolated forest, autoencoder, and statistical baseline based on the fluctuation characteristics, respectively; and adjusting the parameters of the isolated forest, autoencoder, and statistical baseline according to the adjustment instructions, respectively, to obtain an isolated forest, autoencoder, and statistical baseline that meet the preset fluctuation conditions.

[0079] In some embodiments, this application can calculate the adjustment instructions for the isolated forest by obtaining the fluctuation characteristics of the outlier scores, and then adjust the parameters of the isolated forest until an isolated forest that meets the preset fluctuation conditions is obtained. The preset fluctuation conditions can be set by those skilled in the art according to actual conditions, and this application does not impose specific limitations.

[0080] In some embodiments, the present application embodiments can calculate the adjustment instructions of the autoencoder by obtaining the fluctuation characteristics of the reconstruction error, and then adjust the parameters of the autoencoder until an autoencoder that meets the preset fluctuation conditions is obtained.

[0081] In some embodiments, the present application can calculate the adjustment instruction of the statistical baseline by obtaining the fluctuation characteristics of the abnormal baseline, and then adjust the parameters of the statistical baseline until a statistical baseline that meets the preset fluctuation conditions is obtained.

[0082] This application's embodiments optimize model parameters by acquiring the fluctuation characteristics of anomaly scores, reconstruction errors, and anomaly baselines respectively, and generating adjustment instructions accordingly. This enables isolated forests, autoencoders, and statistical baselines to better adapt to data fluctuations, thereby improving the accuracy and stability of memory anomaly detection.

[0083] In step S104, in response to at least two of the abnormal score, reconstruction error and abnormal baseline satisfying the preset abnormal conditions, the memory is determined to be in an abnormal state, and the abnormal level and abnormal cause of the memory are determined according to the abnormal score, reconstruction error and abnormal baseline.

[0084] It is understood that this application embodiment adopts a voting mechanism. If the cumulative number of abnormal votes is greater than or equal to 2, the memory is determined to be in an abnormal state. Specifically, this application embodiment can determine that the memory is in an abnormal state when two or more of the abnormal score, reconstruction error, and abnormal baseline meet the preset abnormal conditions, even when not in scene management mode, in order to balance the false positive rate and the false negative rate. In this case, the anomaly detection models to be used are all trained offline based on historical data from the application's anomaly-free period of the past 7 days to generate the initial model. The isolated forest file is stored in PMML (Predictive Model Markup Language) format, and the autoencoder file is stored in ZIP format. However, in this application embodiment, when in scene management mode, the voting rules are adjusted so that the memory can only be determined to be in an abnormal state when the abnormal score, reconstruction error, and abnormal baseline all meet the preset abnormal conditions. Furthermore, to more accurately distinguish the causes of anomalies, in the scenario management mode, if the heap metric exceeds the limit, but the correlation coefficient between the business metric and the heap metric is greater than 0.6 (indicating that memory pressure is mainly driven by traffic), the anomaly baseline will not issue an "anomaly ticket," thereby effectively reducing false alarms caused by normal business peaks. The preset anomaly conditions can be set by those skilled in the art according to actual conditions, and this application does not impose specific restrictions.

[0085] In some embodiments, the present application embodiments can determine that the memory is in an abnormal state by responding to at least two of the abnormal score, reconstruction error and abnormal baseline satisfying a preset abnormal condition, and then determine the abnormal level and abnormal cause of the memory based on the abnormal score, reconstruction error and abnormal baseline.

[0086] For example, in embodiments of this application, when not in scene management mode, if there are at least two of the following conditions: an anomaly score greater than 0.6, a reconstruction error greater than 0.1, or a heap or old generation utilization rate greater than 90%, the preset anomaly conditions are determined to be met, and the memory is determined to be in an abnormal state. Then, the anomaly level and cause of the memory are determined based on the anomaly score, reconstruction error, and anomaly baseline.

[0087] In the scenario management mode, when the anomaly score is greater than 0.8, the reconstruction error is greater than 0.2, and the heap or old generation utilization rate is greater than 90%, the preset anomaly conditions are met, and the memory is determined to be in an abnormal state. Then, the anomaly level and cause of the memory are determined based on the anomaly score, reconstruction error and anomaly baseline.

[0088] Optionally, in one embodiment of this application, matching the model parameters of the target anomaly detection model based on feature vectors and scene labels includes: matching the model parameters using feature vectors and scene labels as first model parameters in response to the memory being in scene management mode; otherwise, matching the model parameters using feature vectors and scene labels as second model parameters.

[0089] In some embodiments, this application can determine whether the memory is in scene management mode based on feature vectors and scene labels, and if it is in scene management mode, determine the model parameters as the first model parameters.

[0090] In some embodiments, this application can determine the model parameters as second model parameters when the memory is not in scene management mode.

[0091] The embodiments of this application can flexibly match different model parameters based on feature vectors and scene labels, depending on whether the memory is in scene management mode, so as to accurately adapt to different scene requirements and improve the adaptability and detection effect of the target anomaly detection model.

[0092] Optionally, in one embodiment of this application, determining the anomaly level and cause of memory based on the anomaly score, reconstruction error, and anomaly baseline includes: calculating a comprehensive anomaly score of memory based on the average of the anomaly score, reconstruction error, and anomaly baseline; and determining the anomaly level and cause of memory based on the comprehensive anomaly score.

[0093] It is understood that the embodiments of this application can divide the anomaly level into three levels: critical alarm, warning alarm, and alert alarm. The specific level can be set by those skilled in the art according to the actual situation, and this application does not impose any specific restrictions.

[0094] In some embodiments, the present application can calculate the average of the anomaly score, reconstruction error, and anomaly baseline to obtain the corresponding comprehensive anomaly score, and determine the anomaly level and cause based on the comprehensive anomaly score.

[0095] For example, in this application embodiment, the average of the calculated anomaly score, reconstruction error and anomaly baseline can be used as a comprehensive anomaly score, and the anomaly level can be determined based on the comprehensive anomaly score. The anomaly cause can be generated by combining root cause analysis and pushed through a preset channel. At the same time, the anomaly result can be written to Influx DB for post-analysis.

[0096] For example, in this embodiment, when heap utilization exceeds 95% and GC pause rate is greater than 30%, a critical memory overflow alarm is triggered and pushed via both DingTalk and SMS; when heap growth rate exceeds 200KB / s and old generation utilization exceeds 80%, a warning alarm indicating memory leak risk is triggered and pushed via DingTalk; when GC frequency exceeds 30 times / minute and model reconstruction error exceeds the standard, a warning alarm indicating GC storm is triggered and logged. Furthermore, this embodiment outputs the cause of anomalies based on the correlation between indicators, such as a continuously rising heap utilization that does not decrease after GC → suspected memory leak, suggesting checking for diagnostic information such as unreleased cache, to guide operations personnel to quickly locate the root cause of the problem.

[0097] The embodiments of this application can calculate a comprehensive anomaly score by combining the anomaly score, reconstruction error and anomaly baseline, and determine the memory anomaly level and cause accordingly. It can comprehensively consider multiple indicators, making the anomaly judgment more scientific and accurate, and the cause analysis more precise and reliable.

[0098] Optionally, in one embodiment of this application, determining the anomaly level and cause of memory based on the anomaly score, reconstruction error, and anomaly baseline includes: obtaining the false alarm probability of the anomaly level and the anomaly cause; and determining the anomaly level and the anomaly cause based on the anomaly score, reconstruction error, and anomaly baseline in response to the false alarm probability being greater than or equal to a preset probability.

[0099] It is understood that the embodiments of this application can mark false alarms through false alarm feedback, thereby automatically optimizing subsequent detection rules and models. For example, when marking an anomaly level or cause as a false alarm through the user interface or API, the embodiments of this application need to submit relevant indicator data, corresponding time series data, feature vectors, and corresponding scene labels at the same time, and add the marked false alarm to a dedicated "normal data pool" to ensure that it can be included in the next periodic model update training, so that the model learns that such patterns are not abnormal.

[0100] In some embodiments, the present application can redetermine the anomaly level and anomaly cause if the false alarm probability of the anomaly level and anomaly cause is greater than or equal to a preset probability. The preset probability can be set by those skilled in the art according to actual circumstances, and the present application does not impose specific limitations.

[0101] The embodiments of this application can determine the final memory anomaly level and cause by the false alarm probability, effectively reducing the risk of false alarms and improving the accuracy and reliability of memory anomaly judgment.

[0102] The working principle of the Java Virtual Machine memory anomaly detection method proposed in this application will be described below with reference to several embodiments.

[0103] Example 1: Specifically, Figure 2 This is a flowchart illustrating the working principle of a method for detecting memory anomalies in a Java Virtual Machine under a scenario management mode according to an embodiment of this application.

[0104] Step S201: Indicator data collection.

[0105] In this embodiment, the indicator data of multidimensional indicators can be obtained, and the time series data of the indicator data can be read.

[0106] The multidimensional metrics may include, but are not limited to, basic metrics such as maximum heap memory, used heap memory, heap utilization, used, committed, and maximum memory of the young generation, used, committed, and maximum memory of the old generation, used, committed, and maximum memory of the metaspace, etc., which are not specifically limited in this application; performance metrics such as GC counts, total execution time, maximum pause time per event, pause percentage, object promotion rate, heap growth rate, GC frequency, object promotion rate, etc., which are not specifically limited in this application; and business metrics such as QPS, concurrent users, etc., which are not specifically limited in this application.

[0107] Step S202: Data storage.

[0108] In this embodiment, Influx DB can be used to store time-series data. Simultaneously, a Redis cache sliding window is used to calculate historical metric data and corresponding feature vectors, which are also used to store temporary files required for model hot updates, ensuring millisecond-level data access performance. Furthermore, at the data flow level, Influx DB can write metric data in real time, query historical metric data, and retrieve real-time cached data from Redis.

[0109] Step S203: Extract feature vectors.

[0110] In this embodiment of the application, when the memory is in scene management mode, the feature vector is determined to include heap utilization, old generation utilization, GC pause ratio, heap growth rate, GC frequency, promotion rate, QPS growth rate, concurrent user ratio, linear correlation between traffic fluctuation and Java Virtual Machine heap memory fluctuation, etc., and the scene label is equal to the target value.

[0111] Step S204: Scene Management.

[0112] Step S205: Determine the anomaly detection model to be used.

[0113] In this embodiment, based on feature vectors and scene labels, the model parameters of the target anomaly detection model are matched to determine the anomaly detection model to be used. The threshold for the anomaly score of the isolated forest is set to 0.8, the threshold for the reconstruction error of the autoencoder is set to 0.2, and the threshold for the heap or old generation utilization rate in the statistical baseline is set to 94.5%. When the anomaly score, reconstruction error and anomaly baseline all meet the preset anomaly conditions, the memory is determined to be in an abnormal state, and the anomaly level and cause of the memory are output.

[0114] Step S206: Alarm.

[0115] In this embodiment, the anomaly level can be divided into three levels: critical alarm, warning alarm, and alert alarm. When the anomaly level is critical alarm, it is pushed through both DingTalk and SMS; when the anomaly level is warning alarm, it is pushed through DingTalk; and when the anomaly level is alert alarm, it is recorded through logs.

[0116] Step S207: False alarm feedback.

[0117] In this embodiment, false alarms can be marked, and if the false alarm probability of the anomaly level and the anomaly cause is greater than or equal to a preset probability, the anomaly level and the anomaly cause can be re-determined.

[0118] Step S208: Operation and maintenance management.

[0119] This application's embodiments support dynamic updates to the anomaly detection model, configuration of detection rules, and adjustment of indicator collection parameters. At a preset time, such as 3 AM every Monday, the model update process is automatically executed, filtering out normal data from the Influx DB for the past 7 days and retraining the model based on this data, completing a hot update without requiring a service restart. Furthermore, a more flexible management interface is provided, supporting adjustments to various operating parameters such as model thresholds, alarm classification standards, and collection frequency via API or user interface.

[0120] In summary, this application embodiment can form a more intelligent and adaptive closed loop through scene recognition, rule adjustment, and model parameter selection. When the process starts, metric collection and scene recognition are executed in parallel: metric data in scene management mode is acquired and written to Influx DB and Redis; simultaneously, this application embodiment sets the tag values ​​of the scene labels to target values, thereby constructing a 9-dimensional enhanced feature vector including heap utilization, old generation utilization, GC pause ratio, heap growth rate, GC frequency, promotion rate, QPS growth rate, concurrent user ratio, and the linear correlation between traffic fluctuations and Java Virtual Machine heap memory fluctuations; then, based on the memory anomaly score, reconstruction error, and anomaly baseline output by the anomaly detection model, the anomaly level and cause of the memory anomaly are determined. The entire process is continuously optimized through regular model updates.

[0121] Example 2: Specifically, Figure 3 This is a flowchart illustrating the working principle of a method for detecting memory anomalies in a Java Virtual Machine that is not in scene management mode, according to an embodiment of this application.

[0122] Step S301: Indicator data collection.

[0123] Step S302: Data storage.

[0124] Step S303: Extract feature vectors.

[0125] In this embodiment of the application, when the memory is in scene management mode, the feature vector is determined to include heap utilization, old generation utilization, GC pause ratio, heap growth rate, GC frequency, promotion rate, QPS growth rate, concurrent user ratio, etc., and the scene label is not equal to the target value.

[0126] Step S304: Determine the anomaly detection model to be used.

[0127] In this embodiment, based on feature vectors and scene labels, the model parameters of the target anomaly detection model are matched to determine the anomaly detection model to be used. The threshold for the anomaly score of the isolated forest is set to 0.6, the threshold for the reconstruction error of the autoencoder is set to 0.1, and the threshold for the heap or old generation usage rate in the statistical baseline is set to 90%. When at least two of the anomaly score, reconstruction error and anomaly baseline meet the preset anomaly conditions, the memory is determined to be in an abnormal state, and the anomaly level and cause of the memory are output.

[0128] Step S305: Alarm.

[0129] In this embodiment, the anomaly level can be divided into three levels: critical alarm, warning alarm, and alert alarm. When the anomaly level is critical alarm, it is pushed through both DingTalk and SMS; when the anomaly level is warning alarm, it is pushed through DingTalk; and when the anomaly level is alert alarm, it is recorded through logs.

[0130] Step S306: Operation and maintenance management.

[0131] In summary, this application embodiment generates memory anomaly levels and causes through four stages: indicator collection, feature processing, anomaly detection, and alarm output. The anomaly levels and causes are then written into Influx DB for post-analysis.

[0132] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0133] The Java Virtual Machine (JVM) memory anomaly detection method proposed in this application can acquire multi-dimensional indicator data of the JVM memory under test, read the corresponding time series data, and then extract indicator features based on the time interval of the time series data to obtain the corresponding feature vector and scene label of the feature vector. Based on this, the model parameters of the target anomaly detection model are matched to determine the anomaly detection model to be used. The method then utilizes an isolated forest to output the memory anomaly score, an autoencoder to output the memory reconstruction error, and a statistical baseline to output the memory anomaly baseline. The method responds to at least one of the anomaly score, reconstruction error, and anomaly baseline. Two conditions must be met to determine that the memory is in an abnormal state, thereby identifying the level and cause of the anomaly. This solves the problems of manual monitoring, which relies on checking each indicator manually and lacks scenario correlation analysis, leading to inefficiency, slow response, and inability to locate the root cause of anomalies in large-scale clusters. Fixed threshold detection, on the other hand, lacks dynamic adaptive capabilities, making it difficult to balance the risks of false positives and false negatives and meet the technical requirements of complex operation and maintenance scenarios. This approach achieves multi-dimensional indicator fusion and multi-modal detection, improving the accuracy of anomaly detection, dynamically adapting to scenarios, and enabling scenario-adaptive adjustment of detection thresholds and model parameters. This allows for precise location of the root cause of anomalies and reduces operation and maintenance costs.

[0134] Embodiments of this application also provide an anomaly detection device for the memory of a Java Virtual Machine.

[0135] Figure 4 This is a block diagram of a Java Virtual Machine memory anomaly detection device provided according to an embodiment of this application.

[0136] like Figure 4As shown, the Java Virtual Machine memory anomaly detection device 10 includes: a first acquisition module 100, a first generation module 200, an output module 300, and a first determination module 400.

[0137] The first acquisition module 100 is used to acquire the multi-dimensional indicator data of the memory of the Java Virtual Machine under test, and read the time series data of the indicator data.

[0138] The first generation module 200 is used to extract at least one indicator feature from the indicator data based on the time interval of the time series data, so as to obtain the feature vector and the scene label of the feature vector corresponding to the indicator data.

[0139] The output module 300 is used to match the model parameters of the target anomaly detection model based on the feature vector and scene label, determine the anomaly detection model to be used based on the model parameters, and input the index data into the anomaly detection model to be used, so as to use the isolated forest to output the anomaly score of the memory, use the autoencoder to output the reconstruction error of the memory, and use the statistical baseline to output the anomaly baseline of the memory.

[0140] The first determining module 400 is used to determine that the memory is in an abnormal state in response to at least two of the abnormal score, reconstruction error and abnormal baseline satisfying a preset abnormal condition, and to determine the abnormal level and abnormal cause of the memory based on the abnormal score, reconstruction error and abnormal baseline.

[0141] Optionally, in one embodiment of this application, it further includes: a second acquisition module, an extraction module, a first construction module, a second construction module, a second determination module, and a third determination module.

[0142] The second acquisition module is used to acquire target indicator data of multidimensional indicators in the target memory and read target time series data of the target indicator data before matching the model parameters of the target anomaly detection model.

[0143] The extraction module is used to extract at least one indicator feature from the target indicator data based on the target time series data, so as to obtain the target feature vector and the target scene label of the target feature vector corresponding to the target indicator data.

[0144] The first construction module is used to determine the training data for each tree in the isolated forest of the target anomaly detection model based on the target feature vector and the target scene label, so as to construct the isolated forest based on the training data.

[0145] The second construction module is used to determine the layer information and corresponding layer parameters of the input layer, hidden layer and output layer of the autoencoder in the target anomaly detection model based on the target feature vector and the target scene label, so as to construct the autoencoder according to the layer information and layer parameters.

[0146] The second determination module is used to determine the statistical baseline of the target memory based on isolated forest, autoencoder, target feature vector and target scene label.

[0147] The third determination module is used to construct a target anomaly detection model based on isolated forest, autoencoder and statistical baseline, and to determine the model parameters of the target anomaly detection model based on target feature vector and target scene label.

[0148] Optionally, in one embodiment of this application, it further includes: a data acquisition module, a fourth determination module, and a third construction module.

[0149] The acquisition module is used to collect environmental data of the memory in its current state before acquiring the multi-dimensional metrics data of the memory of the Java Virtual Machine under test.

[0150] The fourth module is used to determine the basic, performance, and business metrics of memory based on environmental data.

[0151] The third module is used to build multi-dimensional metrics based on basic metrics, performance metrics, and business metrics.

[0152] Optionally, in one embodiment of this application, the first generation module 200 includes: a first determination unit.

[0153] The first determining unit is used to determine, based on a time interval and in response to the memory being in scene management mode, whether the label value of the scene label is equal to the target value, or whether the label value is not equal to the target value.

[0154] Optionally, in one embodiment of this application, the output module 300 includes a matching unit.

[0155] The matching unit is used to match model parameters as first model parameters based on feature vectors and scene labels, in response to the memory being in scene management mode; otherwise, it matches model parameters as second model parameters based on feature vectors and scene labels.

[0156] Optionally, in one embodiment of this application, it further includes: a fifth determining module and a training module.

[0157] The fifth determining module is used to determine the training data of the target anomaly detection model based on the label values ​​before matching the model parameters of the target anomaly detection model based on the feature vector and scene label.

[0158] The training module is used to train the target anomaly detection model based on the training data until an anomaly detection model that meets the preset training conditions is obtained, so as to construct the target anomaly detection model.

[0159] Optionally, in one embodiment of this application, the first acquisition module 100 includes: an identification unit and a reading unit.

[0160] The identification unit is used to identify the storage location, storage format, and storage size of the indicator data.

[0161] The reading unit is used to identify the time interval of the indicator data based on the storage location, storage format and storage size, so as to read the time series data according to the time interval.

[0162] Optionally, in one embodiment of this application, the first determining module 400 includes: a first acquiring unit and a second determining unit.

[0163] The first acquisition unit is used to acquire the false alarm probability of the anomaly level and the cause of the anomaly.

[0164] The second determining unit is used to determine the anomaly level and anomaly cause based on the anomaly score, reconstruction error and anomaly baseline in response to a false alarm probability greater than or equal to a preset probability.

[0165] Optionally, in one embodiment of this application, the output module 300 includes: a second acquisition unit, a generation unit, and an adjustment unit.

[0166] The second acquisition unit is used to acquire the fluctuation characteristics of the anomaly score, reconstruction error, and anomaly baseline, respectively.

[0167] The generation unit is used to generate adjustment instructions for isolated forests, autoencoders, and statistical baselines based on fluctuation characteristics, respectively.

[0168] The adjustment unit is used to adjust the parameters of the isolated forest, autoencoder, and statistical baseline according to the adjustment instructions, so as to obtain the isolated forest, autoencoder, and statistical baseline that meet the preset fluctuation conditions.

[0169] For a description of the features of the embodiment corresponding to the Java Virtual Machine memory anomaly detection device, please refer to the relevant description of the embodiment corresponding to the Java Virtual Machine memory anomaly detection method, which will not be repeated here.

[0170] The Java Virtual Machine (JVM) memory anomaly detection device proposed in this application can acquire multi-dimensional index data of the JVM memory under test, read the corresponding time series data, and then extract index features based on the time interval of the time series data to obtain the corresponding feature vector and scene label of the feature vector. Based on this, it matches the model parameters of the target anomaly detection model, determines the anomaly detection model to be used, and outputs the memory anomaly score using an isolated forest, outputs the memory reconstruction error using an autoencoder, and outputs the memory anomaly baseline using a statistical baseline. It responds to at least one of the anomaly score, reconstruction error, and anomaly baseline. Two conditions must be met to determine that the memory is in an abnormal state, thereby identifying the level and cause of the anomaly. This solves the problems of manual monitoring, which relies on checking each indicator manually and lacks scenario correlation analysis, leading to inefficiency, slow response, and inability to locate the root cause of anomalies in large-scale clusters. Fixed threshold detection, on the other hand, lacks dynamic adaptive capabilities, making it difficult to balance the risks of false positives and false negatives and meet the technical requirements of complex operation and maintenance scenarios. This approach achieves multi-dimensional indicator fusion and multi-modal detection, improving the accuracy of anomaly detection, dynamically adapting to scenarios, and enabling scenario-adaptive adjustment of detection thresholds and model parameters. This allows for precise location of the root cause of anomalies and reduces operation and maintenance costs.

[0171] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above embodiments of the Java Virtual Machine memory anomaly detection method.

[0172] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above embodiments of the Java Virtual Machine memory anomaly detection method at runtime.

[0173] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0174] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the Java Virtual Machine memory anomaly detection method.

[0175] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above embodiments of the Java Virtual Machine memory anomaly detection method.

[0176] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0177] The above provides a detailed description of a method for detecting memory anomalies in a Java Virtual Machine, as provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for detecting memory anomalies in a Java Virtual Machine, characterized in that, Includes the following steps: Obtain the multidimensional metrics data of the memory of the Java Virtual Machine under test, and read the time series data of the metrics data; Based on the time interval of the time series data, at least one indicator feature is extracted from the indicator data to obtain the feature vector corresponding to the indicator data and the scene label of the feature vector. Based on the feature vector and the scene label, the model parameters of the target anomaly detection model are matched, and the anomaly detection model to be used is determined based on the model parameters. The index data is then input into the anomaly detection model to be used, so as to output the anomaly score of the memory using an isolated forest, output the reconstruction error of the memory using an autoencoder, and output the anomaly baseline of the memory using a statistical baseline. In response to at least two of the anomaly score, the reconstruction error, and the anomaly baseline satisfying a preset anomaly condition, the memory is determined to be in an abnormal state, and the anomaly level and cause of the memory are determined based on the anomaly score, the reconstruction error, and the anomaly baseline.

2. The method according to claim 1, characterized in that, Before matching the model parameters of the target anomaly detection model, the following is also included: Obtain the target indicator data of the multidimensional indicators of the target memory, and read the target time series data of the target indicator data; Based on the target time series data, at least one indicator feature is extracted from the target indicator data to obtain the target feature vector corresponding to the target indicator data and the target scene label of the target feature vector; Based on the target feature vector and the target scene label, the training data for each tree in the isolated forest of the target anomaly detection model is determined, so as to construct the isolated forest according to the training data; Based on the target feature vector and the target scene label, determine the layer information of the input layer, hidden layer and output layer of the autoencoder in the target anomaly detection model and the corresponding layer parameters of each layer, so as to construct the autoencoder according to the layer information and the layer parameters. Based on the isolated forest, the autoencoder, the target feature vector, and the target scene label, a statistical baseline for the target memory is determined; Based on the isolated forest, the autoencoder, and the statistical baseline, the target anomaly detection model is constructed, and the model parameters of the target anomaly detection model are determined based on the target feature vector and the target scene label.

3. The method according to claim 1, characterized in that, Before obtaining the multidimensional metrics data of the Java Virtual Machine under test's memory, the following steps are also included: Collect environmental data of the memory in its current state; Based on the environmental data, the basic indicators, performance indicators, and business indicators of the memory are determined; The multidimensional indicators are constructed based on the basic indicators, the performance indicators, and the business indicators.

4. The method according to claim 1, characterized in that, The step of extracting at least one indicator feature from the indicator data based on the time interval of the time series data to obtain the feature vector corresponding to the indicator data and the scene label of the feature vector includes: Based on the time interval, in response to the memory being in the scene management mode, it is determined that the tag value of the scene tag is equal to the target value; otherwise, it is determined that the tag value is not equal to the target value.

5. The method according to claim 4, characterized in that, The model parameters for matching the target anomaly detection model based on the feature vector and the scene label include: Based on the feature vector and the scene label, in response to the memory being in the scene management mode, the model parameters are matched using the feature vector and the scene label to form the first model parameters; otherwise, the model parameters are matched using the feature vector and the scene label to form the second model parameters.

6. The method according to claim 5, characterized in that, Before matching the model parameters of the target anomaly detection model based on the feature vector and the scene label, the process further includes: Based on the label values, the training data for the target anomaly detection model is determined; Based on the training data, the target anomaly detection model is trained until an anomaly detection model that meets the preset training conditions is obtained, so as to construct the target anomaly detection model.

7. The method according to claim 1, characterized in that, The time series data for reading the indicator data includes: Identify the storage location, storage format, and storage size of the indicator data; Based on the storage location, storage format, and storage size, the time interval in which the indicator data is located is identified, so that the time series data can be read according to the time interval.

8. The method according to claim 1, characterized in that, The step of determining the anomaly level and cause of the memory based on the anomaly score, the reconstruction error, and the anomaly baseline includes: Obtain the false alarm probability of the anomaly level and the cause of the anomaly; In response to the false alarm probability being greater than or equal to a preset probability, the anomaly level and the anomaly cause are determined based on the anomaly score, the reconstruction error, and the anomaly baseline.

9. The method according to claim 1, characterized in that, The method of using an isolated forest to output the outlier score of the memory, using an autoencoder to output the reconstruction error of the memory, and using a statistical baseline to output the outlier baseline of the memory includes: The fluctuation characteristics of the anomaly score, the reconstruction error, and the anomaly baseline are obtained respectively; Adjustment instructions for the isolated forest, the autoencoder, and the statistical baseline are generated based on the fluctuation characteristics, respectively. The parameters of the isolated forest, the autoencoder, and the statistical baseline are adjusted according to the adjustment instructions to obtain an isolated forest, autoencoder, and statistical baseline that meet the preset fluctuation conditions.

10. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the Java Virtual Machine memory anomaly detection method as described in any one of claims 1-9.

Images