Method, device and storage medium for tracing out-of-pile memory exception

Abstract

The application discloses an off-heap memory exception tracing method, device and storage medium. In response to off-heap memory allocation, corresponding memory allocation events are recorded and path signatures of the memory allocation events are determined, the path signatures being used for grouping the memory allocation events. Process physical memory and garbage collection events are periodically collected to obtain corresponding exception reference information. According to the memory allocation events corresponding to each path signature in a preset time interval and the exception reference information, an abnormal path signature existing off-heap memory exception is identified. According to the memory allocation events corresponding to the abnormal path signature, exception tracing is performed to obtain exception tracing information. The application realizes fine exception positioning by setting the path signature, and performs exception identification by taking the heap memory as the exception reference, so that the exception identification is closer to the business scenario.

Description

Technical Field

[0001] This application relates to the field of data storage, and includes, but is not limited to, a method, device, and storage medium for tracing off-heap memory anomalies. Background Technology

[0002] The Java Virtual Machine (JVM) heavily utilizes off-heap memory in high-concurrency, low-latency applications to improve I / O and data processing efficiency. However, the standard off-heap memory entry points commonly used can bypass the normal on-heap object allocation path, improving I / O processing efficiency; their abnormal retention usually manifests as a continuous increase in process memory usage; and existing JVM metrics tools often only show total or subsystem-level changes, making it difficult to directly pinpoint the specific business call path. Summary of the Invention

[0003] In view of this, the off-heap memory anomaly tracing method, device, and storage medium provided in the embodiments of this application can achieve fine-grained off-heap memory anomaly localization.

[0004] The off-heap memory anomaly tracing method, device, and storage medium provided in this application embodiment are implemented as follows: One aspect of this application provides a method for tracing off-heap memory anomalies, the method comprising: In response to off-heap memory allocation, the corresponding memory allocation event is recorded and the path signature of the memory allocation event is determined. The path signature is used to group the memory allocation events. Periodically collect data on process physical memory and garbage collection events to obtain corresponding anomaly reference information; Based on the memory allocation events corresponding to each path signature within a preset time interval, and the abnormal reference information, abnormal path signatures with off-heap memory abnormalities are identified. The abnormal source is traced based on the memory allocation event corresponding to the abnormal path signature to obtain abnormal source information.

[0005] In one possible implementation, recording the corresponding memory allocation event and determining the path signature of the memory allocation event includes: Collect at least one of the following information corresponding to off-heap memory allocation: event identifier, allocation time, allocation size, calling thread, call stack, interface type, and business context information, and record the corresponding memory allocation event; The corresponding path signature is determined based on the recorded memory allocation event content.

[0006] In one possible implementation, determining the corresponding path signature based on the recorded memory allocation event content includes: Determine the hash value of the call stack in the memory allocation event; Generate a corresponding path signature based on at least one of the hash value, the interface type, and the business context information.

[0007] In one possible implementation, the periodic collection of process physical memory and garbage collection events yields corresponding anomaly reference information, including: Periodically collect the physical memory of the process to obtain the process physical memory sequence; Garbage collection events are recorded periodically to obtain a garbage collection event sequence; Anomaly reference information is determined based on the process physical memory sequence and the garbage collection event sequence.

[0008] In one possible implementation, the step of identifying abnormal path signatures with off-heap memory anomalies based on the memory allocation events corresponding to each path signature within a preset time interval and the abnormal reference information includes: Based on the memory allocation events corresponding to each path signature within a preset time interval, determine the off-heap memory information corresponding to the path signature; For each path signature, anomaly identification is performed based on the off-heap memory information, the anomaly reference information within the preset time interval, and the preset anomaly identification rules to determine whether the path signature is an abnormal path signature.

[0009] In one possible implementation, determining the off-heap memory information corresponding to each path signature based on the memory allocation events corresponding to each path signature within a preset time interval includes: For each path signature, the corresponding memory allocation event is obtained by sliding within the preset time interval using a preset time window, and the cumulative allocation amount within the corresponding time window, the residual allocation amount at the end of the time window, the allocation growth amount compared with the previous time window, and the ratio of the residual allocation amount to the global residual amount are statistically analyzed. The off-heap memory information corresponding to the path signature is determined based on at least one of the following: the cumulative allocation amount, the residual allocation amount, the allocation growth amount, and the ratio of the residual allocation amount to the global residual amount for each time window.

[0010] In one possible implementation, the anomaly reference information includes a process physical memory sequence and a garbage collection event sequence, and the anomaly identification rule includes at least one of the following: The cumulative allocation and / or residual allocation corresponding to the first number of consecutive time windows of the path signature continuously increase; Within the time interval corresponding to the second number of garbage collection events in the garbage collection event sequence, the change in the cumulative allocation amount corresponding to the path signature does not meet the preset condition; The changes in the allocation growth of the multiple time windows corresponding to the path signature are synchronized with the changes in the physical memory of each process in the process physical memory sequence; The ratio of the residual allocation amount corresponding to the path signature to the global residual amount is greater than a preset threshold.

[0011] In one possible implementation, the step of tracing the anomaly source based on the memory allocation event corresponding to the anomaly path signature to obtain anomaly source information includes: Obtain the call stack corresponding to the abnormal path signature to determine the corresponding business module; The corresponding business interface and tenant are determined based on the business context information corresponding to the abnormal path signature; The anomaly tracing information is determined based on the business module, business interface, and tenant.

[0012] Another aspect of this application embodiment provides an off-heap memory anomaly tracing device, the device comprising: The event logging module is used to record the corresponding memory allocation event and determine the path signature of the memory allocation event in response to off-heap memory allocation. The path signature is used to group the memory allocation events. The reference information acquisition module is used to periodically collect process physical memory and garbage collection events to obtain corresponding abnormal reference information; An anomaly identification module is used to identify anomaly path signatures that have off-heap memory anomalies based on the memory allocation events corresponding to each path signature within a preset time interval and the anomaly reference information. The anomaly tracing module is used to trace the anomaly based on the memory allocation event corresponding to the anomaly path signature and obtain anomaly tracing information.

[0013] In one possible implementation, the event logging module is further configured to: Collect at least one of the following information corresponding to off-heap memory allocation: event identifier, allocation time, allocation size, calling thread, call stack, interface type, and business context information, and record the corresponding memory allocation event; The corresponding path signature is determined based on the recorded memory allocation event content.

[0014] In one possible implementation, the event logging module is further configured to: Determine the hash value of the call stack in the memory allocation event; Generate a corresponding path signature based on at least one of the hash value, the interface type, and the business context information.

[0015] In one possible implementation, the reference information acquisition module is further configured to: Periodically collect the physical memory of the process to obtain the process physical memory sequence; Garbage collection events are recorded periodically to obtain a garbage collection event sequence; Anomaly reference information is determined based on the process physical memory sequence and the garbage collection event sequence.

[0016] In one possible implementation, the anomaly detection module is further configured to: Based on the memory allocation events corresponding to each path signature within a preset time interval, determine the off-heap memory information corresponding to the path signature; For each path signature, anomaly identification is performed based on the off-heap memory information, the anomaly reference information within the preset time interval, and the preset anomaly identification rules to determine whether the path signature is an abnormal path signature.

[0017] In one possible implementation, the anomaly detection module is further configured to: For each path signature, the corresponding memory allocation event is obtained by sliding within the preset time interval using a preset time window, and the cumulative allocation amount within the corresponding time window, the residual allocation amount at the end of the time window, the allocation growth amount compared with the previous time window, and the ratio of the residual allocation amount to the global residual amount are statistically analyzed. The off-heap memory information corresponding to the path signature is determined based on at least one of the following: the cumulative allocation amount, the residual allocation amount, the allocation growth amount, and the ratio of the residual allocation amount to the global residual amount for each time window.

[0018] In one possible implementation, the anomaly reference information includes a process physical memory sequence and a garbage collection event sequence, and the anomaly identification rule includes at least one of the following: The cumulative allocation and / or residual allocation corresponding to the first number of consecutive time windows of the path signature continuously increase; Within the time interval corresponding to the second number of garbage collection events in the garbage collection event sequence, the change in the cumulative allocation amount corresponding to the path signature does not meet the preset condition; The changes in the allocation growth of the multiple time windows corresponding to the path signature are synchronized with the changes in the physical memory of each process in the process physical memory sequence; The ratio of the residual allocation amount corresponding to the path signature to the global residual amount is greater than a preset threshold.

[0019] In one possible implementation, the anomaly tracing module is further configured to: Obtain the call stack corresponding to the abnormal path signature to determine the corresponding business module; The corresponding business interface and tenant are determined based on the business context information corresponding to the abnormal path signature; The anomaly tracing information is determined based on the business module, business interface, and tenant.

[0020] The electronic device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0021] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method provided in this application embodiment.

[0022] The off-heap memory anomaly tracing method, device, and storage medium provided in this application, in response to off-heap memory allocation, record the corresponding memory allocation event and determine the path signature of the memory allocation event. The path signature is used to group the memory allocation events. Process physical memory and garbage collection events are periodically collected to obtain corresponding anomaly reference information. Based on the memory allocation events corresponding to each path signature within a preset time interval, and the anomaly reference information, abnormal path signatures indicating off-heap memory anomalies are identified. Anomaly tracing is performed based on the memory allocation events corresponding to the abnormal path signatures to obtain anomaly tracing information. This application achieves fine-grained anomaly localization by setting path signatures and uses heap memory as an anomaly reference for anomaly identification, ensuring that anomaly identification is closer to the business scenario. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This diagram illustrates a method for tracing off-heap memory anomalies according to an embodiment of this application. Figure 2 This diagram illustrates an off-heap memory anomaly tracing device according to an embodiment of this application; Figure 3 A schematic diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0027] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0028] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0029] The off-heap memory anomaly tracing method of this application can be executed by any electronic device, including but not limited to mobile phones, wearable devices (such as smartwatches, smart bracelets, smart glasses, etc.), tablet computers, laptops, vehicle terminals, PCs (Personal Computers), etc. The function implemented by this method can be achieved by the processor in the electronic device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the electronic device includes at least a processor and a storage medium.

[0030] The off-heap memory anomaly tracing method of this application embodiment can be used in any application scenario that requires off-heap memory anomaly tracing, such as the application scenario of off-heap memory anomaly tracing in financial trading and high-frequency trading systems, and the application scenario of off-heap memory anomaly tracing in big data processing and real-time computing platforms.

[0031] The Java Virtual Machine (JVM) heavily utilizes off-heap memory in high-concurrency, low-latency applications to improve I / O and data processing efficiency. Typical off-heap memory includes direct memory allocated via `ByteBuffer.allocateDirect()` and memory-mapped files allocated via `FileChannel.map()`. Off-heap memory refers to memory regions allocated outside the Java heap but indirectly accessible to Java applications. In the current approach, this primarily refers to direct memory requested via `ByteBuffer.allocateDirect()` and memory-mapped regions created via `FileChannel.map()`. This type of memory is not part of the storage area where ordinary Java heap objects reside; therefore, its usage cannot be directly determined by observing changes in the size of objects within the heap. Direct memory is a typical form of off-heap memory, usually allocated using Java standard APIs and used for network I / O, buffer processing, and other scenarios. Its characteristic is that the Java heap typically only retains wrapper objects or access handles, while the actual data buffer resides in an off-heap area. Memory-mapped regions are off-heap memory regions formed through file mapping mechanisms, allowing applications to directly access file content through mapped buffers. This method is commonly used in scenarios such as large file reading and writing, log processing, and index access. Optionally, memory leak refers to the phenomenon where a certain type of memory resource continues to be held, cannot be released for a long time, or continues to grow after it no longer has reasonable business value, resulting in an abnormal increase in process memory usage. This application's embodiments focus more on whether memory resources generated by standard Java off-heap entry points are abnormally retained for a long time, rather than being limited to the narrow expression of "objects can never be released" in the traditional sense. This abnormal retention refers to the state in which the off-heap memory usage corresponding to a certain call path continues to increase within multiple observation windows, or does not significantly decrease after the business peak ends and after multiple time periods. Abnormal retention is not necessarily equivalent to "definite leakage" in the final legal or engineering conclusion, but it can serve as an important technical feature for identifying high-risk leakage paths.

[0032] The related technologies have the following drawbacks when tracing off-heap memory anomalies: The granularity of localization is limited. Existing solutions mostly use JVM internal subsystems, process-level indicators, or overall memory changes as observation objects. They can usually reflect the phenomenon of "growth of off-heap memory", but it is difficult to further locate the specific business call path, business module, or business interface. Therefore, they are insufficient to directly support code-level investigation.

[0033] The anomaly detection methods are rather crude. Existing solutions rely heavily on total thresholds, duration thresholds, or human experience for judgment, which has limited ability to distinguish between scenarios with continuous small accumulations, intermittent growth, or those mixed with normal business peaks, and is prone to false alarms or missed alarms.

[0034] The lack of call path-level correlation analysis means that while solutions can usually answer questions like "how much has increased?" or "which addresses are abnormal", they do not adequately reveal the correlation between standard Java off-heap allocation entry points, call paths, time window changes, and business behaviors.

[0035] Lacking attribution capabilities at the business semantic level, existing technologies mostly obtain technical indicators, address information, system indicators, or JVM internal statistics, which usually cannot be reliably mapped to business modules, business interfaces, request chains, or tenant dimensions.

[0036] Some implementation methods have version compatibility issues. Some solutions that rely on JDK inner classes, inner fields, or specific implementation details may have high adaptation costs and insufficient stability between different JDK versions or different virtual machine implementations.

[0037] Therefore, the technical problem solved by the embodiments of this application is how to achieve accurate and precise location of off-heap memory anomalies.

[0038] The following describes in detail the off-heap memory anomaly tracing scheme of this application embodiment with reference to the accompanying drawings.

[0039] Figure 1 A flowchart illustrating a method for tracing off-heap memory anomalies according to an embodiment of this application is shown. Figure 1 As shown, the off-heap memory anomaly tracing method of this application embodiment may include the following steps S10-S40.

[0040] For ease of description, the off-heap memory anomaly tracing method of this application embodiment is described using an electronic device as the execution subject. It should be understood that the execution subject of this application embodiment can also be a processor or chip in an electronic device, and this application embodiment does not impose any limitations.

[0041] Step S10: In response to off-heap memory allocation, record the corresponding memory allocation event and determine the path signature of the memory allocation event.

[0042] In one possible implementation, the electronic device can record the memory allocation event upon detecting off-heap memory allocation. Specifically, the off-heap memory allocation events in this embodiment may include off-heap memory allocation events invoked using java.nio.ByteBuffer.allocateDirect(int capacity) as the standard Application Programming Interface (API) entry point, off-heap memory allocation events invoked using java.nio.channels.FileChannel.map(MapMode mode, long position, long size) as the standard API entry point, and other off-heap memory allocation events invoked using API entry points that can be confirmed as standard Java off-heap allocation entry points.

[0043] Optionally, when the preset Java application starts, the electronic device can load a Java Agent to perform bytecode enhancement at the aforementioned standard API entry points. This allows the device to collect and record relevant call information when any API entry point is invoked, thereby obtaining the corresponding memory allocation event. The relevant call information recorded by the electronic device when an API entry point is invoked may include event identifier, allocation time, allocation size, calling thread, call stack or stack summary, API (interface) type, and business context information. Specifically, when the electronic device detects off-heap memory allocation, it collects at least one of the following: event identifier, allocation time, allocation size, calling thread, call stack, interface type, and business context information, obtaining and recording the corresponding memory allocation event. The corresponding path signature is then determined based on the recorded memory allocation event content.

[0044] In some embodiments, the event identifier is a unique ID for this allocation event, used for subsequent association, deduplication, or log tracing. The allocation time is used to characterize the precise timestamp of this allocation. The allocation size is used to characterize the number of bytes of off-heap memory allocated in this request. The calling thread is used to characterize the name and ID of the Java thread executing this allocation, helping to distinguish the source of concurrent requests. The call stack or stack summary is used to characterize the method call chain that led to this allocation, allowing for complete preservation or extraction of keyframes. The interface type is used to characterize the standard off-heap allocation entry point used in this allocation, distinguishing different memory types. Business context information is business-dimensional data captured from ThreadLocal, MDC, framework interceptors, or explicit calls when the allocation event occurs. Business context information may include tenant identifiers, request chain identifiers, business modules, business interfaces, user identifiers, session identifiers, and custom tags, etc.

[0045] In other embodiments, when a memory allocation event is determined, the electronic device determines a corresponding path signature based on the recorded content of the memory allocation event. This path signature is used to group memory allocation events, and its determination process may include determining the hash value of the call stack in the memory allocation event, and generating a corresponding path signature based on at least one of the hash value, interface type, and business context information. In other words, the electronic device can generate a call path signature for the collected allocation event by extracting the business frames from the call stack. The path signature can be determined based on at least a portion of the hash value of the call stack, the interface type, and the business context information.

[0046] Step S20: Periodically collect process physical memory and garbage collection events to obtain corresponding anomaly reference information.

[0047] In one possible implementation, the electronic device can also periodically collect process physical memory and garbage collection events to obtain corresponding anomaly reference information, which can be used as a reference for identifying off-heap memory anomalies. The process by which the electronic device determines the anomaly reference information may include periodically collecting process physical memory to obtain a process physical memory sequence; periodically recording garbage collection events (GC event occurrence times) to obtain a garbage collection event sequence; and determining the anomaly reference information based on the process physical memory sequence and the garbage collection event sequence.

[0048] Optionally, GC events and process physical memory recorded by electronic devices are not used as a direct means of observing off-heap memory, nor are they regarded as the direct subject of off-heap memory release. Instead, they are used as time reference events to observe whether the off-heap memory corresponding to a certain call path has not significantly decreased after multiple GC cycles.

[0049] Step S30: Identify abnormal path signatures with off-heap memory anomalies based on the memory allocation events corresponding to each path signature within a preset time interval and the abnormal reference information.

[0050] In one possible implementation, the electronic device can classify different memory allocation events based on the path signatures corresponding to each memory allocation event. Furthermore, the electronic device can also perform statistical analysis on the memory allocation events corresponding to each type of path signature to obtain off-heap memory information, and identify abnormal path signatures with off-heap memory anomalies among multiple path signatures based on the off-heap memory information and anomaly reference information. For example, the electronic device can determine the off-heap memory information corresponding to a path signature based on the memory allocation events corresponding to each path signature within a preset time interval. Then, for each path signature, anomaly identification is performed based on the off-heap memory information, the anomaly reference information within the preset time interval, and preset anomaly identification rules to determine whether the path signature is an abnormal path signature.

[0051] Optionally, for each path signature, the electronic device slides to acquire the corresponding memory allocation events within a preset time interval using a preset time window, and calculates the cumulative allocation amount within the corresponding time window, the residual allocation amount at the end of the time window, the allocation growth compared to the previous time window, and the ratio of the residual allocation amount to the global residual amount. The off-heap memory information corresponding to the path signature is determined based on at least one of the following: cumulative allocation amount, residual allocation amount, allocation growth, and the ratio of the residual allocation amount to the global residual amount for each time window. The cumulative allocation amount can be the sum of the allocation sizes of all memory allocation events within the corresponding time window, and the residual allocation amount at the end of the time window can be the total amount of off-heap memory belonging to that path signature but not yet released at the end of the current time window. The allocation growth is the trend of change in the residual allocation amount or cumulative allocation amount of the current time window relative to the allocation amount corresponding to the previous time window, which can be represented by the difference in residual allocation amount or cumulative allocation amount between adjacent time windows. The global residual amount can be the sum of the residual allocation amounts corresponding to all path signatures within the current time window.

[0052] In some embodiments, based on the content of the aforementioned anomaly reference information, the electronic device may pre-set anomaly identification rules including: the cumulative allocation amount and / or residual allocation amount corresponding to a first number of consecutive time windows of the path signature continuously increases; within the time interval corresponding to a second number of garbage collection events in the garbage collection event sequence, the change in the cumulative allocation amount corresponding to the path signature does not meet a preset condition; the change in the allocation growth amount of multiple time windows corresponding to the path signature is synchronized with the change in the physical memory of each process in the process physical memory sequence; and the ratio of the residual allocation amount corresponding to the path signature to the global residual amount is greater than a preset threshold. That is, when the electronic device determines that at least one of the above conditions is met based on the memory allocation events and anomaly reference information corresponding to each path signature, it determines that the off-heap memory corresponding to that path signature is abnormal, and that path signature is an abnormal path signature.

[0053] Optionally, the abnormal path in this application embodiment can be preferably understood as a path suspected of having abnormal delay or leakage risk. In other words, the identification result of this application embodiment is first a "high-risk path location result", which can then be further confirmed by development or operation and maintenance personnel in combination with code logic, business scenarios and manual investigation to determine whether it constitutes a memory leak in the final sense.

[0054] Step S40: Perform anomaly tracing based on the memory allocation event corresponding to the abnormal path signature to obtain anomaly tracing information.

[0055] In one possible implementation, after determining the abnormal path signature, the electronic device can perform anomaly tracing based on the memory allocation event corresponding to the abnormal path signature to obtain anomaly tracing information through business semantic attribution. Optionally, the electronic device can obtain the call stack corresponding to the abnormal path signature to determine the corresponding business module, and determine the corresponding business interface and tenant based on the business context information corresponding to the abnormal path signature. Anomaly tracing information is then determined based on the business module, business interface, and tenant.

[0056] For example, the specific attribution logic in this application embodiment may include: mapping to a business module based on the business package name prefix in the call stack; mapping to a business interface based on the thread context, interceptor context, or request context; mapping to the request chain based on the MDC, TraceId, or call chain context; and mapping to the tenant dimension based on the tenant context. The thread context, interceptor context, request context, as well as the MDC, TraceId, call chain context, and tenant context can be obtained in the memory allocation event.

[0057] Optionally, the anomaly tracing information determined by the electronic device may include business modules, business interfaces, and tenants, as well as at least one of the following: anomaly path, request link identifier, anomaly type, and troubleshooting suggestions. Based on the above anomaly tracing information, this information no longer only includes suspected call stacks but also business-understandable location results. The electronic device can write the results to the storage layer and display an overview trend, suspicious path ranking, business attribution hotspots, and single path details on the front end to assist in troubleshooting. The output results can graphically display the global trend, suspicious path ranking, and single path details, supporting drill-down layer by layer for quick location and continuous tracking.

[0058] Based on the aforementioned technical features, this application's embodiments elevate the analysis object from virtual addresses or process-level indicators to standard off-heap allocation entry events and call path signatures when identifying off-heap memory anomalies. This allows for direct location of the business call path rather than merely the JVM subsystem or process level. Furthermore, by comprehensively judging residual growth within a continuous time window, non-fallback characteristics across multiple GC cycles, path-level growth concentration, and synchronization with changes in process physical memory, false positives and false negatives are significantly reduced, enabling stable identification of abnormal persistence rather than simply relying on static thresholds. Simultaneously, this application's embodiments automatically map abnormal paths to business semantic dimensions such as business modules, business interfaces, request chains, and tenants during the source tracing process. This allows development and operations personnel to directly obtain actionable investigation information, rather than just technical stack or address information. When recording off-heap allocation events, the electronic device selects to set the collection point at standard Java API entry points (such as ByteBuffer.allocateDirect and FileChannel.map), without relying on JDK internal private fields, thus ensuring good cross-version compatibility. In summary, the embodiments of this application significantly reduce the threshold and time cost of troubleshooting Java off-heap memory anomalies, and are especially suitable for memory-sensitive business scenarios such as financial transactions, big data processing, and multi-tenant SaaS.

[0059] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0060] Based on the foregoing embodiments, this application provides an off-heap memory anomaly tracing device. The device includes various modules and units included in each module, which can be implemented by a processor; of course, it can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP), or field programmable gate array (FPGA), etc.

[0061] Figure 2 This diagram illustrates an off-heap memory anomaly tracing device according to an embodiment of this application. Figure 2 As shown, the off-heap memory anomaly tracing device in this embodiment includes: The event logging module 20 is used to record the corresponding memory allocation event and determine the path signature of the memory allocation event in response to off-heap memory allocation. The path signature is used to group the memory allocation events. Reference information acquisition module 21 is used to periodically collect process physical memory and garbage collection events to obtain corresponding abnormal reference information; The anomaly identification module 22 is used to identify abnormal path signatures that have off-heap memory anomalies based on the memory allocation events corresponding to each path signature within a preset time interval and the anomaly reference information. The anomaly tracing module 23 is used to perform anomaly tracing based on the memory allocation event corresponding to the anomaly path signature, and obtain anomaly tracing information.

[0062] In one possible implementation, the event logging module 20 is further configured to: Collect at least one of the following information corresponding to off-heap memory allocation: event identifier, allocation time, allocation size, calling thread, call stack, interface type, and business context information, and record the corresponding memory allocation event; The corresponding path signature is determined based on the recorded memory allocation event content.

[0063] In one possible implementation, the event logging module 20 is further configured to: Determine the hash value of the call stack in the memory allocation event; Generate a corresponding path signature based on at least one of the hash value, the interface type, and the business context information.

[0064] In one possible implementation, the reference information acquisition module 21 is further configured to: Periodically collect the physical memory of the process to obtain the process physical memory sequence; Garbage collection events are recorded periodically to obtain a garbage collection event sequence; Anomaly reference information is determined based on the process physical memory sequence and the garbage collection event sequence.

[0065] In one possible implementation, the anomaly detection module 22 is further configured to: Based on the memory allocation events corresponding to each path signature within a preset time interval, determine the off-heap memory information corresponding to the path signature; For each path signature, anomaly identification is performed based on the off-heap memory information, the anomaly reference information within the preset time interval, and the preset anomaly identification rules to determine whether the path signature is an abnormal path signature.

[0066] In one possible implementation, the anomaly detection module 22 is further configured to: For each path signature, the corresponding memory allocation event is obtained by sliding within the preset time interval using a preset time window, and the cumulative allocation amount within the corresponding time window, the residual allocation amount at the end of the time window, the allocation growth amount compared with the previous time window, and the ratio of the residual allocation amount to the global residual amount are statistically analyzed. The off-heap memory information corresponding to the path signature is determined based on at least one of the following: the cumulative allocation amount, the residual allocation amount, the allocation growth amount, and the ratio of the residual allocation amount to the global residual amount for each time window.

[0067] In one possible implementation, the anomaly reference information includes a process physical memory sequence and a garbage collection event sequence, and the anomaly identification rule includes at least one of the following: The cumulative allocation and / or residual allocation corresponding to the first number of consecutive time windows of the path signature continuously increase; Within the time interval corresponding to the second number of garbage collection events in the garbage collection event sequence, the change in the cumulative allocation amount corresponding to the path signature does not meet the preset condition; The changes in the allocation growth of the multiple time windows corresponding to the path signature are synchronized with the changes in the physical memory of each process in the process physical memory sequence; The ratio of the residual allocation amount corresponding to the path signature to the global residual amount is greater than a preset threshold.

[0068] In one possible implementation, the anomaly tracing module 23 is further configured to: Obtain the call stack corresponding to the abnormal path signature to determine the corresponding business module; The corresponding business interface and tenant are determined based on the business context information corresponding to the abnormal path signature; The anomaly tracing information is determined based on the business module, business interface, and tenant.

[0069] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0070] It should be noted that, in the embodiments of this application... Figure 2The module division of the off-heap memory anomaly tracing device shown is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or be integrated into one unit from two or more units. The integrated units can be implemented in hardware, as software functional units, or a combination of both.

[0071] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0072] Figure 3 A schematic diagram of an electronic device according to an embodiment of this application is shown. For example... Figure 3 As shown in the figure, this application provides an electronic device, which can be a server, and its internal structure diagram can be as follows. Figure 3 As shown, the electronic device includes a processor 320, a memory, and a transceiver 340 connected via a system bus 310. The processor 320 provides computing and control capabilities. The memory includes a non-volatile storage medium 331 and internal memory 332. The non-volatile storage medium 331 stores an operating system, computer programs, and a database. The internal memory 332 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium 331. The database stores data. The transceiver 340 communicates with external terminals via a network connection. The computer program is executed by the processor 320 to implement the aforementioned methods.

[0073] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor 320, implements the steps of the method provided in the above embodiments.

[0074] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0075] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0076] In one possible implementation, the apparatus provided in this application can be implemented as a computer program, which can be configured as follows: Figure 3 The device operates on the electronic device shown. The memory of the electronic device can store the various program modules that make up the above-described apparatus. The computer program composed of the various program modules causes the processor 320 to execute the steps of the methods in the various embodiments of this application described in this specification.

[0077] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0078] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, phrases such as "in one possible implementation," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0079] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0080] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0081] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0082] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0083] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0084] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0085] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0086] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0087] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0088] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0089] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for tracing the source of off-heap memory anomalies, characterized in that, The method includes: In response to off-heap memory allocation, the corresponding memory allocation event is recorded and the path signature of the memory allocation event is determined. The path signature is used to group the memory allocation events. Periodically collect data on process physical memory and garbage collection events to obtain corresponding anomaly reference information; Based on the memory allocation events corresponding to each path signature within a preset time interval, and the abnormal reference information, abnormal path signatures with off-heap memory abnormalities are identified. The abnormal source is traced based on the memory allocation event corresponding to the abnormal path signature to obtain abnormal source information.

2. The method according to claim 1, characterized in that, The process of recording the corresponding memory allocation event and determining the path signature of the memory allocation event includes: Collect at least one of the following information corresponding to off-heap memory allocation: event identifier, allocation time, allocation size, calling thread, call stack, interface type, and business context information, and record the corresponding memory allocation event; The corresponding path signature is determined based on the recorded memory allocation event content.

3. The method according to claim 2, characterized in that, The step of determining the corresponding path signature based on the recorded memory allocation event content includes: Determine the hash value of the call stack in the memory allocation event; Generate a corresponding path signature based on at least one of the hash value, the interface type, and the business context information.

4. The method according to claim 1, characterized in that, The periodic collection of physical memory and garbage collection events during the process yields corresponding anomaly reference information, including: Periodically collect the physical memory of the process to obtain the process physical memory sequence; Garbage collection events are recorded periodically to obtain a garbage collection event sequence; Anomaly reference information is determined based on the process physical memory sequence and the garbage collection event sequence.

5. The method according to claim 1, characterized in that, The step of identifying abnormal path signatures with off-heap memory anomalies based on the memory allocation events corresponding to each path signature within a preset time interval and the abnormal reference information includes: Based on the memory allocation events corresponding to each path signature within a preset time interval, determine the off-heap memory information corresponding to the path signature; For each path signature, anomaly identification is performed based on the off-heap memory information, the anomaly reference information within the preset time interval, and the preset anomaly identification rules to determine whether the path signature is an abnormal path signature.

6. The method according to claim 5, characterized in that, The step of determining the off-heap memory information corresponding to each path signature based on the memory allocation events corresponding to each path signature within a preset time interval includes: For each path signature, the corresponding memory allocation event is obtained by sliding within the preset time interval using a preset time window, and the cumulative allocation amount within the corresponding time window, the residual allocation amount at the end of the time window, the allocation growth amount compared with the previous time window, and the ratio of the residual allocation amount to the global residual amount are statistically analyzed. The off-heap memory information corresponding to the path signature is determined based on at least one of the following: the cumulative allocation amount, the residual allocation amount, the allocation growth amount, and the ratio of the residual allocation amount to the global residual amount for each time window.

7. The method according to claim 6, characterized in that, The anomaly reference information includes a process physical memory sequence and a garbage collection event sequence, and the anomaly identification rule includes at least one of the following: The cumulative allocation and / or residual allocation corresponding to the first number of consecutive time windows of the path signature continuously increase; Within the time interval corresponding to the second number of garbage collection events in the garbage collection event sequence, the change in the cumulative allocation amount corresponding to the path signature does not meet the preset condition; The changes in the allocation growth of the multiple time windows corresponding to the path signature are synchronized with the changes in the physical memory of each process in the process physical memory sequence; The ratio of the residual allocation amount corresponding to the path signature to the global residual amount is greater than a preset threshold.

8. The method according to claim 3, characterized in that, The step of tracing the source of the anomaly based on the memory allocation event corresponding to the anomaly path signature to obtain anomaly source information includes: Obtain the call stack corresponding to the abnormal path signature to determine the corresponding business module; The corresponding business interface and tenant are determined based on the business context information corresponding to the abnormal path signature; The anomaly tracing information is determined based on the business module, business interface, and tenant.

9. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images