Abnormality detection method and device for data interaction request and nonvolatile storage medium

Abstract

The invention discloses a data interaction request anomaly detection method and device and a nonvolatile storage medium. The method comprises the following steps: determining a plurality of mark points on a data interaction path of target storage equipment; when a target storage device receives a real-time data interaction request, respectively obtaining moments for triggering a plurality of mark points, triggering stage identifiers of the plurality of mark points, and obtaining a plurality of timestamps and stage identifiers corresponding to the plurality of timestamps; calculating respective delay time of the plurality of data interaction stages based on the plurality of timestamps and the stage identifiers corresponding to the plurality of timestamps; respectively judging the respective delay time of the plurality of data interaction stages and the respective preset delay time threshold of the plurality of data interaction stages; and determining the data interaction stage of which the delay time is greater than a preset delay time threshold as an abnormal interaction stage. According to the invention, the technical problem that high-precision and low-overhead monitoring of input / output delay faults is difficult to realize in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of data communication technology, and more specifically, to a method, apparatus, and non-volatile storage medium for detecting anomalies in data interaction requests. Background Technology

[0002] In modern operating systems, especially Linux-based servers and embedded devices, the input / output (I / O) performance of block devices (such as hard disk drives (HDDs) and solid-state drives (SSDs)) has a crucial impact on system response speed, application efficiency, and even the stability of the entire system. In applications such as high-performance computing, database management, virtualization, and large-scale data processing, accurate monitoring and fault diagnosis of block I / O latency are key technologies for ensuring smooth system operation and improving user experience. However, excessively high I / O latency can lead to degraded application performance, a worsened user experience, and even compromised system stability.

[0003] Traditional Linux block I / O latency monitoring tools have certain limitations. First, they struggle to efficiently acquire and monitor the latency details of individual I / O requests. Their data capture methods tend to collect large amounts of data, lacking the ability to accurately measure the latency of specific I / O requests. Second, for analyzing large datasets, traditional tools have relatively low filtering efficiency, which not only increases the complexity of data analysis but also limits the practicality of monitoring tools in real-world production environments. Finally, the massive amounts of data generated by traditional tools consume significant system resources, both in terms of storage and transmission. This resource consumption is a substantial technical bottleneck, especially in high-concurrency and large-data-volume scenarios.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method, apparatus, and non-volatile storage medium for detecting anomalies in data interaction requests, thereby at least solving the technical problem that current technologies struggle to achieve high-precision, low-overhead monitoring of input / output delay faults.

[0006] According to one aspect of the present invention, an anomaly detection method for data interaction requests is provided, comprising: determining multiple marker points on a data interaction path of a target storage device, wherein the data interaction path is a data interaction path between the target storage device and an operating system, and the marker points represent the occurrence positions of multiple data interaction stages on the data interaction path; when the target storage device receives a real-time data interaction request, acquiring the time when the multiple marker points are triggered and the stage identifiers of the multiple marker points, thereby obtaining multiple timestamps and the stage identifiers corresponding to the multiple timestamps; calculating the delay time of each of the multiple data interaction stages based on the multiple timestamps and the stage identifiers corresponding to the multiple timestamps; determining the magnitude of the delay time of each of the multiple data interaction stages and the preset delay time threshold of each of the multiple data interaction stages; and determining the data interaction stages with delay times greater than the preset delay time thresholds as abnormal interaction stages.

[0007] Optionally, based on multiple timestamps and their corresponding stage identifiers, the delay time for each of the multiple data interaction stages is calculated, including: taking two timestamps with matching stage identifiers as a group of stage timestamps to obtain multiple groups of stage timestamps, wherein each group of stage timestamps includes a start timestamp and a completion timestamp, and the completion timestamp is later than the start timestamp; and calculating the difference between the completion timestamp and the start timestamp in each stage timestamp to obtain the delay time for each of the multiple data interaction stages.

[0008] Optionally, it also includes storing multiple timestamps and their corresponding stage identifiers in a preset data map, wherein the preset data map is a data structure used to extend the Berkeley package filter program to store and share data.

[0009] Optionally, it also includes: obtaining runtime information corresponding to the abnormal interaction stage, wherein the runtime information includes a completion timestamp, an identifier of the target storage device, an identifier of the real-time data interaction request, a process identifier of the real-time data interaction request, a stage type, the starting sector of the real-time data interaction request, and the data size corresponding to the real-time data interaction request; and transmitting the runtime information to the user terminal through a preset communication mechanism, wherein the preset communication mechanism includes a performance event buffer or a circular buffer.

[0010] Optionally, it also includes: performing multi-dimensional feature analysis on the operation information to obtain the operation frequency of real-time data interaction requests and the data processing volume threshold of the target storage device; and determining the cause of the abnormal interaction stage based on the operation frequency, the data processing volume threshold, and the delay time corresponding to the abnormal interaction stage, wherein the cause of the abnormal interaction stage includes request timeout, read-write ratio imbalance, and operating system memory abnormality.

[0011] Optionally, it also includes: constructing a dynamic Sankey diagram based on the runtime information and the cause of the abnormal interaction phase, wherein the dynamic Sankey diagram includes the delay time of each of the multiple data interaction phases and the relative traffic size of each of the multiple data interaction phases, wherein the relative traffic size represents the weight of the data interaction phase in the entire data interaction process; and displaying the dynamic Sankey diagram on a preset display platform.

[0012] According to another aspect of the present invention, an anomaly detection device for data interaction requests is also provided, comprising: a first determining module, configured to determine multiple marker points on a data interaction path of a target storage device, wherein the data interaction path is a data interaction path between the target storage device and an operating system, and the marker points represent the occurrence positions of multiple data interaction stages on the data interaction path; an acquiring module, configured to acquire, when the target storage device receives a real-time data interaction request, the time when triggering multiple marker points and the stage identifiers of triggering multiple marker points, thereby obtaining multiple timestamps and stage identifiers corresponding to each of the multiple timestamps; a calculating module, configured to calculate the delay time of each of the multiple data interaction stages based on the multiple timestamps and the stage identifiers corresponding to each of the multiple timestamps; a judging module, configured to judge the magnitude of the delay time of each of the multiple data interaction stages and the preset delay time threshold of each of the multiple data interaction stages; and a second determining module, configured to determine the data interaction stages with delay times greater than the preset delay time thresholds as abnormal interaction stages.

[0013] According to another aspect of the present invention, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, the device where the non-volatile storage medium is located executes any of the above-described data interaction request anomaly detection methods.

[0014] According to another aspect of the present invention, a computer device is also provided, the computer device including a processor, the processor being configured to run a program, wherein the program, when running, executes any of the above-described methods for detecting abnormal data interaction requests.

[0015] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements an anomaly detection method for any of the above-described data interaction requests.

[0016] In this embodiment of the invention, an anomaly detection method for data interaction requests is employed. This method identifies multiple marker points on the data interaction path of the target storage device, where the data interaction path is the data interaction path between the target storage device and the operating system. The marker points represent the locations of multiple data interaction stages along the data interaction path. When the target storage device receives a real-time data interaction request, the time when multiple marker points are triggered and the stage identifiers of the triggering marker points are obtained, resulting in multiple timestamps and their corresponding stage identifiers. Based on the multiple timestamps and their corresponding stage identifiers, the delay time of each of the multiple data interaction stages is calculated. The delay time of each of the multiple data interaction stages is then compared to a preset delay time threshold. Data interaction stages with delay times greater than the preset delay time thresholds are identified as abnormal interaction stages. This achieves high-precision, low-overhead diagnosis of operating system input / output latency faults, thereby realizing the technical effect of efficiently monitoring single input / output latency and performing kernel-level fault diagnosis without significantly increasing system load. This solves the technical problem that current technologies struggle to achieve high-precision, low-overhead monitoring of input / output latency faults. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a computer terminal for implementing an anomaly detection method for data interaction requests is shown.

[0019] Figure 2 This is a flowchart illustrating an anomaly detection method for data interaction requests provided according to an embodiment of the present invention;

[0020] Figure 3 This is an input / output process data flow diagram provided according to an optional embodiment of the present invention;

[0021] Figure 4 This is a structural block diagram of an anomaly detection device for data interaction requests provided according to an optional embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0025] I / O (Input / Output): System input / output.

[0026] eBPF (Extended Berkeley Packet Filter) is a revolutionary kernel technology that allows users to run sandboxed programs securely and efficiently without modifying the kernel source code or loading kernel modules. It enables real-time monitoring of the operating system kernel and applications, network filtering, performance analysis, and other functions.

[0027] According to an embodiment of the present invention, an embodiment of an anomaly detection method for data interaction requests is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0028] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing an anomaly detection method for data interaction requests is shown. Figure 1As shown, the computer terminal 10 may include one or more processors (shown as 102a, 102b, ..., 102n in the figure) (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0029] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10. As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0030] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the abnormal detection method for data interaction requests in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-mentioned abnormal detection method for data interaction requests of the application. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0031] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10.

[0032] Figure 2 This is a flowchart illustrating an anomaly detection method for data interaction requests provided according to an embodiment of the present invention, as shown below. Figure 2As shown, the method includes the following steps:

[0033] Step S201: Determine multiple marker points on the data interaction path of the target storage device, wherein the data interaction path is the data interaction path between the target storage device and the operating system, and the marker points represent the locations where multiple data interaction stages occur on the data interaction path.

[0034] In this step, a series of monitoring points or probe locations are pre-selected as markers along the data interaction path between the operating system and the target storage device, i.e., the I / O request processing path. These markers represent the key stages of the I / O request process from the operating system to the storage device and back. The selection of markers is based on in-depth analysis of the operation mechanism of the block-level I / O stack in the operating system kernel, covering key points along the I / O path from request creation, request merging, block-level scheduling, transmission to the storage device, to request completion. Specifically, eBPF technology can be used. eBPF technology allows specific monitoring programs, i.e., eBPF programs, to be inserted at the above markers without modifying the kernel source code or loading additional kernel modules.

[0035] Step S202: When the target storage device receives a real-time data interaction request, the time when multiple marker points are triggered and the stage identifier of the multiple marker points are obtained respectively, resulting in multiple timestamps and the stage identifiers corresponding to each of the multiple timestamps.

[0036] In this step, when the target storage device receives a real-time data interaction request, multiple pre-defined markers on the data interaction path are triggered. When the request triggers the first marker, the program instantly captures the precise moment associated with the request and assigns a specific stage identifier to each request, indicating that the data interaction request is in a specific stage of its corresponding data interaction path. Using eBPF technology, these moments are converted into high-precision timestamps, and the stage identifier is associated with information such as a pointer to each request, the target storage device identifier, the stage type, the starting sector, and the data size.

[0037] Specifically, the eBPF program is loaded into the Linux kernel and linked to various key points along the data interaction path. When a request passes through these points, the eBPF program runs automatically, using built-in time acquisition helper functions, such as `bpf_ktime_get_ns`, to record the timestamp. Simultaneously, to identify the specific processing stage of the request, the program stores the request's context information (such as pointers to `request` or `bio` structures) along with the timestamps in the eBPF map, enabling correlation and comparison at different stages of the request.

[0038] Step S203: Calculate the delay time for each of the multiple data interaction stages based on multiple timestamps and their corresponding stage identifiers.

[0039] In this step, when a request triggers the second marker, the eBPF program obtains the request's identifier and completion timestamp. Using the current identifier, it searches and retrieves a start timestamp with the same identifier in the eBPF storage space. Subsequently, based on the start and completion timestamps, it calculates the request's latency—that is, subtracting the start timestamp from the completion timestamp—to measure the time required to transition from one processing stage to the next. Stage identifiers are used to identify specific stages in this data sequence, ensuring that timestamps for different stages of the same request accurately correspond.

[0040] Step S204: Determine the delay time of each of the multiple data interaction stages and the size of the preset delay time threshold of each of the multiple data interaction stages.

[0041] In this step, after obtaining the latency times of each of the multiple data interaction stages, these latency times are analyzed based on preset latency thresholds. These preset latency thresholds are determined according to historical data, performance baselines, and the needs of specific business scenarios, serving as a reference standard to quickly locate abnormal data interaction stages. Specifically, the latency time of each stage can be compared with a preset latency threshold corresponding to that stage. Once the latency time of a stage exceeds its preset threshold, that stage is considered a potentially abnormal interaction stage, will be recorded on-site, and further fault diagnosis and performance analysis will be performed.

[0042] Step S205: The data interaction stage with a delay time greater than a preset delay time threshold is identified as an abnormal interaction stage.

[0043] In this step, after the latency of each of the multiple data interaction stages is calculated and compared with preset latency thresholds, if the latency of any stage is found to exceed its corresponding preset threshold, that stage is immediately identified by the program as an abnormal interaction stage. This judgment mechanism is based on thresholds set through in-depth analysis of historical performance data and business scenario requirements, aiming to quickly capture I / O processing stages that exceed normal operating ranges and may affect system performance or stability. After identifying abnormal interaction stages, these stages can be further focused on, and a more detailed fault diagnosis process can be executed to accurately locate performance bottlenecks or potential problems, providing direct guidance for system optimization and maintenance.

[0044] Through the above steps, the goal of high-precision, low-overhead diagnosis of operating system input / output delay faults is achieved. This enables efficient monitoring of individual input / output delays and kernel-level fault diagnosis without significantly increasing system load, thereby solving the technical problem that current technologies struggle to achieve high-precision, low-overhead monitoring of input / output delay faults.

[0045] As an optional embodiment, the delay time of each of the multiple data interaction stages is calculated based on multiple timestamps and the stage identifiers corresponding to each timestamp, including: taking two timestamps with matching stage identifiers as a group of stage timestamps to obtain multiple groups of stage timestamps, wherein each group of stage timestamps includes a start timestamp and a completion timestamp, and the completion timestamp is later than the start timestamp; in each stage timestamp, the difference between the completion timestamp and the start timestamp is calculated to obtain the delay time of each of the multiple data interaction stages.

[0046] Optionally, firstly, eBPF helper functions are used to capture multiple timestamps and stage identifiers related to the data interaction request. The timestamps precisely record the start and completion times of the request at different stages, while the stage identifiers track which specific processing stage the request is in on the data interaction path. Specifically, the start timestamp and stage identifier can be stored in an eBPF map. When the request triggers a second marker, the program uses the same eBPF helper function to obtain the current completion timestamp and searches the map for and matches the start timestamp of the same stage identifier. The program groups two timestamps with matching stage identifiers as a set of stage timestamps. Each set contains a start timestamp and a completion timestamp, and the completion timestamp is always later than the start timestamp, reflecting the entire processing cycle of the request in that stage. Secondly, after the stage timestamp groups are determined, a calculation operation can be performed: subtracting the start timestamp from the completion timestamp in each group yields the latency of the request within that specific stage. Finally, based on the latency of each of the multiple data interaction stages, the data is arranged and combined according to the data flow to statistically analyze I / O latency data for performance bottleneck analysis.

[0047] As an optional embodiment, it further includes storing multiple timestamps and their respective corresponding stage identifiers in a preset data map, wherein the preset data map is a data structure used to extend the Berkeley package filter program to store and share data.

[0048] Optionally, during the data interaction request processing, the Extended Berkeley Packet Filter (eBPF) technology is used to capture precise timestamps of multiple key points, as well as the corresponding stage identifiers for each timestamp. To ensure that this information can be efficiently stored and accessed later in the processing flow, all captured timestamps and stage identifiers are stored in a predefined data map. This data map is a storage and shared data structure specifically designed for eBPF programs, intended to provide a fast and secure data exchange mechanism between the kernel and user space. By storing timestamps and stage identifiers in this data map, the program can quickly retrieve the start and end timestamps of relevant requests based on the identifiers within the lifecycle of the data interaction request, thereby supporting subsequent latency calculations and fault diagnosis.

[0049] As an optional embodiment, it further includes: obtaining the running information corresponding to the abnormal interaction stage, wherein the running information includes a completion timestamp, an identifier of the target storage device, an identifier of the real-time data interaction request, a process identifier of the real-time data interaction request, a stage type, the starting sector of the real-time data interaction request, and the data size corresponding to the real-time data interaction request; and transmitting the running information to the user terminal through a preset communication mechanism, wherein the preset communication mechanism includes a performance event buffer or a circular buffer.

[0050] Optionally, after identifying the abnormal interaction phase, further detailed operational information related to that phase is obtained to support in-depth fault diagnosis and performance analysis. The operational information includes the completion timestamp, the identifier of the target storage device, the identifier of the real-time data interaction request, the process identifier of the real-time data interaction request, the phase type, the starting sector of the real-time data interaction request, and the data size corresponding to the real-time data interaction request. Specifically, the completion timestamp precisely records the moment the request is completed; the identifier of the target storage device, i.e., the device ID, is used to clearly identify the storage device involved in the abnormal request; the identifier of the real-time data interaction request ensures the uniqueness of each request, facilitating tracking and identification; the process identifier includes the process ID and process name, providing information about the process that initiated the request; the phase type, i.e., the operation type, indicates the nature or category of each key phase experienced by the data interaction request during processing; and the starting sector is the initial position of the request processing when performing data read / write operations on a disk or any block storage device.

[0051] Secondly, operational information is transmitted to the user terminal through a pre-defined communication mechanism, which includes a performance event buffer or a circular buffer. Specifically, the performance event buffer records and stores real-time performance data, including operational information during abnormal interaction phases; the user terminal can retrieve this data by reading from this buffer. The circular buffer provides a reusable storage mechanism, effectively managing the input and output of large amounts of data, reducing data transmission latency, and ensuring timely transmission of operational information. After receiving the operational information, the user terminal can use customized tools to provide relevant personnel with more intuitive performance fault reports, thereby accelerating the problem-solving process and improving overall system maintenance efficiency.

[0052] As an optional embodiment, it further includes: performing multi-dimensional feature analysis on the operation information to obtain the operation frequency of real-time data interaction requests and the data processing volume threshold of the target storage device; and determining the cause of the abnormal interaction stage based on the operation frequency, the data processing volume threshold, and the delay time corresponding to the abnormal interaction stage, wherein the cause of the abnormal interaction stage includes request timeout, read-write ratio imbalance, and operating system memory abnormality.

[0053] Optionally, the program first performs multi-dimensional feature analysis on the runtime information. One purpose of this feature analysis is to calculate the operation frequency, i.e., the number of data interaction requests per unit time. Simultaneously, feature analysis is also used to obtain the data processing capacity threshold of the target storage device. This threshold is used to measure the storage device's processing capacity limits and whether the current workload is approaching or exceeding this threshold, revealing whether the target storage device is experiencing performance saturation or overload.

[0054] Secondly, based on the results of the multi-dimensional feature analysis and the corresponding latency of the abnormal interaction phase, the specific cause of the abnormal interaction phase can be determined. Specifically, if the latency exceeds the preset range and the operation frequency is high, it indicates a request timeout, meaning that system resources cannot respond to I / O requests in a timely manner under high concurrency, resulting in processing latency. If there is a problem with the storage access mode, such as a large number of sequential reads, it may lead to cache invalidation, increase disk seek time, and cause an imbalance in the read-write ratio. If the latency is abnormal and the data processing volume is abnormal, it indicates a problem with the operating system's memory management, which may be due to inefficient system paging mechanism or unreasonable memory allocation leading to storage access latency.

[0055] As an optional embodiment, it further includes: constructing a dynamic Sankey diagram based on the running information and the cause of the abnormal interaction phase, wherein the dynamic Sankey diagram includes the delay time of each of the multiple data interaction phases and the relative traffic size of each of the multiple data interaction phases, wherein the relative traffic size represents the weight of the data interaction phase in the entire data interaction process; and displaying the dynamic Sankey diagram on a preset display platform.

[0056] Optionally, relative flow rate is a quantitative representation of the amount of data processed in each data interaction stage and its proportion in the entire data interaction process. It is used to characterize the weight of each stage and can be determined by comparing the data processing volume of each stage with the total data interaction volume, revealing the relative importance of a specific stage's contribution to the overall system data flow. A dynamic Sankey diagram is an advanced visualization tool that visually displays the flow of data between different stages and the processing efficiency of each stage through the connection and width variation of streamlines.

[0057] Specifically, a dynamic Sankey diagram is constructed based on operational information and the causes of abnormal interaction phases. Each phase in the diagram is represented as a node, the width of the streamlines between phases reflects the relative flow rate of that phase, and the color or style of the streamlines represents the latency. This feature allows the dynamic Sankey diagram to display the dynamic changes in data flow over time, revealing which phases have excessively long latency and whether the corresponding relative flow rate is reasonable, providing an intuitive visual basis for fault diagnosis and performance optimization. The display platform aims to provide an intuitive display environment for the dynamic Sankey diagram. It is a graphical user interface designed specifically for monitoring and diagnosis, with the ability to update data in real time and render dynamic charts. By integrating the dynamic Sankey diagram into the display platform, administrators can observe the I / O performance status in real time, quickly identify abnormal interaction phases, and then take corresponding optimization measures.

[0058] By deeply processing raw I / O monitoring data and employing multi-dimensional statistical models for feature analysis, core metrics such as IOPS, throughput, and latency are calculated in real time. It accurately captures abnormal states such as timeout requests and read / write imbalances. Analysis results are presented through interactive visualization components, including dynamic Sankey diagrams to display I / O path weights and 3D heatmaps to locate high-latency sectors. Ultimately, a comprehensive diagnostic report is generated, incorporating trend predictions, correlation analysis, and optimization suggestions, forming a complete closed loop from data collection to decision support.

[0059] As an optional embodiment, Figure 3 This is an input / output process data flow diagram provided according to an optional embodiment of the present invention. For example... Figure 3 As shown, in the I / O process data flow, data interaction requests are initiated from the application layer, go through a series of key stages, and are finally completed by the storage device to read or write data, and the results are fed back to the application layer. This process can be subdivided into the main stages of request creation, request merging, block layer scheduling, request delivery to the driver or device, and request completion.

[0060] This optional embodiment, based on eBPF technology, can be applied to multiple key stages in a data flow to achieve low-overhead I / O latency monitoring. Specifically, when a request is created, the eBPF program captures the timestamp and request information of this stage, recording the initial state of the I / O request. When the request is sent to the driver or device, the eBPF program similarly records the time information of this key point until the request is completed, at which point the timestamp is captured again, and the latency from request sending to completion is calculated. By adding eBPF programs before and after each key stage, this optional embodiment can accurately obtain latency information for each stage without significantly increasing system overhead.

[0061] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0062] Through the above description of the embodiments, those skilled in the art can clearly understand that the anomaly detection method for data interaction requests according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0063] According to embodiments of the present invention, an apparatus for implementing the above-described anomaly detection method for data interaction requests is also provided. Figure 4 This is a structural block diagram of an anomaly detection device for data interaction requests provided according to an embodiment of the present invention, such as... Figure 4 As shown, the device includes: a first determining module 41, an acquiring module 42, a calculating module 43, a judging module 44, and a second determining module 45. The device will be described below.

[0064] The first determining module 41 is used to determine multiple marker points on the data interaction path of the target storage device, wherein the data interaction path is the data interaction path between the target storage device and the operating system, and the marker points represent the respective occurrence positions of multiple data interaction stages on the data interaction path.

[0065] The acquisition module 42, connected to the first determination module 41, is used to acquire the time of triggering multiple marker points and the stage identifier of triggering multiple marker points respectively when the target storage device receives a real-time data interaction request, and obtain multiple timestamps and the stage identifiers corresponding to each of the multiple timestamps.

[0066] The calculation module 43, connected to the acquisition module 42, is used to calculate the delay time of each of the multiple data interaction stages based on multiple timestamps and the stage identifiers corresponding to each timestamp.

[0067] The judgment module 44, connected to the calculation module 43, is used to judge the delay time of each of the multiple data interaction stages and the size of the preset delay time threshold of each of the multiple data interaction stages.

[0068] The second determining module 45, connected to the judging module 44, is used to determine the data interaction stage with a delay time greater than a preset delay time threshold as an abnormal interaction stage.

[0069] It should be noted that the first determining module 41, the acquiring module 42, the calculating module 43, the judging module 44, and the second determining module 45 mentioned above correspond to steps S201 to S205 in the embodiments. Multiple modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in the embodiments.

[0070] Embodiments of the present invention may provide a computer device. Optionally, in this embodiment, the computer device may be located in at least one of a plurality of network devices in a computer network. The computer device includes a memory and a processor.

[0071] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the data interaction request anomaly detection method and apparatus in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned data interaction request anomaly detection method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0072] The processor can invoke information and applications stored in the memory via a transmission device to perform the following steps: determining multiple marker points on the data interaction path of the target storage device, wherein the data interaction path is the data interaction path between the target storage device and the operating system, and the marker points represent the occurrence positions of multiple data interaction stages on the data interaction path; when the target storage device receives a real-time data interaction request, obtaining the time that triggers the multiple marker points and the stage identifier that triggers the multiple marker points, obtaining multiple timestamps and their corresponding stage identifiers; calculating the delay time of each of the multiple data interaction stages based on the multiple timestamps and their corresponding stage identifiers; determining the magnitude of the delay time of each of the multiple data interaction stages and the preset delay time threshold of each of the multiple data interaction stages; and identifying data interaction stages with delay times greater than the preset delay time thresholds as abnormal interaction stages.

[0073] Optionally, the processor may also execute program code that performs the following steps: calculating the delay time of each of the multiple data interaction stages based on multiple timestamps and the stage identifiers corresponding to each of the multiple timestamps, including: taking two timestamps with matching stage identifiers as a group of stage timestamps to obtain multiple groups of stage timestamps, wherein a group of stage timestamps includes a start timestamp and a completion timestamp, and the completion timestamp is later than the start timestamp; calculating the difference between the completion timestamp and the start timestamp in each stage timestamp to obtain the delay time of each of the multiple data interaction stages.

[0074] Optionally, the processor may also execute program code that includes the following steps: storing multiple timestamps and their respective stage identifiers in a preset data map, wherein the preset data map is a data structure used to extend the Berkeley package filter program to store and share data.

[0075] Optionally, the processor may also execute program code that includes the following steps: obtaining runtime information corresponding to the abnormal interaction stage, wherein the runtime information includes a completion timestamp, an identifier of the target storage device, an identifier of the real-time data interaction request, a process identifier of the real-time data interaction request, a stage type, the starting sector of the real-time data interaction request, and the data size corresponding to the real-time data interaction request; and transmitting the runtime information to the user terminal through a preset communication mechanism, wherein the preset communication mechanism includes a performance event buffer or a circular buffer.

[0076] Optionally, the processor may also execute program code that includes the following steps: performing multi-dimensional feature analysis on the runtime information to obtain the operation frequency of the real-time data interaction request and the data processing volume threshold of the target storage device; and determining the cause of the abnormal interaction phase based on the operation frequency, the data processing volume threshold, and the delay time corresponding to the abnormal interaction phase, wherein the cause of the abnormal interaction phase includes request timeout, read / write ratio imbalance, and operating system memory anomaly.

[0077] Optionally, the processor may also execute program code that includes the following steps: constructing a dynamic Sankey diagram based on runtime information and the cause of the abnormal interaction phase, wherein the dynamic Sankey diagram includes the delay time of each of the multiple data interaction phases and the relative flow size of each of the multiple data interaction phases, wherein the relative flow size represents the weight of the data interaction phase in the entire data interaction process; and displaying the dynamic Sankey diagram on a preset display platform.

[0078] This invention provides a method for detecting anomalies in data interaction requests. By identifying multiple marker points on the data interaction path between the target storage device and the operating system, where the data interaction path represents the location of each of the multiple data interaction stages along the path; when the target storage device receives a real-time data interaction request, the method acquires the time that triggered the multiple marker points and the stage identifiers that triggered them, resulting in multiple timestamps and their corresponding stage identifiers; based on the timestamps and their corresponding stage identifiers, the method calculates the delay time for each of the multiple data interaction stages; it then compares the delay time of each of the multiple data interaction stages with a preset delay time threshold for each stage; and finally, it identifies data interaction stages with delay times exceeding the preset delay time thresholds as abnormal interaction stages. This achieves high-precision, low-overhead diagnosis of operating system input / output latency faults, thus solving the technical problem of current technologies struggling to achieve high-precision, low-overhead monitoring of input / output latency faults.

[0079] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a non-volatile storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0080] Embodiments of the present invention also provide a non-volatile storage medium. Optionally, in this embodiment, the aforementioned non-volatile storage medium can be used to store the program code executed by the anomaly detection method for data interaction requests provided in the above embodiments.

[0081] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0082] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: determining multiple marker points on the data interaction path of the target storage device, wherein the data interaction path is the data interaction path between the target storage device and the operating system, and the marker points represent the occurrence positions of multiple data interaction stages on the data interaction path; when the target storage device receives a real-time data interaction request, obtaining the time when multiple marker points are triggered and the stage identifiers of the multiple marker points, obtaining multiple timestamps and their corresponding stage identifiers; calculating the delay time of each of the multiple data interaction stages based on the multiple timestamps and their corresponding stage identifiers; determining the magnitude of the delay time of each of the multiple data interaction stages and the preset delay time threshold of each of the multiple data interaction stages; and identifying the data interaction stage with a delay time greater than the preset delay time threshold as an abnormal interaction stage.

[0083] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: calculating the delay time of each of the multiple data interaction stages based on multiple timestamps and the stage identifiers corresponding to each of the multiple timestamps, including: taking two timestamps with matching stage identifiers as a group of stage timestamps to obtain multiple groups of stage timestamps, wherein a group of stage timestamps includes a start timestamp and a completion timestamp, and the completion timestamp is later than the start timestamp; in each stage timestamp, calculating the difference between the completion timestamp and the start timestamp to obtain the delay time of each of the multiple data interaction stages.

[0084] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: further including storing multiple timestamps and stage identifiers corresponding to each timestamp in a preset data map, wherein the preset data map is a data structure for extending the Berkeley package filter program to store and share data.

[0085] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: This further includes: obtaining runtime information corresponding to the abnormal interaction stage, wherein the runtime information includes a completion timestamp, an identifier of the target storage device, an identifier of the real-time data interaction request, a process identifier of the real-time data interaction request, a stage type, the starting sector of the real-time data interaction request, and the data size corresponding to the real-time data interaction request; and transmitting the runtime information to the user terminal through a preset communication mechanism, wherein the preset communication mechanism includes a performance event buffer or a circular buffer.

[0086] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: further including: performing multi-dimensional feature analysis on the running information to obtain the operation frequency of the real-time data interaction request and the data processing volume threshold of the target storage device; determining the cause of the abnormal interaction stage based on the operation frequency, the data processing volume threshold and the delay time corresponding to the abnormal interaction stage, wherein the cause of the abnormal interaction stage includes request timeout, read-write ratio imbalance and operating system memory abnormality.

[0087] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: further including: constructing a dynamic Sankey diagram based on runtime information and the cause of the abnormal interaction phase, wherein the dynamic Sankey diagram includes the delay time of each of the multiple data interaction phases and the relative flow size of each of the multiple data interaction phases, wherein the relative flow size represents the weight of the data interaction phase in the entire data interaction process; and displaying the dynamic Sankey diagram on a preset display platform.

[0088] Embodiments of the present invention also provide a computer program product, including a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it can: determine multiple marker points on a data interaction path of a target storage device, wherein the data interaction path is a data interaction path between the target storage device and the operating system, and the marker points represent the respective occurrence positions of multiple data interaction stages on the data interaction path; when the target storage device receives a real-time data interaction request, obtain the time when multiple marker points are triggered and the stage identifiers of the multiple marker points, and obtain multiple timestamps and their respective stage identifiers; calculate the delay time of each of the multiple data interaction stages based on the multiple timestamps and their respective stage identifiers; determine the magnitude of the delay time of each of the multiple data interaction stages and the preset delay time threshold of each of the multiple data interaction stages; and determine the data interaction stages with delay times greater than the preset delay time thresholds as abnormal interaction stages.

[0089] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0090] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0092] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0093] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0094] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0095] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting anomalies in data interaction requests, characterized in that, include: Multiple marker points are identified on the data interaction path of the target storage device, wherein the data interaction path is the data interaction path between the target storage device and the operating system, and the marker points represent the locations where the various data interaction stages occur on the data interaction path. When the target storage device receives a real-time data interaction request, it obtains the time when the multiple marker points are triggered and the stage identifier of the multiple marker points, and obtains multiple timestamps and the stage identifiers corresponding to the multiple timestamps. Based on the multiple timestamps and the stage identifiers corresponding to each of the multiple timestamps, calculate the delay time of each of the multiple data interaction stages; Each of the multiple data interaction stages is determined to have its own delay time and its own preset delay time threshold. The data interaction phase in which the delay time is greater than the preset delay time threshold is determined as an abnormal interaction phase.

2. The method according to claim 1, characterized in that, The step of calculating the delay time of each of the multiple data interaction stages based on the multiple timestamps and the stage identifiers corresponding to each of the multiple timestamps includes: Two timestamps that match the stage identifier are taken as a group of stage timestamps to obtain multiple groups of stage timestamps. Each group of stage timestamps includes a start timestamp and a finish timestamp, and the finish timestamp is later than the start timestamp. In each of the stage timestamps, the difference between the completion timestamp and the start timestamp is calculated to obtain the delay time of each of the multiple data interaction stages.

3. The method according to claim 1, characterized in that, Also includes: The plurality of timestamps and their respective corresponding stage identifiers are stored in a preset data map, wherein the preset data map is a data structure used to extend the storage and sharing of data in the Berkeley package filter program.

4. The method according to claim 1, characterized in that, Also includes: Obtain the running information corresponding to the abnormal interaction stage, wherein the running information includes a completion timestamp, the identifier of the target storage device, the identifier of the real-time data interaction request, the process identifier of the real-time data interaction request, the stage type, the starting sector of the real-time data interaction request, and the data size corresponding to the real-time data interaction request; The operation information is transmitted to the user terminal through a preset communication mechanism, wherein the preset communication mechanism includes a performance event buffer or a circular buffer.

5. The method according to claim 4, characterized in that, Also includes: Multi-dimensional feature analysis is performed on the operational information to obtain the operation frequency of the real-time data interaction request and the data processing volume threshold of the target storage device; Based on the operation frequency, the data processing volume threshold, and the delay time corresponding to the abnormal interaction phase, the cause of the abnormal interaction phase is determined, wherein the cause of the abnormal interaction phase includes request timeout, read-write ratio imbalance, and operating system memory abnormality.

6. The method according to claim 5, characterized in that, Also includes: Based on the operational information and the reasons for the occurrence of the abnormal interaction phase, a dynamic Sankey diagram is constructed. The dynamic Sankey diagram includes the delay time of each of the multiple data interaction phases and the relative traffic size of each of the multiple data interaction phases. The relative traffic size represents the weight of the data interaction phase in the entire data interaction process. The dynamic Sankey diagram is displayed on a preset display platform.

7. An anomaly detection device for data interaction requests, characterized in that, include: The first determining module is used to determine multiple marker points on the data interaction path of the target storage device, wherein the data interaction path is the data interaction path between the target storage device and the operating system, and the marker points represent the occurrence positions of multiple data interaction stages on the data interaction path. The acquisition module is used to acquire, when the target storage device receives a real-time data interaction request, the time when the multiple marker points are triggered, the stage identifier of the multiple marker points being triggered, and to obtain multiple timestamps and the stage identifiers corresponding to the multiple timestamps. The calculation module is used to calculate the delay time of each of the multiple data interaction stages based on the multiple timestamps and the stage identifiers corresponding to each of the multiple timestamps. The judgment module is used to determine the magnitude of the delay time of each of the multiple data interaction stages and the preset delay time threshold of each of the multiple data interaction stages; The second determining module is used to determine the data interaction stage where the delay time is greater than the preset delay time threshold as an abnormal interaction stage.

8. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device where the non-volatile storage medium is located to execute the anomaly detection method for the data interaction request as described in any one of claims 1 to 6.

9. A computer device, characterized in that, include: Memory and processor The memory stores computer programs; The processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, the processor performs the anomaly detection method for the data interaction request as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the anomaly detection method for data interaction requests as described in any one of claims 1 to 6.

Images