Method for monitoring anomalies of data transmission and electronic device

Abstract

The application discloses an exception monitoring method for data transmission and an electronic device, relates to the technical field of data transmission, and comprises the following steps: periodically acquiring historical call tracking reports and historical resource occupation data of a plurality of servers in a data transmission process; determining a message flow and resource occupation correlation target graph between the servers based on the historical call tracking reports and the historical resource occupation data; constructing a correlation model according to the message flow and resource occupation correlation target graph; inputting current message flow and current resource occupation data corresponding to each server into the correlation model, and judging whether the current message flow and the current resource occupation are abnormal based on a preset relationship formula, a message flow threshold and a resource occupation threshold in the correlation model. The method of the application can quickly and accurately identify abnormal conditions during abnormality judgment.

Description

Technical Field

[0001] This application relates to the field of data transmission technology, and in particular to a method and electronic device for monitoring abnormal data transmission. Background Technology

[0002] With the widespread adoption of distributed systems and microservice architectures in enterprise applications, data transmission is no longer confined to a single service but involves cross-process and cross-node interactions between multiple services. In such multi-service collaborative scenarios, the stability of data transmission directly determines business continuity. Once transmission anomalies occur, serious problems such as misjudgment of device status may arise. Therefore, real-time monitoring of the data transmission process has become a core requirement for ensuring the reliable operation of the system.

[0003] However, existing data transmission monitoring solutions have significant limitations: some solutions only collect server-side resource usage data, such as CPU and memory usage, or only record data transmission call logs, making it impossible to accurately determine the root cause of data transmission anomalies. They cannot adapt to dynamic changes brought about by service architecture iterations and business traffic fluctuations, and are prone to misjudgments or omissions, failing to meet the accurate and adaptive monitoring needs of complex multi-service environments. Summary of the Invention

[0004] This application provides a method and electronic device for monitoring abnormal data transmission, which at least solves the problem in related technologies that only record the number of calls and cannot achieve accurate monitoring of data transmission.

[0005] Firstly, this application provides a method for monitoring abnormal data transmission, including:

[0006] Periodically retrieve historical call tracking reports and historical resource usage data from multiple servers during data transmission;

[0007] Based on historical call tracing reports and historical resource usage data, a target graph is determined to show the correlation between message traffic and resource usage among various servers.

[0008] A correlation model is constructed based on the target graph of the relationship between message traffic and resource consumption. The correlation model includes the preset relationship formula, message traffic threshold and resource consumption threshold for each server. The preset relationship formula is the relationship formula between message traffic and resource consumption for each server.

[0009] Periodically retrieve the current message traffic and current resource usage data for each server.

[0010] Input the current message traffic and current resource usage data corresponding to each server into the association model, and determine whether the current message traffic and current resource usage are abnormal based on the preset relationship formula, message traffic threshold and resource usage threshold in the association model;

[0011] If the current message traffic exceeds the message traffic threshold and the current resource usage data does not satisfy the relationship formula in the association model, then the current message traffic is judged to be abnormal.

[0012] If the current resource usage exceeds the resource usage threshold and the current message traffic does not meet the relationship formula in the association model, then the current resource usage is judged to be abnormal.

[0013] Secondly, this application also provides an electronic device, comprising:

[0014] Memory, used to store computer programs;

[0015] A processor is used to implement the steps of any of the methods in the first aspect when executing a computer program.

[0016] This application provides a method and electronic device for monitoring data transmission anomalies. The method periodically acquires historical call trace reports and historical resource usage data to provide effective data support for anomaly detection. It determines a target graph of the correlation between message traffic and resource usage among various servers, graphically linking message traffic and resource usage to improve the accuracy of anomaly detection through multi-dimensional data correlation. The correlation model constructed based on the target graph of message traffic and resource usage correlation can automatically determine whether anomalies exist on each server during data transmission monitoring. This model learns from normal patterns and correlations in historical data, enabling real-time comparison of the current server's operating status with normal patterns. This significantly improves the efficiency and accuracy of anomaly monitoring and has a certain degree of adaptability, updating as data changes. The preset relationship formulas in the correlation model clearly define the intrinsic connection between message traffic and resource usage on each server. By inputting periodically acquired current message traffic and resource usage data into the model, the mutual influence of these two key indicators can be comprehensively considered, rather than judging whether each indicator exceeds limits in isolation. Message traffic thresholds and resource consumption thresholds are not fixed but can be dynamically adjusted based on the actual system operation and business needs. The anomaly detection mechanism of the correlation model ensures that the system operates under normal conditions, providing reliable protection for data storage, transmission, and processing. Attached Figure Description

[0017] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1This is a schematic diagram illustrating an application scenario corresponding to a data transmission anomaly monitoring method provided in one embodiment of this application.

[0019] Figure 2 A flowchart illustrating an abnormal data transmission monitoring method provided in an embodiment of this application;

[0020] Figure 3 A flowchart illustrating a data transmission anomaly monitoring method provided in another embodiment of this application;

[0021] Figure 4 A schematic diagram of the structure of a data transmission anomaly monitoring device provided in an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, other embodiments obtained by those of ordinary skill in the art without creative effort are all within the protection scope of this application.

[0024] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0025] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] With the deep implementation of distributed systems and microservices architectures in enterprise-level scenarios, data interaction has broken through the boundaries of a single service and transformed into high-frequency collaboration across processes and nodes among multiple service endpoints. In such scenarios of multi-service linkage, the stability of data transmission directly affects the normal operation of the business: if abnormalities occur during the transmission process (such as data latency, packet loss), it will cause the business process to freeze at least, and at worst, lead to serious failures such as misjudgment of device status and reconciliation deviation. Therefore, real-time monitoring of the entire data transmission process has become a key link to ensure the reliable operation of the entire system. However, the current mainstream data transmission monitoring solutions still have significant technical shortcomings: on the one hand, many solutions only stay at the level of single-dimensional data collection. Either they only collect hardware resource data of the service endpoint, such as CPU occupancy rate, memory usage rate, etc.; or they only record the call logs of data transmission, such as the number of requests and response duration. As a result, when an abnormality occurs later, it is impossible to accurately locate the specific source that caused the transmission abnormality. It cannot meet the requirements for monitoring accuracy and adaptability in complex multi-service environments.

[0027] Therefore, when facing the technical problems of the above related technologies, first, regarding the problem that it is difficult to locate the root cause of abnormalities, considering that the essence of data transmission abnormalities is the correlation result of message traffic changes and resource occupancy fluctuations, by periodically obtaining historical call trace reports and historical resource occupancy data, it provides a complete data basis for subsequent root cause localization of abnormalities. The corresponding rules between message traffic changes and resource occupancy changes among different service endpoints are intuitively presented through a graph, providing a quantitative basis for subsequent abnormality determination. Finally, to improve the efficiency and accuracy of abnormality judgment, let the model learn the correlation relationships in the target graph of the correlation between message traffic and resource occupancy, forming a dynamic judgment standard; during real-time monitoring, the model can compare the current message traffic and resource occupancy relationship of the service endpoint with the historical normal rules, thereby improving the efficiency and accuracy of abnormality judgment. By learning, a preset relationship formula will be formed in the correlation model. By inputting the currently obtained message traffic and resource occupancy data into the model at regular intervals, the mutual influence of the two key indicators can be comprehensively considered, rather than judging whether each indicator exceeds the limit in isolation. Abnormality judgment is based on message traffic thresholds and resource occupancy thresholds. This abnormality judgment mechanism can ensure the system runs in a normal state, providing reliable guarantees for data storage, transmission, and processing.

[0028] Figure 1 This is a schematic diagram of an application scenario corresponding to an abnormal monitoring method for data transmission provided by an embodiment of the present application. As Figure 1As shown, the application scenario includes: client device 101, data transmission anomaly monitoring device 102, first server device 103 and second server device 104. Client device 101, first server device 103 and second server device 104 are connected through data transmission anomaly monitoring device 102; there is a communication connection between first server device 103 and second server device 104.

[0029] Optionally, the data transmission anomaly monitoring device 102 can be integrated independently, such as on a server or electronic device, or it can be integrated with a server-side device or a client-side device. This embodiment does not limit this.

[0030] Understandably, a user can invoke the corresponding server for data transmission via client device 101. When a user initiates a call request on client device 101, data transmission anomaly monitoring device 102 first receives the request and determines which specific server to be invoked. For example, if the first server device 103 is invoked, data transmission anomaly monitoring device 102 sends the call request to the first server device 103. While processing the call request, the first server device 103 may also need to invoke other servers, such as the second server device 104. During data transmission between the client and server, and between servers, the data transmission anomaly monitoring device 102 records and stores historical data for each data transmission, on an individual server basis.

[0031] Specifically, the data transmission anomaly monitoring device 102 periodically acquires historical call tracking reports and historical resource usage data from multiple servers during data transmission. Based on these reports, it determines a target graph of message traffic and resource usage relationships between servers and constructs a correlation model. The device periodically acquires current message traffic and resource usage data for each server, inputs these data into the correlation model, and determines whether the current message traffic and resource usage are abnormal based on preset relationship formulas, message traffic thresholds, and resource usage thresholds. If the current message traffic exceeds the message traffic threshold and the current resource usage data does not satisfy the relationship formula in the correlation model, the current message traffic is considered abnormal; if the current resource usage exceeds the resource usage threshold and the current message traffic does not satisfy the relationship formula in the correlation model, the current resource usage is considered abnormal. If an anomaly is detected, an anomaly alert is promptly sent via the client device 101.

[0032] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0033] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0034] Figure 2 This is a flowchart illustrating a data transmission anomaly monitoring method according to an embodiment of this application, as shown below. Figure 2 As shown, the execution entity in this embodiment is a data transmission anomaly monitoring device. This data transmission anomaly monitoring device can be implemented through a computer program, or through a medium storing the relevant computer program, such as a USB flash drive and / or optical disc; alternatively, it can be implemented through a physical device that integrates or installs the relevant computer program, such as a chip or electronic device. The electronic device may be a computer or a server, etc. The data transmission anomaly monitoring method provided in this embodiment includes the following steps:

[0035] S201. Periodically obtain historical call tracking reports and historical resource usage data of multiple servers during data transmission.

[0036] Among them, the historical call tracing report refers to a collection of historical logs that record the entire data transmission link between multiple servers. It may include the following information: call link, the complete path of data from the source server to the target server; message characteristics, the type, size, sending or receiving timestamp, call frequency, and response duration of each transmitted message; and status identifiers, the status records during the transmission process.

[0037] Historical resource usage data refers to the historical consumption records of hardware and system resources by multiple servers during data transmission. This may include: hardware resources such as CPU utilization, memory usage, disk I / O read / write speed, and network bandwidth usage; system resources such as the number of processes, threads, and open file descriptors; and a time dimension such as the timestamp corresponding to the call tracing report, ensuring that resource data can be accurately correlated with transmission behavior.

[0038] Optionally, the abnormal data transmission monitoring device can intercept communication interfaces between servers, such as API calls and message queue consumption, and record the call chain, message characteristics, and status identifiers in real time, generating a single call trace report in a fixed format.

[0039] Optionally, the abnormal data transmission monitoring device can collect server-side hardware and system resource data and generate resource usage data by calling system interfaces, such as the / proc file system in Linux.

[0040] Understandably, the abnormal data transmission monitoring device stores the recorded data, such as by associating it with the server name.

[0041] Optionally, the cycle frequency can be dynamically adjusted according to the business scenario, such as collecting data once every 10 minutes in high-frequency transmission scenarios and once every 30 minutes in low-frequency scenarios. Data acquisition operations can be triggered by scheduled tasks.

[0042] Specifically, when a specified periodic time point is reached, call trace reports and resource usage data from multiple servers within a specified period, such as the past hour, are retrieved. The call trace reports and resource usage data are then associated and bound according to the time identifier and server ID to obtain historical call trace reports and historical resource usage data from multiple servers during data transmission.

[0043] For example, if server A sends a message to server B, then the calling characteristics of A, such as the calling frequency, are bound to the resource usage of A and the resource usage of B.

[0044] S202. Based on historical call tracing reports and historical resource usage data, determine the target graph of message traffic and resource usage correlation between each server.

[0045] The message traffic and resource consumption correlation graph is a visual, structured chart used to intuitively present the inherent correlation between message transmission characteristics and resource consumption among multiple servers. The graph contains two core elements: nodes, representing various servers such as device servers and cloud servers, with node attributes including service ID and service type; and edges, connecting two server nodes with data transmission, with edge attributes including message traffic characteristics and resource consumption correlation; and correlation strength, indicated by the thickness of the edges to show the closeness of the relationship.

[0046] Optionally, the target graph of the relationship between message traffic and resource consumption can be displayed to the user separately.

[0047] Specifically, core features for constructing relationships are extracted from historical call tracing reports and historical resource usage data. For example, message traffic characteristics are extracted from historical call tracing reports to calculate the average message volume and call frequency within historical periods; and the baseline resource usage values ​​for each server when there is no data transmission are extracted from historical resource usage data. The correlation between changes in message volume and resource increments is statistically analyzed. Circular icons represent servers, with icon size weighted according to the total message traffic of the server; the larger the total message volume, the larger the icon. Nodes are labeled with service ID, service type, etc. Directed lines connect each server, with edge thickness determined by the strength of the association; for example, a strong association uses a 3-line width, and a weak association uses a 1-line width. This determines the target graph of message traffic and resource usage relationships between servers. By constructing this graph, abstract message transmission and resource usage data are transformed into intuitive relationships, clearly presenting key patterns such as who transmits messages to whom and how transmission volume affects resources, providing a visual and quantifiable basis for subsequently constructing anomaly detection models.

[0048] S203. Construct a correlation model based on the target graph of the relationship between message traffic and resource consumption. The correlation model includes the preset relationship formula, message traffic threshold and resource consumption threshold for each server. The preset relationship formula is the relationship formula between message traffic and resource consumption for each server.

[0049] Among them, the correlation model is an algorithm model built on the target graph of the correlation between message traffic and resource consumption. It can learn the normal correlation between message transmission characteristics and resource consumption changes between servers, and automatically determine whether there are any abnormalities in data transmission by comparing the current data with the normal patterns in real-time monitoring.

[0050] Among them, the preset relationship formula refers to the mathematical expression that describes the normal correspondence between message traffic and resource consumption for each server. For example, CPU utilization = 0.05 × message volume + 20%, which reflects the reasonable proportion of how many resources should be consumed to process how many messages. Due to differences in functions and performance, the formula parameters are different for different servers.

[0051] Among them, the message traffic threshold refers to the maximum number of messages that can be processed per unit of time for each server, such as 1,000 messages per minute. Exceeding this value means that the message volume may exceed the server's carrying capacity, and abnormal traffic should be noted.

[0052] Among them, the resource consumption threshold refers to the safe upper limit set for resource consumption for each server, such as CPU utilization ≤80% and memory usage ≤90%. Exceeding this value means that resource consumption may affect the normal operation of the server, and resource anomalies should be alerted.

[0053] Specifically, the target graph of the relationship between message traffic and resource consumption is input into the initial model, and the model is trained to learn the characteristics of normal data transmission and abnormal data transmission. This allows the constructed correlation model to determine whether there are any abnormal data transmission situations based on the message traffic and resource consumption generated during real-time data transmission.

[0054] Alternatively, an initial association model can be trained in advance using a large number of training samples. When relevant data is collected periodically and the target map is obtained, the initial association model can automatically adjust the anomaly judgment criteria according to the association relationships in the target map.

[0055] S204. Periodically obtain the current message traffic and current resource usage data for each server.

[0056] Specifically, the data collection frequency is set according to the business characteristics of the server. For example, high-frequency services are collected every 10 seconds, and low-frequency services are collected every minute. This ensures that anomalies can be captured in a timely manner without increasing the system load due to excessive collection. The data collected includes current message traffic, the total number of messages received and processed by the server per unit time; and current resource usage data, including the server's hardware and system resource consumption, such as CPU utilization, memory usage, disk I / O rate, and network bandwidth usage.

[0057] S205. Input the current message traffic and current resource usage data corresponding to each server into the association model, and determine whether the current message traffic and current resource usage are abnormal based on the preset relationship formula, message traffic threshold and resource usage threshold in the association model.

[0058] Specifically, the collected current message traffic is substituted into a preset formula to calculate the theoretical resource consumption, i.e., the amount of resources that should be consumed normally under the current message volume. Conversely, the current resource consumption is substituted into the formula to calculate the theoretical message traffic, i.e., the amount of messages that should be processed normally under the current resource consumption. The actual value is compared with the theoretical value and the threshold to determine the type of anomaly.

[0059] S206. If the current message traffic is greater than the message traffic threshold and the current resource usage data does not satisfy the relationship formula in the association model, then the current message traffic is judged to be abnormal.

[0060] Specifically, if the current message traffic is greater than the message traffic threshold, and the current resource usage data is not equal to the theoretical resource usage, then the current message traffic is determined to be abnormal, that is, the message volume exceeds the standard but the resource consumption does not match, and there may be invalid requests or duplicate requests.

[0061] S207. If the current resource usage is greater than the resource usage threshold and the current message traffic does not meet the relationship formula in the association model, then the current resource usage is judged to be abnormal.

[0062] Specifically, if the current resource usage is greater than the resource usage threshold and the current message flow is not equal to the theoretical message flow, then the current resource usage is determined to be abnormal, that is, the resource consumption exceeds the standard but the message volume does not match, which may indicate a resource leak or server failure.

[0063] Alternatively, the association model can also be a graph neural network, which is trained using a target graph of message traffic and resource consumption relationships between multiple historical servers, including the numerical range of node attributes and the numerical range of edge attributes.

[0064] Graph neural networks are artificial intelligence models specifically designed for processing graph-structured data. They can capture complex patterns of data relationships by learning the relationships between nodes and edges in the graph. In this solution, they are used to learn message interactions and resource dependencies between servers, thereby determining whether the current system state is abnormal.

[0065] Among them, the historical relationship target graph refers to a graph structure built based on the operational data of each server over a period of time, which records the normal message traffic and resource consumption relationship patterns between servers.

[0066] Among them, the current relationship target graph refers to the graph structure built based on real-time collected server data, which reflects the message interaction and resource dependency status between each server at the current moment, and is used to compare with the normal pattern learned by the model.

[0067] Among them, node attributes refer to the characteristic data of each node (representing a server) in the graph, such as the number of messages received or sent per unit time, message processing frequency, etc. The numerical range of node attributes is the normal fluctuation range learned by the graph neural network.

[0068] Among them, edge attributes refer to the feature data of the edge connecting two nodes in the graph, representing the interaction relationship between the servers. They mainly refer to the resource dependency indicators between the servers, such as the rate of change of B's ​​resource usage after A sends a message to B, the consumption of network resources by message transmission, etc. The value range of edge attributes is the normal association range learned by the graph neural network.

[0069] Specifically, the data collection frequency is set according to the business characteristics of the server, such as collecting data every 10 seconds for high-frequency services and every minute for low-frequency services. Data collected includes current message traffic, the number of messages received and sent by each server, and the distribution of message types. Current resource usage data is also collected, including CPU utilization, memory usage, disk I / O, and network bandwidth consumption for each server.

[0070] Furthermore, each server is treated as a node in the graph, with node attributes set to its message traffic characteristics, such as receiving 30 messages / minute and processing 5 messages / second. If two servers interact, an edge is created between the corresponding nodes, with edge attributes set to their resource association characteristics, such as B's CPU utilization increasing by an average of 8% after A sends a message to B. The constructed graph structure is converted into a format recognizable by the graph neural network, and a timestamp is added to mark the generation time of the current graph. The association model has been trained on a large number of historical association graphs, learning the range of node attribute values ​​under normal conditions, such as the normal message volume of a server being 10-50 messages / minute and the range of edge attribute values, such as the CPU change rate during server interactions typically being 5%-15%. These normal range thresholds are loaded when the model is loaded, serving as a benchmark for anomaly detection. The current target graph is input into the graph neural network, and the model automatically extracts the graph's global features, such as the overall message traffic distribution and the density of resource dependency networks, and local features, such as the attribute values ​​of individual nodes and the association strength of individual edges. The model identifies nodes or edges that deviate from the normal pattern by comparing the current features with the normal features learned during training.

[0071] Furthermore, if the model detects that the attribute value of a node exceeds the range of node attribute values, and this deviation cannot be explained by the normal changes of other nodes, then the current message traffic is determined to be abnormal. If the model detects that the attribute value of an edge exceeds the range of edge attribute values, and this deviation does not match the changes in message traffic, then the current resource usage is determined to be abnormal.

[0072] The data transmission anomaly monitoring method provided in this application provides effective data support for anomaly judgment by periodically acquiring historical call trace reports and historical resource usage data. It determines a target graph of the correlation between message traffic and resource usage among various servers, graphically linking message traffic and resource usage, and improving the accuracy of anomaly judgment through multi-dimensional data correlation. The correlation model constructed based on the target graph of message traffic and resource usage correlation can automatically determine whether there are anomalies on each server during data transmission monitoring. This model, by learning normal patterns and correlations in historical data, can compare the current server's operating status with the normal pattern in real time. This greatly improves the efficiency and accuracy of anomaly monitoring and has a certain degree of adaptability, updating as data changes. The preset relationship formula in the correlation model clarifies the inherent connection between message traffic and resource usage on each server. By inputting the periodically acquired current message traffic and resource usage data into the model, the mutual influence of the two key indicators can be comprehensively considered, rather than judging whether each indicator exceeds limits in isolation. The message traffic threshold and resource usage threshold are not fixed but can be dynamically adjusted according to the actual operation of the system and business needs. The anomaly detection mechanism of the association model can ensure that the system operates under normal conditions, providing reliable protection for data storage, transmission and processing.

[0073] As an optional implementation, based on any of the above embodiments, a target graph of message traffic and resource usage correlations between servers is determined based on historical call tracing reports and historical resource usage data, including:

[0074] Obtain the call relationship for each server from the historical call tracing report. The call relationship refers to the hierarchical relationship between the client calling the server and the server calling other servers.

[0075] Based on the calling relationships of each server, an initial graph of message traffic and resource consumption relationships between servers is generated, with the server as the node and the calling relationship as the edge.

[0076] Historical resource usage data is used to optimize the weights of the edges in the initial graph in order to determine the target graph of message traffic and resource usage relationships between each server.

[0077] The "calling relationship" refers to the hierarchical interaction logic in multi-server data transmission, which includes two situations: direct calls from the client to the server and indirect calls from the server to other servers. Essentially, it is the flow relationship of data from the initiator to the receiver, which determines the transmission path of message traffic.

[0078] Among them, the initial graph of the relationship between message traffic and resource consumption is a basic visualization chart built based on the call relationship. It only contains nodes and edges and does not reflect quantitative features such as message traffic size and resource consumption changes.

[0079] Weight optimization refers to using historical resource usage data to add quantitative attributes, such as association strength and resource consumption coefficient, to the edges in the initial graph, so that the edges can reflect the degree of resource impact behind the invoked relationship.

[0080] Specifically, the caller ID, call receiver ID, and call timestamp are obtained from historical call tracing reports to identify the call relationships. Nodes are named using the server ID and arranged hierarchically to avoid overlapping connections. Directed line segments connect the caller and receiver; initially, these segments have a uniform style and do not yet reflect quantitative characteristics, forming the initial graph. From historical resource usage data, resource consumption data corresponding to the call relationships is filtered and grouped from caller to receiver. Resource characteristics for each group, such as the ratio of message traffic change to resource increment, are calculated. Based on predefined weighting rules, if a resource correlation coefficient ≥ 0.05% / line indicates a strong correlation, the line width is increased to 3, and each edge is optimized sequentially to obtain the target graph. The final target graph not only shows who calls whom but also provides a visual understanding of the message volume and resource impact through line segment thickness and label data, offering a visualized quantitative basis for subsequent association model construction.

[0081] The data transmission anomaly monitoring method provided in this application obtains the call relationships corresponding to each server from historical call tracing reports and constructs an initial graph based on this. This graph can intuitively display the hierarchical structure of client calls to servers and mutual calls between servers. By using historical resource usage data to optimize the weights of the edges in the initial graph, the actual correlation between message traffic and resource usage between each server can be more accurately reflected.

[0082] As an optional implementation, based on any of the above embodiments, it further includes:

[0083] Obtain service information corresponding to multiple servers through a desktop distributed bus.

[0084] Based on the service information, the corresponding service interfaces are registered in the desktop distributed bus, and the service interfaces are bound to the corresponding service call paths, so that the client can obtain the service information of each server through the interface self-test tool of the desktop distributed bus.

[0085] In response to receiving a client's command to call a synchronous method of a desktop distributed bus tool, the system sends a call request, which includes service information of the server to be called. Based on the service information of the server to be called, the system determines the service interface and the corresponding service call path of the server to be called.

[0086] The service interface and corresponding service call path of the server to be called are used to invoke the server.

[0087] Among them, the desktop distributed bus, or D-Bus, is the communication hub connecting multiple processes (server and client) in systems such as Linux. It can uniformly forward call requests and transmit service information, allowing different servers and clients to interact without direct connection.

[0088] Service information refers to the identity and capability descriptions exposed by the server, including the service name, the list of supported interfaces, and the interface parameter requirements.

[0089] Among them, the service interface refers to the functional entry point provided by the server. Each interface corresponds to a specific function, and the client needs to call the server function through the interface.

[0090] The service call path refers to the actual processing logic address corresponding to the interface in the server. After binding the interface with the call path, the specific processing code can be located directly when the interface is called, avoiding the situation where the interface is found but the execution logic cannot be found.

[0091] Among them, the interface self-inspect tool, namely the introspect tool, allows the client to obtain the service information registered by the server.

[0092] Among them, the synchronous method call command of the bus tool is busctl -- smart call, which allows the client to call any registered server.

[0093] Specifically, when a server starts up, it generates its own service information, including the service name, interface list, and parameter requirements. This information can be proactively sent to the D-Bus bus via the D-Bus service discovery protocol. The D-Bus bus receives and stores this information: it creates service information files for each server and stores them categorized by service name. The D-Bus bus provides an open service information query interface, allowing servers to update their service information at any time. Subsequent interface registration steps will retrieve the latest service information from this entry point.

[0094] Furthermore, after obtaining the service information from the server, it is parsed hierarchically by service name, interface, and parameters. Each interface is registered in the D-Bus interface registry. After successful registration, D-Bus returns an interface registration ID, which the server can use to update or delete the interface later. Then, the call path corresponding to the interface is obtained from the server, and the service interface and the corresponding service call path are bound in the mapping table.

[0095] Furthermore, upon receiving a call request initiated by a client via busctl--smart call, the request is parsed, and the interface registry is queried using the service name + interface name to confirm that the interface has been registered and is in a normal state.

[0096] Next, the interface-call path mapping table is queried to find the service call path corresponding to the interface, thus determining the interface to be called. Finally, based on the service call path, the process of the server to be called is located, and the call request is forwarded to the server to complete the call.

[0097] The data transmission anomaly monitoring method provided in this application obtains service information corresponding to multiple servers through a desktop distributed bus and registers the corresponding service interfaces in the bus, realizing centralized management of service information, reducing the coupling between the client and the server, and improving system flexibility. The client obtains service information corresponding to each server through the interface self-testing tool of the desktop distributed bus, enabling it to quickly and accurately understand the available service interfaces. The interface self-testing tool can detect the availability and status of service interfaces in real time. When a service interface fails or becomes unavailable, the client can promptly learn of this and select other available interfaces, thereby avoiding call failures and performance degradation caused by calling unavailable interfaces, and improving the success rate and efficiency of service calls. Responding to the synchronous method call command sent by the client, the service interface and call path are determined based on the service information corresponding to the server to be called, and this information is used to perform the service call, realizing an efficient synchronous call process and improving the call response speed.

[0098] As an optional implementation, based on any of the above embodiments, it further includes:

[0099] In response to receiving a call request from a client, the system determines whether the client has data access permissions based on preset permission rules and the call request.

[0100] If it is determined that the client has data access permissions, a tracking identifier is generated based on the service information of the server to be called, and the tracking identifier and the call request are sent to the corresponding server.

[0101] If it is determined that the client does not have the permission to access data, a permission failure message is sent to the client.

[0102] Among them, the preset permission rules refer to the pre-configured permission judgment criteria, which are used to determine whether the client has the right to call the server or obtain data.

[0103] For example, preset permission rules can be: caller permission rules, such as only the root user can call core services and third-party clients can only call public interfaces; service-interface binding rules, such as the sensitive file query interface of the file storage service is only open to the system management client; and data scope permission rules, such as client A can only obtain business data of its own department and cannot obtain data of other departments.

[0104] Among them, data access permission refers to whether a client is qualified to obtain the corresponding data from the target server when calling the server, and it is a specific manifestation of the preset permission rules.

[0105] The tracking identifier is a string that uniquely identifies a single call request. It may include timestamps, client identifiers, random codes, etc. It is used to connect the entire chain of client-initiated request, forwarding, server-side processing, and result return, making it convenient to trace the call process and troubleshoot anomalies.

[0106] Specifically, upon receiving a request, the system first verifies the request format. If the format is valid, it proceeds to the parsing stage, extracting the client identifier, the name of the service to be called, the interface name, and the range of requested data. Preset permission rules are then loaded, and a match is performed based on the parsed information and these rules. For example, the service name and interface name are matched to confirm the existence of permission configurations for that service and interface within the rules. The client identifier is also matched to determine if the current client is in the list of allowed client identifiers, performing single-dimensional or multi-dimensional matching. If all matches pass, the system determines that the client has data access permissions, generates a unique tracking identifier, embeds this identifier into the original request, and adds a tracking identifier field. Based on the service name in the service information, the system locates the corresponding server and forwards the request with the tracking identifier to the server. If a match fails, the system determines that the client lacks data access permissions. Based on the reason for the failure during permission determination, a structured message is generated, including the failure type, specific reason, and suggested actions, and sent to the client.

[0107] The data transmission anomaly monitoring method provided in this application, based on preset permission rules, performs permission checks on client call requests, ensuring that only clients with the corresponding data acquisition permissions can access specific server data. It can identify and reject unreasonable requests, thereby protecting system stability and data integrity. Once it is determined that the client has data acquisition permissions, a tracking identifier is generated based on the service information corresponding to the server to be called, and this identifier is sent to the server along with the call request. The tracking identifier provides a unique identifier for each call request, facilitating tracing and debugging when system problems occur. Through the tracking identifier, developers can quickly locate the source of the call request, the path it takes, and errors that occur during processing, improving problem-solving efficiency. By judging permissions, the system can reject unauthorized client requests at an early stage, avoiding the server's processing of invalid requests and reducing server resource waste. Simultaneously, it also reduces the waiting time for clients waiting for responses to invalid requests, improving the overall system response speed and providing a better user experience.

[0108] As an optional implementation, based on any of the above embodiments, it further includes:

[0109] Receive the call result sent by the server.

[0110] If the call is successful, a call trace report is generated based on the trace identifier.

[0111] If the call fails, a call failure analysis report will be generated based on the tracing identifier.

[0112] The call result refers to the final status feedback returned by the server after processing the client's call request, such as success or failure. A success result may include the processing status (success), business data, processing time, etc.; a failure result may include the processing status (failure), failure code, failure details, etc.

[0113] The call tracing report refers to the call archive generated when a call is successful, based on the tracing identifier and integrating information from the entire chain. It is used to record who initiated the call, what service was called, how it was processed, and what the result was.

[0114] Among them, the call failure analysis report refers to the report generated when a call fails, based on the tracking identifier, integrating the cause of failure, link nodes, and investigation direction. It may include the location of the failure link, possible root causes, and solution suggestions.

[0115] Specifically, after processing the client's call request, the server constructs the call result according to the D-Bus standard format, including a trace identifier. If the call result is successful, information is extracted from the trace identifier, such as client request information: client identifier, request timestamp, request parameters, and information about the service to be called; bus forwarding information: bus request reception time, permission verification result, and server forwarding time; and server processing information: server identifier, processing time, business data, and data generation time. This information is then integrated into a call tracing report in a fixed format, with the report name matching the trace identifier. If the call result is a failure, information is extracted from the trace identifier, such as failure stage data: failure stage, failure code, and failure details; and environmental data: system resource status and network status at the time of the call. This information is also integrated into a call failure analysis report in a fixed format, with the report name matching the trace identifier.

[0116] Optionally, the call records corresponding to each service interface can be obtained periodically; based on the call records and service information corresponding to each service interface, an interface document can be generated and sent to a preset operation and maintenance terminal.

[0117] Among them, the call record corresponding to the service interface refers to the structured log that records the full call behavior of the service interface, covering core information such as the time of the interface call, the initiator, the request parameters, the processing result, and the time consumed. It is the core data that reflects the actual usage of the interface and supports the dynamic content of the document.

[0118] The interface documentation, or API documentation, includes not only the basic functional definitions of the interface, such as the interface name, parameter requirements, and return rules, but also dynamic call characteristics, such as call frequency, success rate, and common parameter values, so that operations and maintenance personnel can have a comprehensive understanding of the interface.

[0119] Among them, the preset operation and maintenance terminal refers to the operation and maintenance side receiving carrier that is configured in advance to receive interface documents, such as the desktop workstation of operation and maintenance personnel, operation and maintenance management platform, etc.

[0120] Specifically, the acquisition cycle is dynamically adjusted based on the interface call frequency; for example, high-frequency interfaces are acquired every 12 hours, while low-frequency interfaces are acquired daily. After acquiring the call records for each service interface, static attributes of each interface are extracted, such as service name, interface name, interface function description, interface creation time, and last update time. Input and output rules, including input parameters and return content, and permission requirements, including the scope of allowed clients and data access restrictions, are also defined. The static interface definition and interface call feature summary are merged according to a preset template to generate a structured interface document. Terminal information is read from pre-configured operation and maintenance terminal information; configuration content may include terminal type, receiving address, push method, and document format requirements. Differentiated push notifications are performed for different terminal types.

[0121] Optionally, after the push is completed, the push results of each terminal are recorded in the document push log for easy follow-up; if the push fails, such as when the terminal is offline or the platform interface times out, it is retried according to the rules.

[0122] Periodically retrieving call logs allows for real-time and accurate understanding of the usage frequency and call time distribution of each service interface. Interface documentation generated based on these call logs and service information details the interface's input / output parameters, calling methods, performance metrics, and other information. Operations personnel can use this documentation to perform targeted system optimizations, such as optimizing interface code logic to improve processing efficiency or adjusting interface concurrency limits to accommodate higher call volumes.

[0123] The data transmission anomaly monitoring method provided in this application allows the system to understand the execution status of each service call in real time by receiving call results sent by the server. Whether successful or not, these results are included in the monitoring scope, enabling operations personnel and managers to have a comprehensive grasp of the system's operational status. Call tracing reports and call failure analysis reports generated based on tracing identifiers can visually display the entire service call process. The tracing identifier acts like a thread, connecting the various stages of the call process and presenting them intuitively to relevant personnel through charts, flowcharts, and other forms. This helps to quickly understand the call path and discover potential bottlenecks and problems. Especially in complex distributed systems, visualization can greatly improve the efficiency of problem localization.

[0124] As an optional implementation, based on any of the above embodiments, it further includes:

[0125] The cause of the call failure is determined based on the call failure analysis report.

[0126] If the failure is due to the first type of problem, the call request, including the tracking identifier, will be resent after a preset time. The first type of problem includes network fluctuations and temporary server overload.

[0127] If the failure is due to the second type of problem, a failure analysis report and a tracer will be sent to the corresponding client. The second type of problem includes server failure and non-existent service interface.

[0128] The first type of problem is a temporary, self-healing type of failure during call failure. Its core characteristic is that the failure is not continuous and there is a probability of recovery after retry, such as network fluctuations or temporary server busyness.

[0129] The second type of problem is a persistent failure that requires manual intervention. Its core characteristic is that the failure cannot be self-healed by retries and the root cause must be addressed to recover, such as server failure or non-existent service interface.

[0130] The preset duration is a pre-configured retry waiting time for retries in the first type of problem. It can be dynamically adjusted according to the fault type. For example, the retry interval for network fluctuations is set to 5 seconds, and the retry interval for server busy is set to 30 seconds to avoid frequent retries in a short period of time, which would increase the system load.

[0131] Specifically, failure-related information, such as failure details and root cause analysis, is extracted from the call failure analysis report. The failure cause type is determined based on the extracted information, such as classifying the failure cause type by field matching and feature verification according to preset rules.

[0132] Furthermore, if the failure is determined to be a Type I problem, a corresponding preset duration is determined based on the problem type. Based on the parameters of the original call request, the tracer is retained, and a retry flag field is added, such as the retry reason and the original failure time, to construct a retry call request. The retry request and preset duration are submitted to the scheduled retry task queue, triggering retries periodically. If the failure is a Type II problem, the call failure analysis report and tracer are packaged and sent to the corresponding client.

[0133] Optionally, if the call succeeds after a retry, a call tracing report is generated according to the normal process, and a retry record is added to the report. The success result is then returned to the client synchronously. If the call still fails after a retry, the number of retries is accumulated. If the maximum number of retries has not been reached, the preset duration is updated according to the retry interval increment rule, and the call is submitted to the scheduled retry task queue again. If the maximum limit has been reached, the retry failure record is appended to the original call failure analysis report and classified as a second type of problem.

[0134] The data transmission anomaly monitoring method provided in this application, when the cause of a call failure is determined to be a first-type problem, resends the call request, including a tracer identifier, after a preset time interval. This avoids permanent call failure due to momentary network failures, increases the probability of successful data acquisition, and helps maintain network stability and reliability. When the cause of failure is a second-type problem, such as server failure or a non-existent service interface, a call failure analysis report and a tracer identifier are sent to the corresponding client. This provides users with clear fault information, allowing them to determine whether the problem is a system issue or a problem with their own operation, and thus take appropriate measures, such as waiting for system repair or checking whether the operation is correct, thereby increasing user trust in the system.

[0135] As an optional implementation, based on any of the above embodiments, it further includes:

[0136] Obtain resource usage data corresponding to multiple servers that the client has pre-subscribed to on the desktop distributed bus.

[0137] The asynchronous signal monitoring command of the desktop distributed bus tool is used to monitor whether the resource usage data of each server changes.

[0138] If the resource usage data changes, the changed resource usage data is sent to the client and stored as historical resource usage data.

[0139] Among them, the asynchronous signal monitoring command is bustcl–smart minitor, which allows the client to obtain real-time changes in the server's resource usage data.

[0140] Specifically, the client submits subscription information through the desktop distributed bus's subscription interface based on its monitoring needs. This information includes, for example, the target server identifier and resource types such as CPU utilization and memory usage. The bus records the subscription information. Once the subscription takes effect, it establishes a data acquisition connection with the target server based on the subscription content and sends a resource acquisition request. After receiving the request, the server starts its local resource acquisition process and returns initial resource data to the bus. The bus temporarily stores the initial resource data acquired for the first time in a real-time resource cache as a baseline value for subsequent data changes.

[0141] Furthermore, the desktop distributed bus initiates asynchronous signal monitoring commands to monitor resource usage data on each server, synchronizing with the server resource collection frequency to ensure timely capture of data changes. A standard is set for how much data change constitutes a fluctuation. Resource data is periodically compared to determine if changes have occurred, retrieving current resource data from the target server's real-time resource cache. The current data is compared with the previously stored baseline value to determine if a change threshold has been reached. If the data has changed, the current data is updated to the new baseline value for the next comparison; otherwise, the original baseline value remains unchanged. If resource usage data changes, the changed resource usage data is sent to the client and stored as historical resource usage data.

[0142] The data transmission anomaly monitoring method provided in this application employs asynchronous signal monitoring commands from a desktop distributed bus tool, enabling real-time monitoring of resource usage changes on each server. Once a change occurs, the updated resource usage data is immediately sent to the client. This real-time feedback mechanism allows the client to quickly obtain the latest resource status from the server. By using asynchronous signal monitoring commands and automatically sending changed data, automated monitoring and notification of resource usage are achieved. There is no need for frequent manual queries of the resource status on each server, and the stored historical resource usage data provides rich data resources, facilitating data analysis and trend prediction.

[0143] As an optional implementation, based on any of the above embodiments, it further includes:

[0144] If the judgment result is that the current message traffic is abnormal or the current resource usage is abnormal, an early warning message is sent to the preset operation and maintenance terminal. The early warning message includes the abnormality type, the abnormal server identifier, the current abnormal data, and the associated tracking identifier.

[0145] If the judgment result indicates that the current message traffic is abnormal, then the call frequency of the corresponding client is limited.

[0146] If the determination result is that the current resource usage is abnormal, then the receiving of call requests from the server with abnormal current resource usage will be suspended.

[0147] Among them, abnormal message traffic refers to a situation in data transmission monitoring where the amount of message transmission between the client and the server deviates from the normal range in the target graph of the correlation between message traffic and resource consumption. For example, the amount of messages sent by a client to the server suddenly increases by 50%, without a corresponding increase in resource consumption.

[0148] Among them, abnormal current resource usage refers to a situation where the consumption of hardware or system resources on the server deviates from the normal correlation pattern during data transmission. For example, the message volume remains unchanged, but the CPU utilization rate increases from 40% to 95%.

[0149] Among them, limiting the call frequency of the corresponding client refers to setting a maximum call threshold per unit time for clients with abnormal message traffic. For example, if client A was originally allowed to call the server 100 times per minute, it is limited to 20 times per minute after the abnormality, so as to avoid abnormal traffic continuously occupying server resources.

[0150] Among them, suspending the receipt of call requests from abnormal servers means that for servers with abnormal resource usage, the data transmission anomaly monitoring device temporarily blocks new call requests sent to that server.

[0151] Specifically, the anomaly type, anomaly server identifier, current anomaly data, and associated tracking identifier are integrated into the warning message template to obtain complete warning information. The warning information is then pushed to the operation and maintenance terminals from a pre-configured list of terminals.

[0152] Furthermore, after pushing the alert information, if the result indicates that the current message traffic is abnormal, the call frequency of the corresponding client is adjusted according to the degree of abnormality. For example, a minor abnormality is limited to 50% of the normal threshold; a moderate abnormality is limited to 30% of the normal threshold, and so on.

[0153] Furthermore, after pushing the warning information, if the judgment result is that the current resource usage is abnormal, the status of the abnormal server is marked as paused to receive requests, and key information such as server identifier, pause start time, current resource status, and recovery conditions are recorded.

[0154] Optionally, after pausing request acceptance, when the client sends a new request to the abnormal server, a message indicating that the server is temporarily unavailable is returned directly. This message may include the abnormal server identifier, the current abnormal resource status, the estimated recovery time, and suggested actions.

[0155] The data transmission anomaly monitoring method provided in this application quickly sends early warning information to a preset maintenance terminal when abnormal message traffic or resource usage is detected, enabling maintenance personnel to be aware of the system's abnormality immediately. Through timely warnings, maintenance personnel can take immediate measures, such as expanding server resources or optimizing system configuration, to prevent system crashes due to excessive traffic and ensure stable system operation. When abnormal message traffic occurs, limiting the call frequency of the corresponding client can prevent excessive requests from continuously impacting the system and prevent abnormal traffic from triggering a chain reaction in other parts of the system. When abnormal resource usage occurs, suspending the reception of call requests from the currently abnormal server can prevent the server from being unable to process requests normally due to resource exhaustion, thereby affecting the stability of the entire system.

[0156] As an optional implementation, based on any of the above embodiments, it further includes:

[0157] In response to receiving at least one target message through the desktop distributed bus, at least one target message is sent to the preset partition corresponding to the distributed message queue. The target message includes at least one of the following: a call request, a server call result, and resource usage data.

[0158] The target messages are processed sequentially according to the preset processing rules.

[0159] Among them, target messages refer to various core data information transmitted through the desktop distributed bus, such as call requests, server call results, resource usage data, etc.

[0160] A distributed message queue is a component used to store and transmit messages in a distributed system. It enables asynchronous message transmission and buffering, preventing message congestion or loss. For example, when multiple clients send requests simultaneously, a message queue can temporarily store these requests and then distribute them to the server for processing in sequence.

[0161] In this context, a pre-defined partition refers to a message storage area pre-divided within the distributed message queue. Each partition is responsible for storing messages of a specific type or origin. For example, the request partition stores the call request, while the result partition stores the server-side call result. Partition management enables message classification, processing, and load balancing.

[0162] Among them, the preset processing rules refer to the processing specifications formulated for different types of messages, including processing order, processing method, processing time limit, etc.

[0163] Specifically, the system receives target messages and parses key information, such as message type, determining whether it's a call request, server call result, or resource usage data. Additional identifiers are added, including message source, priority, and timestamp. Based on a preset message type-partition mapping rule, the system determines the partition where each message should be stored. The target message is then sent to the designated partition via the message queue interface. Upon receiving the message, the message queue returns a confirmation signal indicating successful storage. Messages are retrieved from each partition of the message queue at a preset frequency, such as once every second, following a priority ordering principle within the partition. For example, within the same partition, urgent priority messages are processed first, followed by ordinary priority messages.

[0164] Optionally, after each message is processed, information such as message identifier, processing time, processing result, and processing time are recorded to ensure traceability. If a message times out or fails to process, the message is marked as pending retry and put back into the retry partition of the message queue to wait for the next round of processing.

[0165] The data transmission anomaly monitoring method provided in this application typically utilizes a distributed message queue with message persistence capabilities, enabling the storage of received target messages on persistent media such as disks. Even if the system experiences a failure or restart, messages will not be lost and can be processed again after system recovery. When the system receives a large number of sudden messages, the message queue can act as a smoothing mechanism. It temporarily stores the sudden influx of messages in the queue and then releases them smoothly according to the system's processing capacity.

[0166] Figure 3 A flowchart illustrating an anomaly monitoring method for data transmission provided in another embodiment of this application is shown below. Figure 3 As shown, the abnormal data transmission monitoring method provided in this embodiment is a processing scheme for the abnormal data transmission monitoring device when the client sends a call request to the server. The abnormal data transmission monitoring method provided in this embodiment includes the following steps:

[0167] S301. Receive a call request sent by the client through a synchronous method call command. The call request includes the service information corresponding to the server to be called, which the client obtains through the interface self-test tool.

[0168] S302. Determine whether the client has data access permission based on preset permission rules and call requests.

[0169] S303. If it is determined that the client has data access permission, a tracking identifier is generated based on the service information corresponding to the server to be called, and the tracking identifier and the call request are sent to the corresponding server.

[0170] S304. Receive the call result sent by the server.

[0171] S305. If the call result is successful, a call trace report is generated based on the trace identifier.

[0172] S306. If the call result is a failure, a call failure analysis report will be generated based on the tracing identifier.

[0173] S307. If the failure is due to a Type I problem, the call request, including the trace identifier, will be resent for a preset duration.

[0174] S308. Obtain the changed resource usage data and send the changed resource usage data to the client via asynchronous signal monitoring command.

[0175] S309. Input the call tracking report and the changed resource usage data into the association model, and determine whether there are any abnormal situations based on the preset relationship formula, message traffic threshold and resource usage threshold in the association model.

[0176] S310. If the current message traffic is greater than the message traffic threshold and the current resource usage data does not satisfy the relationship formula in the association model, then the current message traffic is judged to be abnormal.

[0177] S311. If the current resource usage is greater than the resource usage threshold and the current message traffic does not meet the relationship formula in the association model, then the current resource usage is judged to be abnormal.

[0178] It should be noted that the execution order of S305-S306 is not important.

[0179] In this embodiment, the implementation method and technical effect of S301-S311 are similar to those of the corresponding solutions in the above embodiments, and will not be repeated here.

[0180] Figure 4 This is a schematic diagram of the structure of a data transmission anomaly monitoring device provided in an embodiment of this application. Figure 4 As shown, the abnormal data transmission monitoring device provided in this embodiment is located in the electronic device. The abnormal data transmission monitoring device 40 provided in this embodiment includes: an acquisition module 41, a determination module 42, a construction module 43, an input module 44, and a judgment module 45.

[0181] Specifically, the acquisition module 41 is used to periodically acquire historical call tracing reports and historical resource usage data of multiple servers during data transmission; the determination module 42 is used to determine the target graph of message traffic and resource usage correlation between each server based on the historical call tracing reports and historical resource usage data; the construction module 43 is used to construct a correlation model based on the target graph of message traffic and resource usage correlation; the correlation model includes a preset relationship formula, message traffic threshold, and resource usage threshold corresponding to each server, and the preset relationship formula is the relationship formula between message traffic and resource usage corresponding to each server; the acquisition module 41 is also used to periodically acquire the target graph of message traffic and resource usage of each server. The system includes: a current message traffic and current resource usage data for each server; an input module 44 for inputting the current message traffic and current resource usage data for each server into the association model; and a judgment module 45 for judging whether the current message traffic and current resource usage are abnormal based on the preset relationship formula, message traffic threshold, and resource usage threshold in the association model. If the current message traffic is greater than the message traffic threshold and the current resource usage data does not satisfy the relationship formula in the association model, then the current message traffic is judged to be abnormal. If the current resource usage is greater than the resource usage threshold and the current message traffic does not satisfy the relationship formula in the association model, then the current resource usage is judged to be abnormal.

[0182] Optionally, the determining module 42, when determining the target graph of message traffic and resource usage relationships between servers based on historical call tracing reports and historical resource usage data, is specifically used for: obtaining the called relationships corresponding to each server from the historical call tracing reports, where the called relationship refers to the hierarchical relationship of a client calling a server and a server calling other servers; generating an initial graph of message traffic and resource usage relationships between servers based on the called relationships corresponding to each server, with servers as nodes and called relationships as edges; and optimizing the weights of the edges in the initial graph using historical resource usage data to determine the target graph of message traffic and resource usage relationships between servers.

[0183] Optionally, the abnormal data transmission monitoring device provided in this application further includes a registration module, a binding module, and a calling module.

[0184] Accordingly, the acquisition module 41 is also used to acquire service information corresponding to multiple servers through the desktop distributed bus; the registration module is used to register the corresponding service interface in the desktop distributed bus based on the service information; the binding module is used to bind the service interface with the corresponding service call path so that the client can acquire the service information corresponding to each server through the interface self-test tool of the desktop distributed bus; the determination module 42 is also used to respond to receiving the client sending a call request by calling the synchronous method call command of the desktop distributed bus tool, the call request including the service information corresponding to the server to be called, and determine the service interface and the corresponding service call path of the server to be called based on the service information corresponding to the server to be called; the calling module is used to call the server to be called using the service interface and the corresponding service call path of the server to be called.

[0185] Optionally, the abnormal data transmission monitoring device provided in this application further includes a generation module and a sending module.

[0186] Accordingly, the determining module 42 is also used to, in response to receiving a call request sent by the client, determine whether the client has data access permission based on the preset permission rules and the call request; the generating module is used to, if it is determined that the client has data access permission, generate a tracking identifier based on the service information corresponding to the server to be called; the sending module is used to send the tracking identifier and the call request to the corresponding server; the sending module is also used to, if it is determined that the client does not have data access permission, send a permission failure prompt message to the client.

[0187] Optionally, the abnormal data transmission monitoring device provided in this application also includes a receiving module.

[0188] Accordingly, the receiving module is used to receive the call result sent by the server; the generating module is also used to generate a call tracing report based on the tracing identifier if the call result is successful, and to generate a call failure analysis report based on the tracing identifier if the call result is unsuccessful.

[0189] Optionally, the abnormal data transmission monitoring device provided in this application also includes an analysis module.

[0190] Accordingly, the analysis module is used to determine the cause of call failure based on the call failure analysis report; the sending module is also used to resend the call request including the trace identifier according to a preset time if the cause of failure is the first type of problem. The first type of problem includes network fluctuations and temporary server busyness; if the cause of failure is the second type of problem, the call failure analysis report and the trace identifier are sent to the corresponding client. The second type of problem includes server failure and non-existent service interface.

[0191] Optionally, the abnormal data transmission monitoring device provided in this application further includes a monitoring module and a storage module.

[0192] Accordingly, the acquisition module 41 is also used to acquire resource usage data corresponding to multiple servers that the client has subscribed to in advance on the desktop distributed bus; the monitoring module is used to monitor whether the resource usage data in each server has changed using the asynchronous signal monitoring command of the desktop distributed bus tool; the sending module is also used to send the changed resource usage data to the client if the resource usage data changes; and the storage module is used to store the changed resource usage data as historical resource usage data.

[0193] Optionally, the sending module is further configured to send a warning message to a preset maintenance terminal if the judgment result is that the current message traffic is abnormal or the current resource usage is abnormal. The warning message includes the abnormality type, the abnormal server identifier, the current abnormal data, and the associated tracking identifier. The judgment module is further configured to limit the call frequency of the corresponding client if the judgment result is that the current message traffic is abnormal, and to suspend receiving call requests from the server with abnormal current resource usage if the judgment result is that the current resource usage is abnormal.

[0194] Optionally, the abnormal data transmission monitoring device provided in this application further includes a processing module.

[0195] Accordingly, the sending module is also configured to, in response to receiving at least one target message through the desktop distributed bus, send at least one target message to a preset partition corresponding to the distributed message queue, wherein the target message includes at least one of a call request, a server call result, and resource usage data; the processing module is configured to process the target messages sequentially according to preset processing rules.

[0196] It should be noted that the technical effects of the abnormal data transmission monitoring device in this embodiment have been explained in the embodiments of the abnormal data transmission monitoring method described above, and therefore will not be repeated in this embodiment.

[0197] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 51 and a memory 52. ​​Optionally, the electronic device 50 further includes a communication component 53. The processor 51, memory 52, and communication component 53 are connected via a bus.

[0198] In a specific implementation, at least one processor 51 executes computer execution instructions stored in memory 52, causing at least one processor 51 to execute the above-described data transmission anomaly monitoring method embodiment.

[0199] The specific implementation process of processor 51 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0200] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0201] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0202] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0203] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described embodiments of the abnormal data transmission monitoring method.

[0204] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0205] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described embodiments of the abnormal data transmission monitoring method.

[0206] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described abnormal data transmission monitoring method embodiments.

[0207] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0208] The above provides a detailed description of the abnormal data transmission monitoring method and electronic device provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for monitoring anomalies in data transmission, characterized in that, include: The system periodically retrieves historical call tracing reports and historical resource usage data from multiple servers during data transmission. The historical call tracing reports refer to a collection of historical logs recording the entire data transmission chain between the multiple servers. The historical resource usage data refers to historical records of hardware and system resource consumption by the multiple servers during data transmission. Based on the historical call tracking report and the historical resource usage data, a target graph of the relationship between message traffic and resource usage among the servers is determined; wherein, the target graph of the relationship between message traffic and resource usage is a visual structured chart used to intuitively present the inherent relationship between message transmission characteristics and resource consumption among the multiple servers; A correlation model is constructed based on the target graph of the relationship between message traffic and resource usage. The correlation model includes a preset relationship formula, a message traffic threshold, and a resource usage threshold for each server. The preset relationship formula is the formula for the relationship between message traffic and resource usage for each server. The preset relationship formula refers to a mathematical expression that is pre-set for each server to describe the normal correspondence between message traffic and resource usage. The formula parameters are different for different servers. Periodically retrieve the current message traffic and current resource usage data for each server. Input the current message traffic and current resource usage data corresponding to each of the aforementioned servers into the association model, and determine whether the current message traffic and current resource usage are abnormal based on the preset relationship formula, message traffic threshold and resource usage threshold in the association model; If the current message traffic exceeds the message traffic threshold and the current resource usage data does not satisfy the relationship formula in the association model, then the current message traffic is judged to be abnormal. If the current resource usage exceeds the resource usage threshold and the current message traffic does not meet the relationship formula in the association model, then the current resource usage is judged to be abnormal. If the judgment result is that the current message traffic is abnormal or the current resource usage is abnormal, an early warning message is sent to the preset operation and maintenance terminal. The early warning message includes the abnormality type, the abnormal server identifier, the current abnormal data, and the associated tracking identifier. If the judgment result is that the current message traffic is abnormal, the call frequency of the corresponding client is limited; if the judgment result is that the current resource usage is abnormal, the receiving of call requests from the server with abnormal current resource usage is suspended. The system obtains resource usage data corresponding to multiple servers that the client has pre-subscribed to on the desktop distributed bus; it uses the asynchronous signal monitoring command of the desktop distributed bus tool to monitor whether the resource usage data in each server has changed; if the resource usage data has changed, it sends the changed resource usage data to the client and stores the changed resource usage data as historical resource usage data.

2. The method according to claim 1, characterized in that, The process of determining the target graph of message traffic and resource usage correlations between servers based on the historical call tracing report and the historical resource usage data includes: The call relationship of each server is obtained from the historical call tracing report. The call relationship refers to the hierarchical relationship of the client calling the server and the server calling other servers. Based on the calling relationships of each of the aforementioned servers, an initial graph of message traffic and resource consumption relationships between the servers is generated, with the servers as nodes and the calling relationships as edges. The historical resource usage data is used to optimize the weights of the edges in the initial graph to determine the target graph of message traffic and resource usage relationships between each server.

3. The method according to claim 2, characterized in that, Also includes: Obtain service information corresponding to multiple servers through a desktop distributed bus; Based on the service information, the corresponding service interface is registered in the desktop distributed bus, and the service interface is bound to the corresponding service call path, so that the client can obtain the service information corresponding to each server through the interface self-test tool of the desktop distributed bus. In response to receiving a client sending a call request by calling the synchronization method of the desktop distributed bus tool, the call request includes service information corresponding to the server to be called, and the service interface and corresponding service call path of the server to be called are determined based on the service information corresponding to the server to be called; The service interface and corresponding service call path of the service to be called are used to call the service to be called.

4. The method according to claim 3, characterized in that, Also includes: In response to receiving a call request from a client, determine whether the client has data access permission based on preset permission rules and the call request; If it is determined that the client has data access permission, a tracking identifier is generated based on the service information corresponding to the server to be called, and the tracking identifier and the call request are sent to the corresponding server. If it is determined that the client does not have data access permission, a permission failure message is sent to the client.

5. The method according to claim 4, characterized in that, Also includes: Receive the call result sent by the server; If the call is successful, a call tracing report is generated based on the tracing identifier; If the call fails, a call failure analysis report is generated based on the trace identifier.

6. The method according to claim 5, characterized in that, Also includes: The cause of the call failure is determined based on the aforementioned call failure analysis report; If the failure is due to the first type of problem, the call request, including the tracking identifier, will be resent after a preset time. The first type of problem includes network fluctuations and temporary server overload. If the failure is due to the second type of problem, the call failure analysis report and the trace identifier will be sent to the corresponding client. The second type of problem includes server failure and non-existent service interface.

7. The method according to claim 3, characterized in that, Also includes: In response to receiving at least one target message through the desktop distributed bus, the at least one target message is sent to a preset partition corresponding to the distributed message queue, wherein the target message includes at least one of a call request, a server call result, and resource usage data; The target messages are processed sequentially according to preset processing rules.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the method as described in any one of claims 1 to 7.

Images