Communication method, system and apparatus
By acquiring the execution time and bandwidth data of CPU node loading instructions, analyzing CPU pipeline stalls, and combining multi-dimensional detection results, the problems of misjudgment and missed judgment in CE storm fault detection in existing technologies are solved, achieving higher detection accuracy.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- HUAWEI TECH CO LTD
- Filing Date
- 2025-11-18
- Publication Date
- 2026-06-04
AI Technical Summary
Existing technologies lack consideration for the impact on CPU performance when detecting server physical memory CE storm failures, leading to misjudgments or omissions, and the preset judgment thresholds lack a unified standard.
By acquiring the execution time and bandwidth data of CPU node load instructions, we can analyze CPU pipeline stalls, determine whether a CE storm failure has occurred in the local memory, and improve accuracy by combining multi-dimensional detection results.
It enables accurate determination of whether a CE storm failure has occurred in the local memory of a CPU node without relying on hardware alarms or interrupt notifications, thus improving the accuracy of detection.
Smart Images

Figure CN2025135670_04062026_PF_FP_ABST
Abstract
Description
A communication method, system and device
[0001] This application claims priority to Chinese Patent Application No. 202411766069.5, filed with the China National Intellectual Property Administration on November 30, 2024, entitled "A Communication Method, System and Apparatus", the entire contents of which are incorporated herein by reference. Technical Field
[0002] This application relates to the field of communication technology, and in particular to a communication method, system and apparatus. Background Technology
[0003] Traditional telecommunications systems consist of various dedicated hardware devices. With the development of communication technology and the growth of network scale, telecommunications systems are becoming increasingly complex. To reduce complexity, network function virtualization (NFV) technology can be used to migrate the functions of various entities used in the telecommunications network from the current dedicated hardware platform to general-purpose commercial off-the-shelf (COTS) servers. In other words, NFV technology pools and virtualizes infrastructure hardware resources, thereby providing virtual resources for upper-layer applications, achieving application-hardware decoupling, and enabling each application to quickly add virtual resources to rapidly expand system capacity or quickly reduce virtual resources to shrink system capacity.
[0004] During server operation and maintenance, physical memory may experience correctable error (CE) storms. CE storms can cause increased central processing unit (CPU) utilization and decreased performance, potentially affecting normal scheduling of business applications, and damaging business flow control and key performance indicators (KPIs). Currently, hardware signature detection methods are commonly used to determine whether physical memory has experienced a CE storm. These methods involve using the out-of-band baseboard management controller (BMC) or basic input / output system (BIOS) firmware to statistically analyze the temporal and / or spatial distribution of CE occurrences in physical memory. When a preset threshold is reached, a CE storm is considered to have occurred. However, this method does not consider the impact of CE errors on CPU performance, or the extent of that impact. Furthermore, the preset threshold is an empirical parameter, which may lead to missed or false positives in practical applications. Summary of the Invention
[0005] This application provides a communication method, system, and apparatus that enables the detection of whether a CE storm fault has occurred in the local memory corresponding to a CPU node without relying on hardware alarms or hardware interrupt notifications.
[0006] In a first aspect, embodiments of this application provide a communication method, including:
[0007] The system acquires first data and first bandwidth. The first data includes at least one of a first duration and a second duration. The first duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by a central processing unit (CPU) node, from issuing a physical memory load request to obtaining a physical memory data response. The second duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by the CPU node, from issuing a physical memory load request to obtaining a physical memory data response. The first bandwidth is the bandwidth used by all memory controllers in the CPU node to read physical memory.
[0008] Based on the first data and the first bandwidth, determine whether a correctable error CE storm failure has occurred in the local memory corresponding to the CPU node.
[0009] By acquiring and analyzing the first data and the first bandwidth, it is helpful to determine whether a CE storm fault has occurred in the local memory corresponding to the CPU node. This enables the detection of whether a CE storm fault has occurred in the local memory corresponding to the CPU node without relying on hardware alarms or hardware interrupt notifications, thereby improving the accuracy of CE storm fault detection.
[0010] In one possible design, the single access response speed is determined based on the first data; the batch data transfer rate is determined based on the first bandwidth; and the CE storm failure of the local memory is determined based on the single access response speed and the batch data transfer rate. Analyzing the first data and the first bandwidth helps determine whether reading physical memory affects CPU performance and the extent of that impact, thereby determining whether a CE storm has occurred in the local memory. This allows for the detection of CE storm failures in the local memory without relying on hardware alarms or hardware interrupt notifications, improving the accuracy of CE storm detection.
[0011] In another possible design, the first data includes the first duration; when the first duration is greater than or equal to a first threshold within a preset time period, and the first bandwidth is less than or equal to a third threshold within the preset time period, it is determined that the local memory has experienced the CE storm failure; when the first duration is less than the first threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that the local memory has not experienced the CE storm failure. By determining whether the first duration and the first bandwidth meet the preset conditions, it is beneficial to determine whether reading physical memory affects CPU performance and the degree of impact, thereby determining whether the local memory has experienced a CE storm failure. This achieves the ability to detect whether the local memory has experienced a CE storm failure without relying on hardware alarms or hardware interrupt notifications, improving the accuracy of CE storm failure detection.
[0012] In another possible design, the first data includes the second duration; when the second duration is greater than or equal to a second threshold within a preset time period, and the first bandwidth is less than or equal to a third threshold within the preset time period, it is determined that the local memory has experienced the CE storm failure; when the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that the local memory has not experienced the CE storm failure. By determining whether the second duration and the first bandwidth meet the preset conditions, it is beneficial to determine whether reading physical memory affects CPU performance and the degree of impact, thereby determining whether the local memory has experienced a CE storm failure. This achieves the ability to detect whether the local memory has experienced a CE storm failure without relying on hardware alarms or hardware interrupt notifications, improving the accuracy of CE storm failure detection.
[0013] In another possible design, the first data includes the first duration and the second duration. When the first duration is greater than or equal to a first threshold within a preset time period, the second duration is greater than or equal to a second threshold within the preset time period, and the first bandwidth is less than or equal to a third threshold within the preset time period, it is determined that the local memory has experienced a CE storm failure. When the first duration is less than the first threshold within the preset time period, the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that the local memory has not experienced a CE storm failure. By determining whether the first duration, second duration, and first bandwidth meet preset conditions, it is beneficial to determine whether reading physical memory affects CPU performance and the degree of impact, thereby determining whether a CE storm failure has occurred in the local memory. This achieves the ability to detect whether a CE storm failure has occurred in the local memory without relying on hardware alarms or hardware interrupt notifications, improving the accuracy of CE storm failure detection.
[0014] In another possible design, the total duration of CPU pipeline pauses during the process of the CPU node executing multiple load instructions, from issuing a physical memory load request to obtaining the physical memory data response, and the number of load instructions are obtained. The total duration of CPU pipeline pauses is divided by the number of load instructions to obtain the first duration. By obtaining the total duration of CPU pipeline pauses and the number of load instructions, it is beneficial to calculate the first duration, thereby determining whether reading physical memory affects CPU performance and the degree of impact.
[0015] In another possible design, the total CPU pipeline time and the number of load instructions are obtained during the process of the CPU node executing multiple load instructions, from issuing a physical memory load request to obtaining the physical memory data response. The total CPU pipeline time is then divided by the number of load instructions to obtain the second time. By obtaining the total CPU pipeline time and the number of load instructions, it is beneficial to calculate the second time, thereby determining whether reading physical memory affects CPU performance and the degree of that impact.
[0016] In another possible design, the number of bytes read from physical memory by all memory controllers in the CPU node and a preset measurement interval are obtained; the number of bytes is divided by the preset measurement interval to obtain the first bandwidth. By obtaining the number of bytes read from physical memory by all memory controllers in the CPU node and the preset measurement interval, it is beneficial to calculate the first bandwidth, thereby determining whether reading physical memory affects CPU performance and the degree of impact.
[0017] In another possible design, a first detection result is determined based on the first data and the first bandwidth; a second detection result is determined based on first information, whereby the first information includes at least one of the following: hardware feature detection result, host CPU usage abnormal increase alarm result, and business key performance indicator (KPI) abnormal alarm result; and the local memory is determined to have experienced the CE storm failure based on the first and second detection results. By overlaying detection results from other dimensions for comprehensive judgment, the reliability of the results can be improved.
[0018] In another possible design, a first detection result is determined based on the first data; a second detection result is determined based on first information, wherein the first information includes at least one of hardware feature detection results, alarm results of abnormally high host CPU usage, and alarm results of abnormal business key performance indicators (KPIs); and the local memory is determined to have experienced the CE storm failure based on the first and second detection results. By combining detection results from other dimensions for comprehensive judgment, the reliability of the results can be improved.
[0019] In another possible design, a CE storm failure is determined to have occurred in the local memory when both the first and second detection results indicate that the local memory has experienced a CE storm failure. By combining detection results from other dimensions for a comprehensive judgment, the reliability of the results can be improved.
[0020] In another possible design, when the first detection result is that the local memory has not experienced the CE storm failure, and / or the second detection result is that the local memory has not experienced the CE storm failure, the determination that the local memory has not experienced the CE storm failure is made by comprehensively judging by superimposing the detection results of other dimensions, which is beneficial to improving the reliability of the results.
[0021] In one possible design, the method of the first aspect can be executed by the first entity, or by a module applied to the first entity (e.g., a processor, chip, or chip system), or by a logical node, logical module, or software capable of implementing all or part of the functions of the first entity.
[0022] In another possible design, the method of the first aspect can be executed by the second entity, or by a module applied to the second entity (e.g., a processor, chip, or chip system), or by a logical node, logical module, or software capable of implementing all or part of the functions of the second entity.
[0023] Secondly, embodiments of this application provide a communication device configured to implement the methods and functions described in the first aspect. This communication device is implemented in hardware / software. It includes modules corresponding to the aforementioned functions.
[0024] In one possible design, the communication device can be a first entity, a chip, chip system, or processor that supports the first entity in implementing the above-described method, or a logic node, logic module, or software that can implement all or part of the functions of the first entity.
[0025] In another possible design, the communication device can be a second entity, a chip, chip system, or processor that supports the second entity in implementing the above methods, or a logic node, logic module, or software that can implement all or part of the functions of the second entity.
[0026] Thirdly, embodiments of this application provide a communication device including one or more processors. The one or more processors enable the communication device to implement the methods in any possible design or implementation of the first aspect described above.
[0027] Fourthly, embodiments of this application provide a communication system comprising at least one first entity and at least one second entity. The first entity is configured to acquire the total duration of CPU pipeline pauses and the number of load instructions during the process of a CPU node executing multiple load instructions from issuing a physical memory load request to obtaining a physical memory data response, and divide the total duration of CPU pipeline pauses by the number of load instructions to obtain a first duration; and / or, acquire the total duration of CPU pipeline pauses and the number of load instructions during the process of a CPU node executing multiple load instructions from issuing a physical memory load request to obtaining a physical memory data response, and divide the total duration of CPU pipeline pauses by the number of load instructions to obtain a second duration; and acquire the number of bytes read from physical memory by all memory controllers in the CPU node and a preset measurement interval, and divide the number of bytes by the preset measurement interval to obtain a first bandwidth; the second entity is configured to acquire first data and the first bandwidth from the first entity, the first data including at least one of the first duration and the second duration, and determine whether a CE storm failure has occurred in the local memory corresponding to the CPU node based on the first data and the first bandwidth.
[0028] In one possible design, the communication system may further include at least one third entity, which is used to determine a first detection result based on first data and a first bandwidth, and to determine a second detection result based on first information, wherein the first information includes at least one of hardware feature detection results, alarm results of abnormally high host CPU usage, and alarm results of abnormal business KPIs, and to determine whether a CE storm failure has occurred in the local memory based on the first detection result and the second detection result.
[0029] In another possible design, the communication system may also include at least one third entity, which is used to determine a first detection result based on first data, and to determine a second detection result based on first information, wherein the first information includes at least one of hardware feature detection results, alarm results of abnormally high host CPU usage, and alarm results of abnormal business KPIs, and to determine whether a CE storm failure has occurred in the local memory based on the first detection result and the second detection result.
[0030] Fifthly, embodiments of this application provide a communication system comprising at least one first entity and at least one second entity. The first entity is configured to acquire the total duration of CPU pipeline pauses during the process of a CPU node executing multiple load instructions from issuing a physical memory load request to obtaining a physical memory data response, and / or the total duration of CPU pipeline pauses during the process of a CPU node executing multiple load instructions from issuing a physical memory load request to obtaining a physical memory data response, the number of load instructions, the number of bytes read from physical memory by all memory controllers in the CPU node, and a preset measurement interval. The second entity is configured to acquire the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline pauses, the number of load instructions, the number of bytes, and the preset measurement interval from the first entity; divide the total duration of CPU pipeline pauses by the number of load instructions to obtain a first duration; and / or divide the total duration of CPU pipeline pauses by the number of load instructions to obtain a second duration; and divide the number of bytes by the preset measurement interval to obtain a first bandwidth; and determine whether a CE storm failure has occurred in the local memory corresponding to the CPU node based on at least one of the first duration and the second duration and the first bandwidth.
[0031] In one possible design, the communication system may further include at least one third entity, which is used to determine a first detection result based on first data and a first bandwidth, and to determine a second detection result based on first information, wherein the first information includes at least one of hardware feature detection results, alarm results of abnormally high host CPU usage, and alarm results of abnormal business KPIs, and to determine whether a CE storm failure has occurred in the local memory based on the first detection result and the second detection result.
[0032] In another possible design, the communication system may also include at least one third entity, which is used to determine a first detection result based on first data, and to determine a second detection result based on first information, wherein the first information includes at least one of hardware feature detection results, alarm results of abnormally high host CPU usage, and alarm results of abnormal business KPIs, and to determine whether a CE storm failure has occurred in the local memory based on the first detection result and the second detection result.
[0033] In a sixth aspect, embodiments of this application provide a computer-readable storage medium for storing a computer program that, when executed, causes the method described in any one of the first aspects to be implemented.
[0034] In a seventh aspect, embodiments of this application provide a computer program product containing instructions that, when executed, cause the method described in any one of the first aspects to be implemented.
[0035] Eighthly, embodiments of this application provide a chip including a processor and a communication interface for communicating with external or internal devices, the processor enabling the chip to implement the methods described above.
[0036] In one possible design, the chip may further include a memory storing computer programs or instructions, which the processor executes, either from the stored computer programs or instructions or derived from other programs or instructions. When the computer program or instructions are executed, the processor causes the chip to implement the methods described above.
[0037] In another possible design, the chip can be integrated into a communication system or communication device. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.
[0039] Figure 1 is a schematic diagram of an NFV system;
[0040] Figure 2 is a flowchart illustrating a communication method provided in an embodiment of this application;
[0041] Figure 3 is a flowchart illustrating another communication method provided in an embodiment of this application;
[0042] Figure 4 is a flowchart illustrating another communication method provided in an embodiment of this application;
[0043] Figure 5 is a flowchart illustrating another communication method provided in an embodiment of this application;
[0044] Figure 6 is a flowchart illustrating another communication method provided in an embodiment of this application;
[0045] Figure 7 is a schematic diagram of the structure of a communication device provided in an embodiment of this application;
[0046] Figure 8 is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation
[0047] The following explanations of some of the terms used in this application are provided to facilitate understanding by those skilled in the art.
[0048] (1) Network function virtualization (NFV) technology: refers to migrating the functions of various entities used in telecommunications networks from the current dedicated hardware platform to general-purpose commercial-off-the-shelf (COTS) servers.
[0049] (2) Correctable error (CE) storm failure: refers to a large number of correctable CE errors appearing in the physical memory of the server in a short period of time, causing the central processing unit (CPU) to increase its utilization rate and decrease its performance. In severe cases, it affects the normal scheduling of business programs and may cause damage to business flow control and KPIs.
[0050] (3) Non-uniform memory access (NUMA) architecture: This is a processor and memory topology that allows each processor node to have a portion of local memory and a relatively short access latency with the processor on that node. Accessing the memory of other nodes across nodes will result in a large latency, a phenomenon known as "non-uniform access".
[0051] The embodiments of this application are described below with reference to the accompanying drawings.
[0052] It should be understood that in the description of this application, "at least one" means one or more, and "multiple" means two or more. In addition, the words "first," "second," etc., unless otherwise stated, are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance or order.
[0053] It should be understood that in the description of this application, the indication includes direct indication (also known as explicit indication) and implicit indication. Direct indication information A refers to information A being included; implicit indication information A refers to information A being indicated through the correspondence between information A and information B, and the direct indication information B. The correspondence between information A and information B can be predefined, pre-stored, pre-burned, or pre-configured.
[0054] It should be understood that, in the description of this application, information C is used to determine information D, including both situations where information D is determined solely based on information C and situations where it is determined based on information C and other information. Furthermore, information C can also be used to determine information D indirectly, for example, where information D is determined based on information E, and information E is determined based on information C.
[0055] Furthermore, in this application, "Entity A sends message A to entity B" can be understood as meaning that the destination of message A or an intermediate entity in the transmission path between the destination and entity B is entity B, which may include sending the message directly or indirectly to entity B. Similarly, "Entity B receives message A from entity A" can be understood as meaning that the source of message A or an intermediate entity in the transmission path between the source and entity A is entity A, which may include receiving the message directly or indirectly from entity A. The message may undergo necessary processing between the source and destination, such as format changes, but the destination can understand a valid message from the source. Similar expressions in this application can be interpreted in a similar way and will not be elaborated further here.
[0056] The technical solutions provided in this application can be applied to various communication systems, such as 5G or new radio (NR) systems, long term evolution (LTE) systems, LTE frequency division duplex (FDD) systems, LTE time division duplex (TDD) systems, wireless local area network (WLAN) systems, satellite communication systems, future communication systems, or integrated systems of multiple systems. The technical solutions provided in this application can also be applied to device-to-device (D2D) communication, vehicle-to-everything (V2X) communication, machine-to-machine (M2M) communication, machine-type communication (MTC), and Internet of Things (IoT) communication systems, or other communication systems.
[0057] In a communication system, an entity can send signals to or receive signals from another entity. Signals can include information, signaling, or data; an entity can also be replaced by a network element, network entity, device, communication equipment, communication module, node, or communication node, etc. This disclosure uses an entity as an example for description.
[0058] Traditional telecommunications systems consist of various dedicated hardware devices, with different applications using different hardware. With the development of communication technologies and the growth of network scale, telecommunications systems have become increasingly complex, bringing numerous challenges, such as the development and launch of new services, system operation and maintenance, and resource utilization. To reduce complexity, NFV technology can be used to transform the various entities used in the telecommunications network into independent applications. These applications can be flexibly deployed on a unified infrastructure platform built on standard servers, storage, switches, and other devices. Furthermore, through virtualization technology, infrastructure hardware resources can be pooled and virtualized, providing virtual resources to upper-layer applications, achieving application-hardware decoupling. This allows each application to quickly add virtual resources to rapidly expand system capacity or quickly reduce virtual resources to shrink system capacity, greatly improving network resilience. In addition, by using general-purpose COTS servers to form a shared resource pool, newly developed services do not require separate hardware deployment, significantly shortening the time to market for new services.
[0059] The foundation of NFV technology includes cloud computing and virtualization technologies. General-purpose COTS (Compute-on-Demand) hardware devices such as computing, storage, and networking equipment can be decomposed into various virtual resources for use by upper-layer applications through virtualization. Virtualization technology decouples applications from hardware, significantly increasing the speed of virtual resource provisioning; cloud computing technology enables elastic scaling of applications, matching virtual resources with business loads, thus improving not only the utilization efficiency of virtual resources but also the system's response speed.
[0060] Figure 1 shows a schematic diagram of an NFV system. This NFV system can be used in various networks, such as a data center network, a carrier network, or a local area network. The NFV system includes an NFV management and orchestration (NFV MANO) 101, an NFV infrastructure (NFVI) layer 102, multiple virtual network functions (VNFs) 103, multiple element management (EM) 104, and an operation-support system / business support system (OSS / BSS) 105. Detailed descriptions of each module are as follows.
[0061] NFV MANO 101: Used to perform monitoring and management of NFVI 102 and VNF 103.
[0062] The NFV MANO 101 includes an NFV orchestrator (NFVO), one or more VNF managers (VNFMs), and a virtualized infrastructure manager (VIM). Details are as follows:
[0063] (1) NFVO: It can implement network services (such as L2 and L3 VPN services) on NFVI 102, and can also execute resource-related requests from one or more VNFMs, send configuration information to VNFMs, and collect status information of VNF 103. NFVO can communicate with VIM to allocate and / or reserve resources and exchange configuration and status information of virtualization hardware resources.
[0064] (2) VNFM: It can perform various management functions. For example, it can manage one or more VNF 103s, including but not limited to instantiation, updating, querying, scaling and / or terminating VNF 103s.
[0065] (3) VIM: It can perform resource management functions. For example, it manages the allocation of infrastructure resources (e.g., adding resources to virtual containers) and operational functions (e.g., collecting NFVI fault information). VNFM and VIM can communicate with each other to allocate resources and exchange configuration and status information of virtualized hardware resources.
[0066] NFVI 102: Includes hardware resources, software resources, or a combination of both to complete the deployment of a virtualized environment. That is to say, the hardware resource layer and the virtualization layer are used to provide virtualization resources, such as virtual machines and other forms of virtual containers, for VNF 103.
[0067] NFVI 102 comprises a virtual resource layer, a virtualization layer, and a hardware resource layer. The virtual resource layer includes virtual computing, virtual storage, and virtual networking. The hardware resource layer includes computing hardware, storage hardware, and network hardware. Details are as follows:
[0068] (1) Virtual computing and virtual storage: can be provided to VNF 103 in the form of virtual machines and / or other virtual containers. For example, one or more VNF 103s can be deployed on a virtual machine.
[0069] (2) Virtual network: can include virtual switch, which is used to provide connection between virtual machines and other virtual machines.
[0070] (3) Virtualization layer: It can abstract hardware resources from the physical layer and decouple VNF 103 to provide virtualization resources to VNF 103. The virtualization layer can also abstract network hardware to form a virtual network.
[0071] (4) Computing hardware: This can be commercially available hardware and / or user-customized hardware used to provide processing and computing resources.
[0072] (5) Storage hardware: This can be storage capacity provided within the network or storage capacity residing in the storage hardware itself (local storage located within the server).
[0073] (6) Network hardware: The transport network in the network hardware can be virtualized using a centralized control plane and a separate forwarding plane (e.g., software-defined networking, SDN).
[0074] Optionally, computing and storage hardware resources can also be pooled together.
[0075] Optionally, the network hardware may also be switches, routers, and / or any other network devices configured to have switching capabilities.
[0076] Optionally, the network hardware can also span multiple domains and may include multiple networks interconnected by one or more transport networks.
[0077] VNF 103: Can be configured to virtualize at least one network function performed through a physical network device.
[0078] For example, VNF 103 can be configured to provide the functions available to different entities in an Internet Protocol Multimedia Subsystem (IMS) network, such as proxy-call session control function (P-CSCF), serving-call session control function (S-CSCF), or home subscriber server (HSS) network functions.
[0079] EM 104: With configuration, one or more VNF103s can be managed.
[0080] VNFM can interact with VNF 103 and EM 104 to manage the lifecycle of VNF 103 and exchange configuration and status information.
[0081] OSS / BSS105 mainly consists of network management, system management, billing, sales, accounting and customer service, and the systems are organically integrated through a unified information bus.
[0082] It should be noted that Figure 1 is merely an exemplary structural diagram of an NFV system, and in practical applications, the NFV system in Figure 1 can be modified accordingly based on specific circumstances. Furthermore, it should be noted that the technical solution provided in this application can also be applied to modules such as NFVI, VIM, and / or VNFM in the NFV system shown in Figure 1.
[0083] During server (or NFV system) maintenance, physical memory may experience a CE storm failure. A CE storm failure can cause increased CPU utilization and decreased performance, severely impacting the normal scheduling of business applications and potentially causing damage to business traffic control and KPIs. When such failures occur, due to a lack of effective detection methods and direct evidence, the root cause often remains unidentified for a considerable period.
[0084] Currently, hardware feature detection is mainly used to determine whether physical memory has experienced a CE storm. Hardware feature detection involves statistically analyzing the temporal and / or spatial distribution of CE errors in physical memory using an out-of-band BMC or BIOS firmware program. When a preset threshold is reached, a CE storm is considered to have occurred. However, hardware feature detection has the following drawbacks: (1) It simply counts the distribution of CE errors without considering whether these CE errors affect CPU performance or the extent of that impact; (2) The preset threshold is set based on expert experience and there is no unified industry standard. In practical applications, the threshold may be set too strictly or too leniently, leading to missed or incorrect detections.
[0085] In addition, there are some improvement methods. For example, adding a judgment on increased CPU utilization to the hardware feature detection conditions can reduce false positives to some extent. However, there are many reasons for increased CPU utilization (e.g., sudden business operations, human operation, software program errors (bugs) etc.), and logically, these possible causes cannot be ruled out.
[0086] To address the aforementioned technical problems, the embodiments of this application provide the following solutions.
[0087] As shown in Figure 2, Figure 2 is a flowchart illustrating a communication method provided in an embodiment of this application. This communication method includes, but is not limited to, the following steps:
[0088] S201: Obtain first data and first bandwidth. The first data includes at least one of a first duration and a second duration. The first duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by a CPU node, from issuing a physical memory load request to obtaining a physical memory data response. The second duration is the average duration of CPU pipeline execution during a single execution of a load instruction by a CPU node, from issuing a physical memory load request to obtaining a physical memory data response. The first bandwidth is the bandwidth of all memory controllers in the CPU node reading physical memory.
[0089] The units for the first and second durations are CPU cycles; the unit for the first bandwidth is bytes per second (byte / s); the first duration, second duration, and first bandwidth are comprehensive indicators of the performance monitoring unit (PMU), which can be calculated from the PMU's raw metrics. In CPU pipeline technology, various instruction-level parallelism techniques (e.g., out-of-order execution, speculative execution, branch prediction, hyper-threading, etc.) are used to improve pipeline efficiency and reduce pauses. When the degree of instruction-level parallelism is low, the first and second durations may be very close; conversely, when the degree of instruction-level parallelism is high, the first duration will be shorter than the second duration because the CPU pipeline can execute other instructions to hide some of the latency in memory accesses.
[0090] In one possible implementation, the first entity / second entity can directly obtain the first duration and / or the second duration, as well as the first bandwidth.
[0091] In another possible implementation, the first entity / second entity can first obtain the total duration of CPU pipeline pauses during the process of multiple executions of load instructions by the CPU node from issuing a physical memory load request to obtaining a physical memory data response, and / or the total duration of CPU pipeline pauses during the process of multiple executions of load instructions by the CPU node from issuing a physical memory load request to obtaining a physical memory data response, the number of load instructions, and the number of bytes read from physical memory by all memory controllers in the CPU node; then, the first entity / second entity divides the total duration of CPU pipeline pauses by the number of load instructions to obtain a first duration, and / or divides the total duration of CPU pipeline pauses by the number of load instructions to obtain a second duration, and divides the number of bytes by a preset measurement interval to obtain a first bandwidth.
[0092] Among them, the total CPU pipeline duration, the number of loaded instructions and the number of bytes are all raw PMU indicators. The preset measurement interval is pre-configured and is not a raw PMU indicator. The unit of the preset measurement interval can be seconds, microseconds, etc., and this application does not limit it.
[0093] It should be noted that each of the PMU raw metrics mentioned above is aggregated and measured on a CPU node basis. A CPU node may include multiple CPU core units, which share several memory controllers.
[0094] For the COTS server in the telecom cloud, the raw PMU metrics mentioned above can be measured using the performance monitoring counter (PMC) used to implement the event counter function.
[0095] For example, the measurement methods for the 10 common CPU generations and corresponding PMU raw metrics of telecom cloud COTS servers are shown in Table 1. Specifically, the architectures of the above 10 CPU generations include x86 and ARM V8. The suppliers for x86 include Intel and Hygon / AMD, and the supplier for ARM V8 includes HiSilicon. The corresponding Intel CPU generations include Xeon SP4 Sapphire Rapids, Xeon SP3 Ice Lake, Xeon SP2 Cascade Lake, Xeon SP1 Skylake, Xeon E5 V4 Broadwell-EP, Xeon E5 V3 Haswell-EP, and Xeon E5 V2 Ivy Bridge-EP. The number of bytes corresponding to these seven CPU generations is measured using sum(uncore_imc_# / cas_count_read / )*64B. The total CPU pipeline pause time for the four CPU generations Xeon SP4 Sapphire Rapids, Xeon SP3 Ice Lake, Xeon SP2 Cascade Lake, and Xeon SP1 Skylake is measured using cycle_activity.stalls_l3_miss, the total CPU pipeline time is measured using cycle_activity.cycles_l3_miss, and the number of load instructions is measured using LLC-load-misses or sum(node-loads,node-load-misses). The Xeon E5 V4 Broadwell-EP, Xeon E5 V3... The total CPU pipeline pause time for the three CPU generations, Haswell-EP and Xeon E5 V2 Ivy Bridge-EP, was measured using cycle_activity.stalls_l2_pending; the total CPU pipeline time was measured using cycle_activity.cycles_l2_pending; and the number of loaded instructions was measured using LLC-loads.
[0096] The corresponding CPU generations for Hygon / AMD include ZEN 1. ZEN 1 does not support measuring the total duration of CPU pipeline pauses. The total CPU pipeline duration and the number of loaded instructions for ZEN 1 are measured using Average L3 Read Miss Latency (in core clocks), and the number of bytes is measured using Combined DRAM B / bytes of all channels on an NPS1 node (die). Here, Average L3 Read Miss Latency (in core clocks) refers to L3Event[0xFF0F000000400090]*16 / L3Event[0xFF0F000000401F9A]; and Combined DRAM B / bytes of all channels on an NPS1 node (die) refers to... node(die) refers to (DfEvent[0x0000000000403807]+DfEvent[0x0000000000403847]+DfEvent[0x0000000000403887]+DfEvent[0x00000000004038 C7]+DfEvent[0x0000000100403807]+DfEvent[0x0000000100403847]+DfEvent[0x0000000100403887]+DfEvent[0x00000001004038C7])*64B.
[0097] HiSilicon corresponds to the Kunpeng-920C and Kunpeng-920 CPU generations. The total CPU pipeline duration for Kunpeng-920C and Kunpeng-920 is not supported for measurement. The number of load instructions for these two CPU generations is measured using LL_CACHE_RD (r0036), and the number of bytes is measured using sum(hisi_sccl#_ddrc# / flux_rd / )*32B. The total pipeline pause duration for the Kunpeng-920C CPU is measured using MEM_STALL_L2MISS (r7008), and the total pipeline pause duration for the Kunpeng-920 CPU is measured using MEM_STALL_L2MISS (r7007).
[0098] Table 1
[0099] In Table 1, the Kunpeng-920 and 920C CPUs did not use the MEM_STALL_L3MISS measurement method to determine the total CPU pipeline pause duration; instead, they used the MEM_STALL_L2MISS measurement method as an approximation. Although the MEM_STALL_L2MISS measurement includes L3 cache latency, the latency variation of the L3 cache is negligible compared to the latency variation introduced by the CE storm failure. Similarly, the Xeon E5 V4 Broadwell-EP, Xeon E5 V3 Haswell-EP, and Xeon E5 V2 Ivy Bridge-EP CPUs did not use the cycle_activity.stalls_l3_miss measurement method to determine the total CPU pipeline pause duration; instead, they used the cycle_activity.stalls_l2_pending measurement method as an approximation.
[0100] Optionally, for CPU generations not covered in Table 1, the measurement methods for the corresponding PMU raw metrics can be obtained by consulting the technical reference manual (TRM) provided by the CPU vendor.
[0101] S202: Based on the first data and the first bandwidth, determine whether a CE storm failure has occurred in the local memory corresponding to the CPU node.
[0102] In this context, the local memory corresponding to a CPU node can be the local memory of a NUMA node. The CPU core unit and its local memory together form a NUMA node. The CPU prioritizes accessing the local memory of its NUMA node to obtain optimal performance.
[0103] Specifically, the first / second entity determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node in the following three ways: Method 1, determine whether the first duration and the first bandwidth meet the preset conditions; Method 2, determine whether the second duration and the first bandwidth meet the preset conditions; Method 3, determine whether the first duration, the second duration, and the first bandwidth meet the preset conditions.
[0104] The specific process of determining whether the local memory corresponding to the CPU node has experienced a CE storm fault can be referred to the corresponding description in the method embodiments shown in Figures 3-6, and will not be described in detail here.
[0105] In this embodiment, by acquiring the first data and the first bandwidth, the first entity / second entity can determine whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the first duration and / or the second duration and the first bandwidth in the first data. This enables the detection of whether a CE storm fault has occurred in the local memory corresponding to the CPU node without relying on hardware alarms or hardware interrupt notifications, thereby improving the accuracy of CE storm fault detection.
[0106] This application uses the example of NFVI as the first entity, VIM as the second entity, VNFM as the third entity, management data analytics function (MDAF) entity, cloud-network collaborative fault handling system, VIM, OSS / BSS or physical infrastructure manager (PIM) to illustrate the communication method shown in Figures 3-6. Among them, the CPU PMU indicator acquisition module, CPU core L3 cache related event counter, CPU core pipeline related event counter, and CPU uncore memory controller related event counter shown in Figures 3-6 can be deployed in NFVI. The various processes involved in these modules can be understood as functions of NFVI. The memory CE storm fault determination module can be deployed in NFVI or VIM. The various processes involved in this module can be understood as functions of NFVI or VIM. The memory CE storm fault comprehensive determination module can be deployed in VNFM, MDAF, cloud-network collaborative fault handling system, VIM, OSS / BSS, or PIM. The various processes involved in this module can be understood as functions of VNFM, MDAF, cloud-network collaborative fault handling system, VIM, OSS / BSS, or PIM.
[0107] Optionally, the six names—CPU PMU indicator acquisition module, CPU core L3 cache related event counter, CPU core pipeline related event counter, CPU uncore memory controller related event counter, memory CE storm fault determination module, and memory CE storm fault comprehensive determination module—are used as examples and do not constitute a limitation on the embodiments of this application. With the development of communication technology and NFV technology, these six modules may adopt other names. For example, the CPU core L3 cache related event counter can also be described as a register used to implement the CPU core L3 cache related event counting function; the CPU core pipeline related event counter can also be described as a register used to implement the CPU core pipeline related event counting function; and so on, which will not be elaborated here.
[0108] As shown in Figure 3, Figure 3 is a flowchart illustrating another communication method provided in an embodiment of this application. The details are as follows.
[0109] In this scenario, the CPU PMU metric acquisition module is used for acquiring raw PMU metrics and calculating comprehensive PMU metrics, while the memory CE storm fault determination module is used for fault detection and judgment. This allows for the detection of whether a CE storm fault has occurred in the local memory corresponding to the CPU node without relying on hardware alarms or hardware interrupt notifications, thus improving the accuracy of CE storm fault detection. The steps in this embodiment include at least:
[0110] S301: The CPU PMU indicator acquisition module obtains the number of load instructions from the CPU core L3 cache related event counter.
[0111] Here, the CPU core L3 cache related event counter can be the PMC used to implement the CPU core L3 cache related event counter function; the number of load instructions can be the L3 cache miss load instruction count.
[0112] S302: The CPU PMU metrics acquisition module obtains the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline from the CPU core pipeline-related event counter.
[0113] Here, the CPU core pipeline-related event counter can be the PMC used to implement the CPU core pipeline-related event counter function; the total CPU pipeline pause time refers to the total CPU pipeline pause time during the process of multiple CPU node executions of load instructions from issuing a physical memory load request to obtaining physical memory data response. The total CPU pipeline pause time can be the L3 cache miss pipeline pause count; the total CPU pipeline time refers to the total CPU pipeline time during the process of multiple CPU node executions of load instructions from issuing a physical memory load request to obtaining physical memory data response. The total CPU pipeline time can be the L3 cache miss pipeline cycle count.
[0114] It should be noted that, since different CPU models support different PMC ranges, for the transmission of the two PMU raw metrics, namely the total duration of CPU pipeline pauses and the total duration of CPU pipeline, the total duration of CPU pipeline pauses is preferred, and the total duration of CPU pipeline is the second choice.
[0115] S303: The CPU PMU metrics acquisition module obtains the number of bytes from the CPU uncore memory controller related event counters.
[0116] Here, the CPU uncore memory controller related event counter can be the PMC used to implement the CPU uncore memory controller related event counter function; the number of bytes refers to the number of bytes of physical memory read by all memory controllers in the CPU node.
[0117] S304: The CPU PMU indicator acquisition module determines the first data based on the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline and the number of loaded instructions. The first data includes at least one of the first duration and the second duration.
[0118] In one possible implementation, after obtaining the total duration of CPU pipeline stalls and the number of loaded instructions, the CPU PMU metric acquisition module performs metric aggregation at the CPU node granularity, dividing the total duration of CPU pipeline stalls by the number of loaded instructions to obtain the first duration. The first duration satisfies: mem load SPI = mem load instruction stalls / mem load instruction counts;
[0119] Here, mem load SPI represents the first duration, mem load instruction stalls represents the total duration of CPU pipeline stalls, and mem load instruction counts represents the number of load instructions. To reduce calculation errors, the total duration of CPU pipeline stalls and the number of load instructions need to be collected simultaneously.
[0120] In another possible implementation, after obtaining the total CPU pipeline duration and the number of loaded instructions, the CPU PMU metric acquisition module performs CPU node-level metric aggregation, dividing the total CPU pipeline duration by the number of loaded instructions to obtain the second duration. The second duration satisfies: mem load CPI = mem load instruction cycles / mem load instruction counts;
[0121] Here, mem load CPI represents the second duration, mem load instruction cycles represents the total CPU pipeline duration, and mem load instruction counts represents the number of loaded instructions. To reduce calculation errors, the total CPU pipeline duration and the number of loaded instructions need to be collected simultaneously.
[0122] In another possible implementation, after the CPU PMU metric acquisition module obtains the total duration of CPU pipeline pauses, the total duration of CPU pipeline, and the number of loaded instructions, it performs metric aggregation at the CPU node level. It divides the total duration of CPU pipeline pauses by the number of loaded instructions to obtain the first duration, and divides the total duration of CPU pipeline by the number of loaded instructions to obtain the second duration.
[0123] S305: The CPU PMU indicator acquisition module determines the first bandwidth based on the number of bytes and the preset measurement interval.
[0124] The preset measurement interval is a pre-configured value and is not a raw indicator of the PMU.
[0125] Specifically, the CPU PMU metric acquisition module divides the number of bytes by a preset measurement interval to obtain the first bandwidth. The first bandwidth satisfies: mem load BW = mem read bytes counts / interval;
[0126] Where mem load BW represents the first bandwidth, mem read bytes counts represents the number of bytes, and interval represents the preset measurement interval.
[0127] S306: The CPU PMU indicator acquisition module sends the first data and the first bandwidth to the memory CE storm fault determination module.
[0128] In one possible implementation, the CPU PMU metric acquisition module deployed in NFVI can report the comprehensive PMU metrics, such as the first duration and / or the second duration and the first bandwidth, to the memory CE storm fault determination module deployed in VIM through the external interface between NFVI and VIM.
[0129] In another possible implementation, the CPU PMU metric acquisition module deployed in NFVI can report the comprehensive PMU metrics, such as the first duration and / or the second duration and the first bandwidth, to the memory CE storm fault determination module deployed in NFVI through an internal interface.
[0130] Among them, the first duration, the second duration, and the first bandwidth, these PMU comprehensive indicators, can be transmitted simultaneously or separately, and this application does not limit them in this regard.
[0131] S307: The memory CE storm fault determination module determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the first data and the first bandwidth.
[0132] Specifically, the memory CE storm fault determination module can determine whether a CE storm fault has occurred in the local memory corresponding to a CPU node in the following three ways:
[0133] Method 1: Determine whether the first duration and first bandwidth meet preset conditions. Specifically, if the first duration is greater than or equal to the first threshold within a preset time period and the first bandwidth is less than or equal to the third threshold within a preset time period, it is determined that a CE storm failure has occurred in the local memory; if the first duration is less than the first threshold within a preset time period, or the first bandwidth is greater than the third threshold within a preset time period, it is determined that a CE storm failure has not occurred in the local memory.
[0134] Here, the preset time period is pre-set; the first duration and the first bandwidth can be the results of the current measurement cycle; the first threshold can be n times the result of the previous measurement cycle, and the third threshold can be m times the result of the previous measurement cycle, where n is greater than or equal to 1, and m is greater than 0 and less than or equal to 1. Alternatively, the first threshold can also be n times the average of the results of the previous few measurement cycles, and the third threshold can also be m times the average of the results of the previous few measurement cycles. This application does not limit these possibilities. Furthermore, the current measurement cycle and the previous / previous measurement cycles can be continuous or discontinuous; the previous few measurement cycles can also be continuous or discontinuous. This application does not limit these possibilities either.
[0135] For example, the preset time period is 3 minutes, the current measurement period is time period 1, and the previous measurement period is time period 0; if the first duration measured in time period 1 is greater than or equal to n times the first duration measured in time period 0 within 3 minutes, and the first bandwidth measured in time period 1 is less than or equal to m times the first bandwidth measured in time period 0 within 3 minutes, it indicates that the result of time period 1 has changed abruptly compared to the result of time period 0, and it can be determined that a CE storm failure occurred in the local memory corresponding to the CPU node in time period 1.
[0136] For example, if the preset time period is 3 minutes, the current measurement period is time period 2, and the previous measurement periods of the current measurement period include time period 0 and time period 1; if the first duration measured in time period 2 is greater than or equal to n times the average of the first duration measured in time period 0 and time period 1 within 3 minutes, and the first bandwidth measured in time period 2 is less than or equal to m times the average of the first bandwidth measured in time period 0 and time period 1 within 3 minutes, it indicates that the result of time period 2 has a sudden change compared to the average of the results of time period 0 and time period 1, and it can be determined that a CE storm failure has occurred in the local memory corresponding to the CPU node in time period 2.
[0137] For example, if the preset time period is 3 minutes, the current measurement period is time period 5, and the previous measurement periods of the current measurement period include time period 0 and time period 1; if the first duration measured in time period 5 is greater than or equal to n times the average of the first duration measured in time period 0 and time period 1 within 3 minutes, and the first bandwidth measured in time period 5 is less than or equal to m times the average of the first bandwidth measured in time period 0 and time period 1 within 3 minutes, it indicates that the result of time period 5 has a sudden change compared to the average of the results of time period 0 and time period 1, and it can be determined that a CE storm failure occurred in the local memory corresponding to the CPU node in time period 5.
[0138] Method 2: Determine whether the second duration and the first bandwidth meet preset conditions. If the second duration is greater than or equal to the second threshold within the preset time period, and the first bandwidth is less than or equal to the third threshold within the preset time period, it is determined that a CE storm failure has occurred in the local memory. If the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that a CE storm failure has not occurred in the local memory.
[0139] Here, the preset time period is pre-set. The second duration and the first bandwidth can be the results of the current measurement cycle. The second threshold can be k times the result of the previous measurement cycle, and the third threshold can be m times the result of the previous measurement cycle, where k is greater than or equal to 1, and m is greater than 0 and less than or equal to 1. Alternatively, the second threshold can also be k times the average of the results of the previous few measurement cycles, and the third threshold can also be m times the average of the results of the previous few measurement cycles. This application does not limit this. Furthermore, the current measurement cycle and the previous / previous measurement cycles can be continuous or discontinuous, and the previous few measurement cycles can also be continuous or discontinuous. This application does not limit this either.
[0140] Method 3: Determine whether the first duration, second duration, and first bandwidth meet preset conditions. If the first duration is greater than or equal to the first threshold within the preset time period, the second duration is greater than or equal to the second threshold within the preset time period, and the first bandwidth is less than or equal to the third threshold within the preset time period, it is determined that a CE storm failure has occurred in the local memory. If the first duration is less than the first threshold within the preset time period, the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that a CE storm failure has not occurred in the local memory.
[0141] Here, the preset time period is pre-set. The first duration, second duration, and first bandwidth can be the result of the current measurement cycle. The first threshold can be n times the result of the previous measurement cycle, the second threshold can be k times the result of the previous measurement cycle, and the third threshold can be m times the result of the previous measurement cycle, where n is greater than or equal to 1, k is greater than or equal to 1, and m is greater than 0 and less than or equal to 1. Alternatively, the first threshold can also be n times the average of the results of the previous few measurement cycles, the second threshold can also be k times the average of the results of the previous few measurement cycles, and the third threshold can also be m times the average of the results of the previous few measurement cycles. This application does not limit this. Furthermore, the current measurement cycle and the previous / previous measurement cycles can be continuous or discontinuous, and the previous few measurement cycles can also be continuous or discontinuous. This application does not limit this either.
[0142] Optionally, the memory CE storm fault determination module can also determine the single access response speed based on the first duration and / or the second duration in the first data, and determine the batch data transfer rate based on the first bandwidth. Then, the memory CE storm fault determination module determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the single access response speed and the batch data transfer rate.
[0143] Among them, the single access response speed and batch data transfer rate are used to reflect the degree of impact of reading physical memory on CPU performance. The degree of impact of reading physical memory on CPU performance can be used as a basic basis for detecting CE storm failure. In other words, CE storm failure will affect CPU performance.
[0144] For example, the impact of a CE storm on CPU performance is shown in Table 2. The CE storm is categorized into its symptoms and its underlying causes, while CPU performance is divided into micro and macro levels. Specifically, the symptoms of a CE storm include a large number of frequent CE errors and a surge in CPU utilization. The underlying cause of the problem is increased latency in accessing physical memory. The evaluation metric for the impact of memory latency on micro-level CPU performance is the second latency, which serves as indirect evidence of the affected CPU performance. Here, memory latency reflects the response speed of a single access, and will not be elaborated further.
[0145] The essence of a CPU utilization spike is frequent CPU pipeline pauses. The evaluation metric for the impact on micro-level CPU performance and memory latency is the first duration, which is direct evidence of impaired CPU performance. Similarly, the evaluation metric for the impact of numerous frequent CE errors and CPU utilization spikes on macro-level CPU performance and memory bandwidth is the first bandwidth, which is direct evidence of impaired CPU memory performance. Here, memory bandwidth reflects the batch data transfer rate, and the first bandwidth can also be used to exclude interference scenarios such as cross-NUMA access and CPU frequency modulation.
[0146] Table 2
[0147] Optionally, if the memory CE storm fault determination module is deployed in NFVI, it can also determine the first detection result based on the first data and the first bandwidth. Furthermore, the memory CE storm fault determination module can send the first detection result to the VIM via the NFVI-VIM external interface. The first detection result indicates whether a CE storm fault has occurred or not in the local memory corresponding to the CPU node, and the VIM can present it as a device alarm.
[0148] Optionally, the memory CE storm fault determination module can also determine the first detection result based on the first data.
[0149] In one possible approach, the memory CE storm fault determination module can determine whether a CE storm fault has occurred or not in the local memory corresponding to the CPU node by judging whether a preset condition is met for a first duration.
[0150] Specifically, when the first duration is greater than or equal to the first threshold within a preset time period, it is determined that a CE storm failure has occurred in the local memory; when the first duration is less than the first threshold within a preset time period, it is determined that a CE storm failure has not occurred in the local memory.
[0151] In another possible approach, the memory CE storm fault determination module can determine whether a CE storm fault has occurred or not in the local memory corresponding to the CPU node by judging whether the second duration meets the preset conditions.
[0152] Specifically, when the second duration is greater than or equal to the second threshold within the preset time period, it is determined that a CE storm failure has occurred in the local memory; when the second duration is less than the second threshold within the preset time period, it is determined that a CE storm failure has not occurred in the local memory.
[0153] In another possible approach, the memory CE storm fault determination module can determine whether a CE storm fault has occurred or not in the local memory corresponding to the CPU node by judging whether the first duration and the second duration meet preset conditions.
[0154] Specifically, when the first duration is greater than or equal to the first threshold within a preset time period and the second duration is greater than or equal to the second threshold within a preset time period, it is determined that a CE storm failure has occurred in the local memory; when the first duration is less than the first threshold within a preset time period or the second duration is less than the second threshold within a preset time period, it is determined that a CE storm failure has not occurred in the local memory.
[0155] It should be noted that the embodiment shown in Figure 3 calculates the comprehensive PMU index using the CPU PMU index acquisition module. Alternatively, the comprehensive PMU index can also be calculated using the memory CE storm fault determination module.
[0156] As shown in Figure 4, Figure 4 is a flowchart illustrating another communication method provided in an embodiment of this application. The details are as follows.
[0157] In this scenario, the CPU PMU metric acquisition module is used for raw PMU metric acquisition, and the memory CE storm fault determination module is used for comprehensive PMU metric calculation and fault detection and judgment. This allows for the detection of whether a CE storm fault has occurred in the local memory corresponding to the CPU node without relying on hardware alarms or hardware interrupt notifications, thus improving the accuracy of CE storm fault detection. The steps in this embodiment include at least:
[0158] S401: The CPU PMU metrics acquisition module obtains the number of load instructions from the CPU core L3 cache related event counter.
[0159] S402: The CPU PMU metrics acquisition module obtains the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline from the CPU core pipeline-related event counter.
[0160] S403: The CPU PMU metrics acquisition module obtains the number of bytes from the CPU uncore memory controller related event counters.
[0161] The specific implementation methods of steps S401 to S403 are the same as those of steps S301 to S303 in the previous embodiment. You can refer to steps S301 to S303, and they will not be repeated here.
[0162] S404: The number of load commands sent by the CPU PMU indicator acquisition module to the memory CE storm fault determination module, the total duration of CPU pipeline pauses and / or the total CPU pipeline duration and the number of bytes.
[0163] In one possible implementation, the CPU PMU metrics acquisition module deployed in NFVI can report raw PMU metrics such as the number of loaded instructions, the total duration of CPU pipeline pauses and / or the total duration of CPU pipelines and the number of bytes to the memory CE storm fault determination module deployed in VIM through the external interface between NFVI and VIM.
[0164] In another possible implementation, the CPU PMU metrics acquisition module deployed in NFVI can report raw PMU metrics such as the number of loaded instructions, the total duration of CPU pipeline pauses and / or the total duration of CPU pipelines and the number of bytes to the memory CE storm fault determination module deployed in NFVI through an internal interface.
[0165] Among them, the raw PMU metrics such as the number of load instructions, the total CPU pipeline pause time, the total CPU pipeline time, and the number of bytes can be transmitted simultaneously or separately, and this application does not limit this.
[0166] S405: The memory CE storm fault determination module determines first data based on the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline pauses and the number of loaded instructions. The first data includes at least one of the first duration and the second duration.
[0167] S406: The memory CE storm fault determination module determines the first bandwidth based on the number of bytes and the preset measurement interval.
[0168] The memory CE storm fault determination module in steps S405 to S406 is used to execute the various processes involving the CPU PMU indicator acquisition module in steps S304 to S305. The specific implementation method can refer to steps S304 to S305 in the previous embodiment, which will not be repeated here.
[0169] S407: The memory CE storm fault determination module determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the first data and the first bandwidth.
[0170] The specific implementation of step S407 is the same as that of step S307 in the previous embodiment, and can be referred to step S307, which will not be repeated here.
[0171] It should be noted that the embodiments shown in Figures 3 and 4 determine whether a CE storm failure has occurred in the local memory corresponding to the CPU node by judging the comprehensive indicators of the PMU. In addition, the results of other independent dimensions can be superimposed for a comprehensive judgment.
[0172] As shown in Figure 5, Figure 5 is a flowchart illustrating another communication method provided in an embodiment of this application. The details are as follows.
[0173] In this scenario, the CPU PMU indicator acquisition module is used for acquiring raw PMU indicators and calculating comprehensive PMU indicators, the memory CE storm fault determination module is used for fault detection and judgment, and the memory CE storm fault comprehensive determination module is used to overlay the detection results of other independent dimensions for comprehensive judgment, thereby achieving mutual corroboration of features from different dimensions, which helps to improve the credibility of the results. The steps in this embodiment include at least:
[0174] S501: The CPU PMU metrics acquisition module obtains the number of load instructions from the CPU core L3 cache related event counter.
[0175] S502: The CPU PMU metrics acquisition module obtains the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline from the CPU core pipeline-related event counter.
[0176] S503: The CPU PMU metrics acquisition module obtains the number of bytes from the CPU uncore memory controller related event counters.
[0177] S504: The CPU PMU indicator acquisition module determines the first data based on the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline and the number of loaded instructions. The first data includes at least one of the first duration and the second duration.
[0178] S505: The CPU PMU indicator acquisition module determines the first bandwidth based on the number of bytes and the preset measurement interval.
[0179] S506: The CPU PMU indicator acquisition module sends the first data and the first bandwidth to the memory CE storm fault determination module.
[0180] The specific implementation methods of steps S501 to S506 are the same as those of steps S301 to S306 in the above embodiment. You can refer to steps S301 to S306, which will not be repeated here.
[0181] S507: The memory CE storm fault determination module determines the first detection result based on the first data and the first bandwidth.
[0182] Here, the first detection result is used to indicate whether a CE storm failure has occurred or not in the local memory corresponding to the CPU node.
[0183] Specifically, the memory CE storm fault determination module determines whether a CE storm fault has occurred or not in the local memory corresponding to the CPU node by judging whether the first duration and / or the second duration and the first bandwidth meet preset conditions. The specific judgment process is described in step S307, and will not be repeated here.
[0184] Optionally, the memory CE storm fault determination module here can also determine the first detection result based on the first data. S508: The memory CE storm fault determination module sends the first detection result to the memory CE storm fault comprehensive determination module.
[0185] In one possible implementation, the memory CE storm fault determination module deployed in VIM can report the first detection result to the memory CE storm fault comprehensive determination module deployed in VIM through an internal interface.
[0186] In another possible implementation, the memory CE storm fault determination module deployed in NFVI can report the first detection result to the memory CE storm fault comprehensive determination module deployed in VIM through the external interface between NFVI and VIM.
[0187] S509: The memory CE storm fault comprehensive judgment module obtains the first information.
[0188] The first piece of information includes at least one of the following: hardware feature detection results, alarm results of abnormally high host CPU usage, or alarm results of abnormal business KPIs.
[0189] In one possible approach, the memory CE storm fault comprehensive judgment module can obtain hardware feature detection results from the out-of-band BMC or BIOS firmware, and / or obtain alarm results of abnormal increase in host CPU usage from the host CPU utilization monitoring entity, and / or obtain alarm results of abnormal business KPI from the business KPI monitoring entity.
[0190] Specifically, the out-of-band BMC or BIOS firmware can be deployed in the NFVI; the host CPU utilization monitoring entity can be deployed in the NFVI; and the business KPI monitoring entity can be deployed in the VNF.
[0191] S510: The memory CE storm fault comprehensive judgment module determines the second detection result based on the first information.
[0192] Specifically, the memory CE storm fault comprehensive judgment module can comprehensively judge the hardware feature detection results, the alarm results of abnormal increase in host CPU usage, and / or the alarm results of abnormal business KPI, so as to determine whether the local memory corresponding to the CPU node has experienced or not experienced a CE storm fault.
[0193] S511: The memory CE storm fault comprehensive judgment module determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the first detection result and the second detection result.
[0194] Specifically, the memory CE storm fault comprehensive judgment module can overlay the first and second detection results, and determine whether a CE storm fault has occurred in the local memory corresponding to the CPU node by associating the occurrence time with the host where the fault occurred. The specific judgment process mainly includes the following two scenarios:
[0195] Scenario 1: Coverage priority, any fault detected in any dimension is presented as a result. Specifically, when the first detection result is that a CE storm fault has occurred in the local memory, and / or the second detection result is that a CE storm fault has occurred in the local memory, it is determined that a CE storm fault has occurred in the local memory.
[0196] For example, if the first detection result is that the local memory has experienced a CE storm failure, and the second detection result is that the local memory has not experienced a CE storm failure, then it is determined that the local memory corresponding to the CPU node has experienced a CE storm failure.
[0197] Scenario 2: Accuracy takes precedence; failure to detect a fault in any dimension will not be presented as a result. Specifically, if the first detection result is that no CE storm fault has occurred in the local memory, and / or the second detection result is that no CE storm fault has occurred in the local memory, then it is determined that no CE storm fault has occurred in the local memory.
[0198] For example, if the first detection result is that the local memory has experienced a CE storm failure, and the second detection result is that the local memory has not experienced a CE storm failure, then it is determined that the local memory corresponding to the CPU node has not experienced a CE storm failure.
[0199] It should be noted that in the embodiment shown in Figure 5, the PMU comprehensive index is calculated through the CPU PMU index acquisition module before comprehensive judgment. Alternatively, the PMU comprehensive index can also be calculated through the memory CE storm fault determination module.
[0200] As shown in Figure 6, Figure 6 is a flowchart illustrating another communication method provided in an embodiment of this application. The details are as follows.
[0201] In this scenario, the CPU PMU indicator acquisition module is used for acquiring raw PMU indicators, the memory CE storm fault determination module is used for calculating comprehensive PMU indicators and detecting and judging faults, and the memory CE storm fault comprehensive determination module is used to overlay the detection results of other independent dimensions for comprehensive judgment, thereby achieving mutual corroboration of features from different dimensions, which helps to improve the credibility of the results. The steps in this embodiment include at least:
[0202] S601: The CPU PMU indicator acquisition module obtains the number of load instructions from the CPU core L3 cache related event counter.
[0203] S602: The CPU PMU indicator acquisition module obtains the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline from the CPU core pipeline-related event counter.
[0204] S603: The CPU PMU metrics acquisition module obtains the number of bytes from the CPU uncore memory controller related event counters.
[0205] S604: The number of load commands sent by the CPU PMU indicator acquisition module to the memory CE storm fault determination module, the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline pauses and the number of bytes.
[0206] S605: The memory CE storm fault determination module determines first data based on the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline pauses and the number of loaded instructions. The first data includes at least one of the first duration and the second duration.
[0207] S606: The memory CE storm fault determination module determines the first bandwidth based on the number of bytes and the preset measurement interval.
[0208] The specific implementation methods of steps S601 to S606 are the same as those of steps S401 to S406 in the above embodiment. You can refer to steps S401 to S406, and they will not be repeated here.
[0209] S607: The memory CE storm fault determination module determines the first detection result based on the first data and the first bandwidth.
[0210] S608: The memory CE storm fault determination module sends the first detection result to the memory CE storm fault comprehensive determination module.
[0211] S609: The memory CE storm fault comprehensive judgment module obtains the first information.
[0212] S610: The memory CE storm fault comprehensive judgment module determines the second detection result based on the first information.
[0213] S611: The memory CE storm fault comprehensive judgment module determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the first detection result and the second detection result.
[0214] The specific implementation of steps S608 to S611 is the same as that of steps S508 to S511 in the previous embodiment. You can refer to steps S508 to S511, and they will not be repeated here.
[0215] By acquiring the total duration of CPU pipeline pauses and the number of loaded instructions, the first / second entity calculates a first duration, and / or by acquiring the total duration of CPU pipeline pauses and the number of loaded instructions, the first / second entity calculates a second duration. By acquiring the number of bytes read from physical memory by all memory controllers in the CPU node and a preset measurement interval, the first / second entity calculates a first bandwidth. By determining whether the first duration and / or the second duration and the first bandwidth meet preset conditions, the first / second entity can determine whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the PMU comprehensive indicator judgment method. This achieves the ability to detect whether a CE storm fault has occurred in the local memory corresponding to the CPU node without relying on hardware alarms or hardware interrupt notifications, improving the accuracy of CE storm fault detection. In addition, the third entity can also overlay detection results from other dimensions for comprehensive judgment, thereby improving the reliability of the results.
[0216] The methods of the embodiments of this application have been described in detail above. The following is a description of the apparatus provided in the embodiments of this application.
[0217] As shown in Figure 7, which is a schematic diagram of a communication device according to an embodiment of this application, the communication device includes an acquisition module 701 and a processing module 702. The detailed descriptions of each module are as follows.
[0218] The acquisition module 701 is used to acquire first data and first bandwidth. The first data includes at least one of a first duration and a second duration. The first duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by a central processing unit (CPU) node, from issuing a physical memory load request to obtaining a physical memory data response. The second duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by a CPU node, from issuing a physical memory load request to obtaining a physical memory data response. The first bandwidth is the bandwidth of all memory controllers in the CPU node for reading physical memory.
[0219] The processing module 702 is used to determine, based on the first data and the first bandwidth, whether a correctable error CE storm failure has occurred in the local memory corresponding to the CPU node.
[0220] Optionally, the processing module 702 is further configured to determine the single access response speed based on the first data; determine the batch data transfer rate based on the first bandwidth; and determine whether a CE storm failure has occurred in the local memory based on the single access response speed and the batch data transfer rate.
[0221] Optionally, the first data includes a first duration; the processing module 702 is further configured to determine that a CE storm failure has occurred in the local memory when the first duration is greater than or equal to a first threshold within a preset time period and the first bandwidth is less than or equal to a third threshold within a preset time period; and to determine that a CE storm failure has not occurred in the local memory when the first duration is less than the first threshold within a preset time period or the first bandwidth is greater than the third threshold within a preset time period.
[0222] Optionally, the first data includes the second duration; the processing module 702 is further configured to determine that a CE storm failure has occurred in the local memory when the second duration is greater than or equal to the second threshold within a preset time period and the first bandwidth is less than or equal to the third threshold within a preset time period; and to determine that a CE storm failure has not occurred in the local memory when the second duration is less than the second threshold within a preset time period or the first bandwidth is greater than the third threshold within a preset time period.
[0223] Optionally, the first data includes a first duration and a second duration; the processing module 702 is further configured to determine that a CE storm failure has occurred in the local memory when the first duration is greater than or equal to a first threshold within a preset time period, the second duration is greater than or equal to a second threshold within a preset time period, and the first bandwidth is less than or equal to a third threshold within a preset time period; and to determine that a CE storm failure has not occurred in the local memory when the first duration is less than the first threshold within a preset time period, the second duration is less than the second threshold within a preset time period, or the first bandwidth is greater than the third threshold within a preset time period.
[0224] Optionally, the acquisition module 701 is further used to acquire the total duration of CPU pipeline pauses and the number of load instructions during the process of the CPU node executing load instructions multiple times from issuing a physical memory load request to obtaining a physical memory data response; the processing module 702 is further used to divide the total duration of CPU pipeline pauses by the number of load instructions to obtain a first duration.
[0225] Optionally, the acquisition module 701 is further used to acquire the total CPU pipeline time and the number of load instructions during the process of the CPU node executing load instructions multiple times from issuing a physical memory load request to obtaining a physical memory data response; the processing module 702 is further used to divide the total CPU pipeline time by the number of load instructions to obtain a second time.
[0226] Optionally, the acquisition module 701 is further configured to acquire the number of bytes read from physical memory by all memory controllers in the CPU node and the preset measurement interval; the processing module 702 is further configured to divide the number of bytes by the preset measurement interval to obtain the first bandwidth.
[0227] Optionally, the processing module 702 is further configured to determine a first detection result based on the first data and the first bandwidth; determine a second detection result based on the first information, wherein the first information includes at least one of the following: hardware feature detection result, alarm result of abnormal increase in host CPU usage, and alarm result of abnormal business key performance indicator (KPI); and determine whether a CE storm failure has occurred in the local memory based on the first detection result and the second detection result.
[0228] Optionally, the processing module 702 is further configured to determine a first detection result based on the first data; determine a second detection result based on the first information, wherein the first information includes at least one of the following: hardware feature detection result, alarm result of abnormal increase in host CPU usage, and alarm result of abnormal business key performance indicator (KPI); and determine whether a CE storm failure has occurred in the local memory based on the first detection result and the second detection result.
[0229] Optionally, the processing module 702 is further configured to determine that a CE storm failure has occurred in the local memory when the first detection result is that a CE storm failure has occurred in the local memory, and / or the second detection result is that a CE storm failure has occurred in the local memory.
[0230] Optionally, the processing module 702 is further configured to determine that no CE storm failure has occurred in the local memory when the first detection result is that no CE storm failure has occurred in the local memory, and / or the second detection result is that no CE storm failure has occurred in the local memory.
[0231] It should be noted that the implementation of each module can also refer to the corresponding descriptions of the method embodiments shown in Figures 2-6, and execute the methods and functions performed by the first entity, the second entity and / or the third entity in the above embodiments.
[0232] Figure 8 is a schematic diagram of a server provided in an embodiment of this application. The server can be a chip or processing system in a communication system or communication device, and can implement any of the methods and functions in any of the foregoing embodiments.
[0233] As shown in Figure 8, the server includes a processor 801 configured to perform the actions described in the above method embodiments. Optionally, the server also includes a transceiver 802. Optionally, the server also includes a memory 803 configured to store a computer program, which the processor 801 retrieves and runs to control the transceiver 802 to transmit and receive signals. Optionally, the server may also include an antenna configured to transmit uplink data or uplink control signaling output by the transceiver 802 via a wireless signal.
[0234] The processor 801, transceiver 802 and memory 803 can communicate with each other through internal connection paths to transmit control and / or data signals.
[0235] The processor 801 and the memory 803 can be combined into a single processing device, with the processor 801 configured to execute program code stored in the memory 803 to perform the aforementioned functions. In specific implementations, the memory 803 can be integrated into the processor 801 or be independent of the processor 801.
[0236] The transceiver 802 described above can also be referred to as a transceiver unit or transceiver module. The transceiver 802 may include a receiver (or receiver circuit) and a transmitter (or transmitter circuit). The receiver is configured to receive signals, and the transmitter is configured to transmit signals.
[0237] It should be understood that the server shown in Figure 8 is capable of implementing the various processes in the method embodiments shown in Figures 2-6. The operations and / or functions of each module in this server are respectively for implementing the corresponding processes in the above method embodiments. For details, please refer to the descriptions in the above method embodiments; to avoid repetition, detailed descriptions are appropriately omitted here.
[0238] The processor 801 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 801 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, etc. The communication bus 804 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in Figure 8, but this does not indicate that there is only one bus or one type of bus. The communication bus 804 is configured to enable communication between these components. In this embodiment, the transceiver 802 is configured to communicate with other node devices for signaling or data. The memory 803 may include volatile memory, such as nonvolatile random access memory (NVRAM), phase change RAM (PRAM), magnetoresistive RAM (MRAM), etc., and may also include non-volatile memory, such as at least one disk storage device, electrically erasable programmable read-only memory (EEPROM), flash memory devices, such as NOR flash memory or NAND flash memory, semiconductor devices, such as solid-state disks (SSDs), etc. The memory 803 may also be at least one storage device located remotely from the aforementioned processor 801. The memory 803 may also store a set of computer program code or configuration information. The processor 801 may also execute the program stored in the memory 803. The processor 801 may cooperate with the memory 803 and the transceiver 802 to perform any of the methods and functions involved in the above-described embodiments.
[0239] This application also provides a communication system, which includes at least one first entity and at least one second entity, wherein the first entity and the second entity are used to implement the functions involved in any of the above embodiments.
[0240] This application also provides a chip system including a processor for supporting a communication system or communication device to implement the functions involved in any of the above embodiments, such as calculating a first duration, and / or calculating a second duration and calculating a first bandwidth.
[0241] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the method of any one of the embodiments shown in FIG2-FIG6.
[0242] This application also provides a computer-readable medium storing a computer program that, when run on a computer, causes the computer to perform the method of any one of the embodiments shown in Figures 2-6.
[0243] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the communication device, the unit or module within the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0244] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0245] It should be understood that the "and / or" appearing in the embodiments of this application is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.
[0246] It should be understood that in the embodiments of this application, "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.
[0247] It should be understood that the symbol " / " appearing in the embodiments of this application can indicate that the preceding and following objects are in an "or" relationship. Additionally, the symbol " / " can also represent a division sign, i.e., performing a division operation. For example, A / B can mean A divided by B.
[0248] It should be understood that some or all of the steps in the embodiments of this application may be performed. These steps or operations are merely examples. In the embodiments of this application, other operations or variations of various operations may also be performed. Furthermore, the steps may be performed in different orders as presented in the embodiments of this application, and it is not necessary to perform all the operations in the embodiments of this application.
[0249] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. Any modifications, equivalent substitutions, or improvements made within the principles of this application should be included within the scope of protection of this application.
Claims
1. A communication method characterized by comprising: The method comprises: acquiring first data and first bandwidth, the first data comprising at least one of a first time length and a second time length, the first time length being an average of a time length of CPU pipeline stall in a process of a CPU node executing a load instruction once from issuing a physical memory load request to acquiring a physical memory data response, the second time length being an average of a time length of the CPU pipeline in the process of the CPU node executing the load instruction once from issuing the physical memory load request to acquiring the physical memory data response, and the first bandwidth being a bandwidth of all memory controllers in the CPU node reading the physical memory; determining whether a local memory corresponding to the CPU node has a correctable error (CE) storm failure according to the first data and the first bandwidth.
2. The method of claim 1, wherein, The determining whether the local memory corresponding to the CPU node has the CE storm failure according to the first data and the first bandwidth comprises: determining a single access response speed according to the first data; determining a batch data transmission rate according to the first bandwidth; determining whether the local memory has the CE storm failure according to the single access response speed and the batch data transmission rate.
3. The method of claim 1 or 2, wherein, The first data comprises the first time length, and the determining whether the local memory corresponding to the CPU node has the CE storm failure according to the first data and the first bandwidth comprises: when the first time length is greater than or equal to a first threshold value within a preset time period and the first bandwidth is less than or equal to a third threshold value within the preset time period, determining that the local memory has the CE storm failure; when the first time length is less than the first threshold value within the preset time period or the first bandwidth is greater than the third threshold value within the preset time period, determining that the local memory does not have the CE storm failure.
4. The method of claim 1 or 2, wherein, The first data comprises the second time length, and the determining whether the local memory corresponding to the CPU node has the CE storm failure according to the first data and the first bandwidth comprises: when the second time length is greater than or equal to a second threshold value within a preset time period and the first bandwidth is less than or equal to a third threshold value within the preset time period, determining that the local memory has the CE storm failure; when the second time length is less than the second threshold value within the preset time period or the first bandwidth is greater than the third threshold value within a preset time period, determining that the local memory does not have the CE storm failure.
5. The method of claim 1 or 2, wherein, The first time length and the second time length are included in the first data, and the determining whether the local memory corresponding to the CPU node has the CE storm failure according the first data and the first bandwidth comprises: when the first time length is greater than or equal the first threshold value within a preset time period, the second time length is greater than or equal to the second threshold value within the preset time period, and the first bandwidth is less than or equal to the third threshold value within the preset time period, determining that the local memory has the CE storm failure; When the first duration is less than the first threshold within the preset time period, the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that the local memory does not have the CE storm failure.
6. The method according to any one of claims 1 to 5, wherein, The first data includes: The total duration of CPU pipeline stall in the process of sending a physical memory load request to obtaining a physical memory data response and the number of load instructions executed by the CPU node multiple times are obtained. The total duration of CPU pipeline stall is divided by the number of load instructions to obtain the first duration.
7. The method according to any one of claims 1 to 5, wherein The first data includes: The total duration of CPU pipeline stall in the process of sending a physical memory load request to obtaining a physical memory data response and the number of load instructions executed by the CPU node multiple times are obtained. The total duration of CPU pipeline stall is divided by the number of load instructions to obtain the second duration.
8. The method of any one of claims 1-5, wherein, The first bandwidth includes: The number of bytes of physical memory read by all memory controllers in the CPU node and a preset measurement interval are obtained. The number of bytes is divided by the preset measurement interval to obtain the first bandwidth.
9. The method according to any one of claims 1 to 8, wherein, The method further includes: According to the first data and the first bandwidth, a first detection result is determined. According to first information, a second detection result is determined, and the first information includes at least one of a hardware feature detection result, a host CPU occupation abnormally high alarm result, and a service key performance indicator KPI abnormal alarm result. According to the first detection result and the second detection result, it is determined whether the local memory has the CE storm failure.
10. The method of any one of claims 1-8, wherein, The method further includes: According to the first data, a first detection result is determined. According to first information, a second detection result is determined, and the first information includes at least one of a hardware feature detection result, a host CPU occupation abnormally high alarm result, and a service key performance indicator KPI abnormal alarm result. According to the first detection result and the second detection result, it is determined whether the local memory has the CE storm failure.
11. The method of claim 9 or 10, wherein, The method further includes: When the first detection result is that the local memory has the CE storm failure, and / or the second detection result is that the local memory has the CE storm failure, it is determined that the local memory has the CE storm failure.
12. The method of claim 9 or 10, wherein, The method further includes: When the first detection result that the local memory does not have the CE storm failure, and / or the second detection result that the local memory does not have the CE storm failure, it is determined that the local memory does not have the CE storm failure.
13. A communications device, characterized by It includes: acquire a first data and a first bandwidth, the first data comprising at least one of a first duration and a second duration, the first duration being a mean value of a duration of a CPU pipeline stall in a process of issuing a physical memory load request to acquiring a physical memory data response for a single execution of a load instruction by a CPU node, the second duration being a mean value of a duration of a CPU pipeline in the process of issuing the physical memory load request to acquiring the physical memory data response for the single execution of the load instruction by the CPU node, the first bandwidth being a bandwidth of reading the physical memory by all memory controllers in the CPU node; determine, according to the first data and the first bandwidth, whether a CE storm fault occurs in a local memory corresponding to the CPU node.
14. The apparatus of claim 13, wherein the processing module is further configured to determine a single access response speed according to the first data, determine a batch data transmission rate according to the first bandwidth, and determine whether the CE storm fault occurs in the local memory according to the single access response speed and the batch data transmission rate.
15. The apparatus of claim 13 or 14, wherein, the first data comprises the first duration; the processing module is further configured to determine that the CE storm fault has occurred in the local memory when the first duration is greater than or equal to a first threshold value within a preset time period and the first bandwidth is less than or equal to a third threshold value within the preset time period, and determine that the CE storm fault has not occurred in the local memory when the first duration is less than the first threshold value within the preset time period or the first bandwidth is greater than the third threshold value within the preset time period.
16. The apparatus of claim 13 or 14, wherein, the first data comprises the second duration; the processing module is further configured to determine that the CE storm fault has occurred in the first local memory when the second duration is greater than or equal to a second threshold value within a preset time period and the first bandwidth is less than or equal to a third bandwidth within the preset time period, and determine that the CE storm fault has not occurred in the local storage when the second duration is less than the second threshold value within the preset time period or the first bandwidth is greater than the third threshold value within a preset time period.
17. The apparatus of claim 13 or 14, wherein, the first data comprises the first duration and the second duration; the processing module is further configured to determine that the CE storm fault has occurred when the first duration is greater than or equal to a first threshold value within a preset time period, the second duration is greater than or equal to a second threshold value within the preset time period, and the first bandwidth is less than or equal to a third threshold value within the preset time period; and determine that the CE storm fault has not occurred when the first duration is less than the first threshold value within the preset time period, the second duration is less than the second threshold value within the preset time period, or the first bandwidth is greater than the third threshold value within the preset time period.
18. The apparatus of any one of claims 13-17, wherein The obtaining module is further configured to obtain a total duration of CPU pipeline stall in a process from sending a physical memory loading request to obtaining a physical memory data response when the CPU node executes the loading instruction multiple times and a number of the loading instructions. The processing module is further configured to divide the total duration of CPU pipeline stall by the number of the loading instructions to obtain the first duration.
19. The apparatus of any of claims 13-17, wherein The obtaining module is further configured to obtain a total duration of CPU pipeline stall in a process from sending a physical memory loading request to obtaining a physical memory data response when the CPU node executes the loading instruction multiple times and a number of the loading instructions. The processing module is further configured to divide the total duration of CPU pipeline stall by the number of the loading instructions to obtain the second duration.
20. The apparatus of any of claims 13-17, wherein The obtaining module is further configured to obtain a number of bytes of physical memory read by all memory controllers in the CPU node and a preset measurement interval. The processing module is further configured to divide the number of bytes by the preset measurement interval to obtain the first bandwidth.
21. The apparatus of any of claims 13-20, wherein The processing module is further configured to determine a first detection result according to the first data and the first bandwidth, determine a second detection result according to first information, and determine whether the local memory has the CE storm fault according to the first detection result and the second detection result, the first information including at least one of a hardware feature detection result, a host CPU occupation abnormally high alarm result, and a service key performance indicator (KPI) abnormal alarm result.
22. The apparatus of any of claims 13-20, wherein The processing module is further configured to determine a first detection result according to the first data, determine a second detection result according to first information, and determine whether the local memory has the CE storm fault according to the first detection result and the second detection result, the first information including at least one of a hardware feature detection result, a host CPU occupation abnormally high alarm result, and a service key performance indicator (KP1) abnormal alarm result. The computer readable storage medium includes a computer program, when the computer program is run by the processor, the method of any one of claims 1-12 is realized. The computer program product includes a computer program, when the computer program is run on a computer, the computer executes the method of any one of claims 1-12.
23. A computer-readable storage medium, characterized in that, 24. A computer program product, characterised in that,