A communication method, system and apparatus
By acquiring the CPU node's load instruction response time and bandwidth data, and analyzing CPU pipeline stall conditions, the accuracy problem of CE storm fault detection in existing technologies is solved, enabling precise judgment of the impact on CPU performance and improving detection accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUAWEI TECH CO LTD
- Filing Date
- 2024-11-30
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies cannot accurately determine the impact of correctable error (CE) storm failures on CPU performance when detecting server physical memory, leading to possible missed or false detections, and there is a lack of unified standards for preset judgment thresholds.
By acquiring the CPU node's load instruction response time and bandwidth data, analyzing CPU pipeline stalls, and combining this with preset thresholds to determine CE storm faults, accurate detection can be achieved without relying on hardware alarms or interrupt notifications.
It improves the accuracy of CE storm fault detection, enabling more precise judgment of the impact of CE storm faults on CPU performance and reducing false positives and false negatives.
Smart Images

Figure CN122137731A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of communication technology, and in particular to a communication method, system and apparatus. Background Technology
[0002] Traditional telecommunications systems consist of various dedicated hardware devices. With the development of communication technology and the growth of network scale, telecommunications systems are becoming increasingly complex. To reduce complexity, network function virtualization (NFV) technology can be used to migrate the functions of various entities used in the telecommunications network from the current dedicated hardware platform to general-purpose commercial off-the-shelf (COTS) servers. In other words, NFV technology pools and virtualizes infrastructure hardware resources, thereby providing virtual resources for upper-layer applications, decoupling applications from hardware, and enabling each application to quickly add virtual resources to rapidly expand system capacity or quickly reduce virtual resources to shrink system capacity.
[0003] During server operation and maintenance, physical memory may experience correctable error (CE) storms. CE storms can cause increased central processing unit (CPU) utilization and performance degradation, and in severe cases, may affect the normal scheduling of business applications, damage business flow control, and impair key performance indicators (KPIs). Currently, hardware signature detection methods are commonly used to determine whether physical memory has experienced a CE storm. These methods use the out-of-band baseboard management controller (BMC) or basic input / output system (BIOS) firmware to statistically analyze the temporal and / or spatial distribution of CE occurrences in physical memory. When a preset threshold is reached, a CE storm is considered to have occurred. However, this method does not consider the impact of CE errors on CPU performance, or the extent of that impact. Furthermore, the preset threshold is an empirical parameter, which may lead to missed or false positives in practical applications. Summary of the Invention
[0004] This application provides a communication method, system, and apparatus that enables the detection of whether a CE storm fault has occurred in the local memory corresponding to a CPU node without relying on hardware alarms or hardware interrupt notifications.
[0005] In a first aspect, embodiments of this application provide a communication method, including:
[0006] The system acquires first data and first bandwidth. The first data includes at least one of a first duration and a second duration. The first duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by a central processing unit (CPU) node, from issuing a physical memory load request to obtaining a physical memory data response. The second duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by the CPU node, from issuing a physical memory load request to obtaining a physical memory data response. The first bandwidth is the bandwidth used by all memory controllers in the CPU node to read physical memory.
[0007] Based on the first data and the first bandwidth, determine whether a correctable error CE storm failure has occurred in the local memory corresponding to the CPU node.
[0008] By acquiring and analyzing the first data and the first bandwidth, it is helpful to determine whether a CE storm fault has occurred in the local memory corresponding to the CPU node. This enables the detection of whether a CE storm fault has occurred in the local memory corresponding to the CPU node without relying on hardware alarms or hardware interrupt notifications, thereby improving the accuracy of CE storm fault detection.
[0009] In one possible design, the single access response speed is determined based on the first data; the batch data transfer rate is determined based on the first bandwidth; and the CE storm failure of the local memory is determined based on the single access response speed and the batch data transfer rate. Analyzing the first data and the first bandwidth helps determine whether reading physical memory affects CPU performance and the extent of that impact, thereby determining whether a CE storm has occurred in the local memory. This allows for the detection of CE storm failures in the local memory without relying on hardware alarms or hardware interrupt notifications, improving the accuracy of CE storm detection.
[0010] In another possible design, the first data includes the first duration; when the first duration is greater than or equal to a first threshold within a preset time period, and the first bandwidth is less than or equal to a third threshold within the preset time period, it is determined that the local memory has experienced the CE storm failure; when the first duration is less than the first threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that the local memory has not experienced the CE storm failure. By determining whether the first duration and the first bandwidth meet the preset conditions, it is beneficial to determine whether reading physical memory affects CPU performance and the degree of impact, thereby determining whether the local memory has experienced a CE storm failure. This achieves the ability to detect whether the local memory has experienced a CE storm failure without relying on hardware alarms or hardware interrupt notifications, improving the accuracy of CE storm failure detection.
[0011] In another possible design, the first data includes the second duration; when the second duration is greater than or equal to a second threshold within a preset time period, and the first bandwidth is less than or equal to a third threshold within the preset time period, it is determined that the local memory has experienced the CE storm failure; when the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that the local memory has not experienced the CE storm failure. By determining whether the second duration and the first bandwidth meet the preset conditions, it is beneficial to determine whether reading physical memory affects CPU performance and the degree of impact, thereby determining whether the local memory has experienced a CE storm failure. This achieves the ability to detect whether the local memory has experienced a CE storm failure without relying on hardware alarms or hardware interrupt notifications, improving the accuracy of CE storm failure detection.
[0012] In another possible design, the first data includes the first duration and the second duration. When the first duration is greater than or equal to a first threshold within a preset time period, the second duration is greater than or equal to a second threshold within the preset time period, and the first bandwidth is less than or equal to a third threshold within the preset time period, it is determined that the local memory has experienced a CE storm failure. When the first duration is less than the first threshold within the preset time period, the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that the local memory has not experienced a CE storm failure. By determining whether the first duration, second duration, and first bandwidth meet preset conditions, it is beneficial to determine whether reading physical memory affects CPU performance and the degree of impact, thereby determining whether a CE storm failure has occurred in the local memory. This achieves the ability to detect whether a CE storm failure has occurred in the local memory without relying on hardware alarms or hardware interrupt notifications, improving the accuracy of CE storm failure detection.
[0013] In another possible design, the total duration of CPU pipeline pauses during the process of the CPU node executing multiple load instructions, from issuing a physical memory load request to obtaining the physical memory data response, and the number of load instructions are obtained. The total duration of CPU pipeline pauses is divided by the number of load instructions to obtain the first duration. By obtaining the total duration of CPU pipeline pauses and the number of load instructions, it is beneficial to calculate the first duration, thereby determining whether reading physical memory affects CPU performance and the degree of impact.
[0014] In another possible design, the total CPU pipeline time and the number of load instructions are obtained during the process of the CPU node executing multiple load instructions, from issuing a physical memory load request to obtaining the physical memory data response. The total CPU pipeline time is then divided by the number of load instructions to obtain the second time. By obtaining the total CPU pipeline time and the number of load instructions, it is beneficial to calculate the second time, thereby determining whether reading physical memory affects CPU performance and the degree of that impact.
[0015] In another possible design, the number of bytes read from physical memory by all memory controllers in the CPU node and a preset measurement interval are obtained; the number of bytes is divided by the preset measurement interval to obtain the first bandwidth. By obtaining the number of bytes read from physical memory by all memory controllers in the CPU node and the preset measurement interval, it is beneficial to calculate the first bandwidth, thereby determining whether reading physical memory affects CPU performance and the degree of impact.
[0016] In another possible design, a first measurement result is determined based on the first data and the first bandwidth; a second detection result is determined based on first information, wherein the first information includes at least one of hardware feature detection results, alarm results of abnormally high host CPU usage, and alarm results of abnormal business key performance indicators (KPIs); and whether the local memory has experienced the CE storm failure is determined based on the first detection result and the second detection result. A comprehensive judgment by overlaying detection results from other dimensions helps improve the reliability of the results.
[0017] In another possible design, a CE storm failure is determined to have occurred in the local memory when both the first and second detection results indicate that the local memory has experienced a CE storm failure. By combining detection results from other dimensions for a comprehensive judgment, the reliability of the results can be improved.
[0018] In another possible design, when the first detection result is that the local memory has not experienced the CE storm failure, and / or the second detection result is that the local memory has not experienced the CE storm failure, the determination that the local memory has not experienced the CE storm failure is made by comprehensively judging by superimposing the detection results of other dimensions, which is beneficial to improving the reliability of the results.
[0019] In one possible design, the method of the first aspect can be executed by the first entity, or by a module applied to the first entity (e.g., a processor, chip, or chip system), or by a logical node, logical module, or software capable of implementing all or part of the functions of the first entity.
[0020] In another possible design, the method of the first aspect can be executed by the second entity, or by a module applied to the second entity (e.g., a processor, chip, or chip system), or by a logical node, logical module, or software capable of implementing all or part of the functions of the second entity.
[0021] Secondly, embodiments of this application provide a communication device configured to implement the methods and functions described in the first aspect. This communication device is implemented in hardware / software. It includes modules corresponding to the aforementioned functions.
[0022] In one possible design, the communication device can be a first entity, a chip, chip system, or processor that supports the first entity in implementing the above-described method, or a logic node, logic module, or software that can implement all or part of the functions of the first entity.
[0023] In another possible design, the communication device can be a second entity, a chip, chip system, or processor that supports the second entity in implementing the above methods, or a logic node, logic module, or software that can implement all or part of the functions of the second entity.
[0024] Thirdly, embodiments of this application provide a communication device including one or more processors. The one or more processors enable the communication device to implement the methods in any possible design or implementation of the first aspect described above.
[0025] Fourthly, embodiments of this application provide a communication system comprising at least one first entity and at least one second entity. The first entity is configured to acquire the total duration of CPU pipeline pauses and the number of load instructions during the process of a CPU node executing multiple load instructions from issuing a physical memory load request to obtaining a physical memory data response, and divide the total duration of CPU pipeline pauses by the number of load instructions to obtain a first duration; and / or, acquire the total duration of CPU pipeline pauses and the number of load instructions during the process of a CPU node executing multiple load instructions from issuing a physical memory load request to obtaining a physical memory data response, and divide the total duration of CPU pipeline pauses by the number of load instructions to obtain a second duration; and acquire the number of bytes read from physical memory by all memory controllers in the CPU node and a preset measurement interval, and divide the number of bytes by the preset measurement interval to obtain a first bandwidth; the second entity is configured to acquire first data and the first bandwidth from the first entity, the first data including at least one of the first duration and the second duration, and determine whether a CE storm failure has occurred in the local memory corresponding to the CPU node based on the first data and the first bandwidth.
[0026] In one possible design, the communication system may further include at least one third entity, which is used to determine a first measurement result based on first data and a first bandwidth, and to determine a second detection result based on first information, the first information including at least one of hardware feature detection results, alarm results of abnormally high host CPU usage, and alarm results of abnormal business KPIs, and to determine whether a CE storm failure has occurred in the local memory based on the first detection result and the second detection result.
[0027] Fifthly, embodiments of this application provide a communication system comprising at least one first entity and at least one second entity. The first entity is configured to acquire the total duration of CPU pipeline pauses during the process of a CPU node executing multiple load instructions from issuing a physical memory load request to obtaining a physical memory data response, and / or the total duration of CPU pipeline pauses during the process of a CPU node executing multiple load instructions from issuing a physical memory load request to obtaining a physical memory data response, the number of load instructions, the number of bytes read from physical memory by all memory controllers in the CPU node, and a preset measurement interval. The second entity is configured to acquire the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline pauses, the number of load instructions, the number of bytes, and the preset measurement interval from the first entity; divide the total duration of CPU pipeline pauses by the number of load instructions to obtain a first duration; and / or divide the total duration of CPU pipeline pauses by the number of load instructions to obtain a second duration; and divide the number of bytes by the preset measurement interval to obtain a first bandwidth; and determine whether a CE storm failure has occurred in the local memory corresponding to the CPU node based on at least one of the first duration and the second duration and the first bandwidth.
[0028] In one possible design, the communication system may further include at least one third entity, which is used to determine a first measurement result based on first data and a first bandwidth, and to determine a second detection result based on first information, the first information including at least one of hardware feature detection results, alarm results of abnormally high host CPU usage, and alarm results of abnormal business KPIs, and to determine whether a CE storm failure has occurred in the local memory based on the first detection result and the second detection result.
[0029] In a sixth aspect, embodiments of this application provide a computer-readable storage medium for storing a computer program that, when executed, causes the method described in any one of the first aspects to be implemented.
[0030] In a seventh aspect, embodiments of this application provide a computer program product containing instructions that, when executed, cause the method described in any one of the first aspects to be implemented.
[0031] Eighthly, embodiments of this application provide a chip including a processor and a communication interface for communicating with external or internal devices, the processor enabling the chip to implement the methods described above.
[0032] In one possible design, the chip may further include a memory storing computer programs or instructions, which the processor executes, either from the stored computer programs or instructions or derived from other programs or instructions. When the computer program or instructions are executed, the processor causes the chip to implement the methods described above.
[0033] In another possible design, the chip can be integrated into a communication system or communication device. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.
[0035] Figure 1 This is a schematic diagram of an NFV system.
[0036] Figure 2 This is a flowchart illustrating a communication method provided in an embodiment of this application;
[0037] Figure 3 This is a flowchart illustrating another communication method provided in an embodiment of this application;
[0038] Figure 4 This is a flowchart illustrating another communication method provided in an embodiment of this application;
[0039] Figure 5 This is a flowchart illustrating another communication method provided in an embodiment of this application;
[0040] Figure 6 This is a flowchart illustrating another communication method provided in an embodiment of this application;
[0041] Figure 7 This is a schematic diagram of the structure of a communication device provided in an embodiment of this application;
[0042] Figure 8 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation
[0043] The following explanations of some of the terms used in this application are provided to facilitate understanding by those skilled in the art.
[0044] (1) Network function virtualization (NFV) technology: refers to migrating the functions of various entities used in telecommunications networks from the current dedicated hardware platform to general-purpose commercial-off-the-shelf (COTS) servers.
[0045] (2) Correctable error (CE) storm failure: refers to a large number of correctable CE errors appearing in the physical memory of the server in a short period of time, causing the central processing unit (CPU) to increase its utilization rate and decrease its performance. In severe cases, it affects the normal scheduling of business programs and may cause damage to business flow control and KPIs.
[0046] (3) Non-uniform memory access (NUMA) architecture: This is a processor and memory topology that allows each processor node to have a portion of local memory and a relatively short access latency with the processor on that node. Accessing the memory of other nodes across nodes will result in a large latency, a phenomenon known as "non-uniform access".
[0047] The embodiments of this application are described below with reference to the accompanying drawings.
[0048] It should be understood that in the description of this application, "at least one" means one or more, and "multiple" means two or more. In addition, the words "first," "second," etc., unless otherwise stated, are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance or order.
[0049] It should be understood that in the description of this application, the indication includes direct indication (also known as explicit indication) and implicit indication. Direct indication information A refers to information A being included; implicit indication information A refers to information A being indicated through the correspondence between information A and information B, and the direct indication information B. The correspondence between information A and information B can be predefined, pre-stored, pre-burned, or pre-configured.
[0050] It should be understood that, in the description of this application, information C is used to determine information D, including both situations where information D is determined solely based on information C and situations where it is determined based on information C and other information. Furthermore, information C can also be used to determine information D indirectly, for example, where information D is determined based on information E, and information E is determined based on information C.
[0051] Furthermore, in this application, "Entity A sends message A to entity B" can be understood as meaning that the destination of message A or an intermediate entity in the transmission path between the destination and entity B is entity B, which may include sending the message directly or indirectly to entity B. Similarly, "Entity B receives message A from entity A" can be understood as meaning that the source of message A or an intermediate entity in the transmission path between the source and entity A is entity A, which may include receiving the message directly or indirectly from entity A. The message may undergo necessary processing between the source and destination, such as format changes, but the destination can understand a valid message from the source. Similar expressions in this application can be interpreted in a similar way and will not be elaborated further here.
[0052] The technical solutions provided in this application can be applied to various communication systems, such as 5G or new radio (NR) systems, long term evolution (LTE) systems, LTE frequency division duplex (FDD) systems, LTE time division duplex (TDD) systems, wireless local area network (WLAN) systems, satellite communication systems, future communication systems, or integrated systems of multiple systems. The technical solutions provided in this application can also be applied to device-to-device (D2D) communication, vehicle-to-everything (V2X) communication, machine-to-machine (M2M) communication, machine-type communication (MTC), and Internet of Things (IoT) communication systems, or other communication systems.
[0053] In a communication system, an entity can send signals to or receive signals from another entity. Signals can include information, signaling, or data; an entity can also be replaced by a network element, network entity, device, communication equipment, communication module, node, or communication node, etc. This disclosure uses an entity as an example for description.
[0054] Traditional telecommunications systems consist of various dedicated hardware devices, with different applications using different hardware. With the development of communication technologies and the growth of network scale, telecommunications systems have become increasingly complex, bringing numerous challenges, such as the development and launch of new services, system operation and maintenance, and resource utilization. To reduce complexity, NFV technology can be used to transform the various entities used in the telecommunications network into independent applications. These applications can be flexibly deployed on a unified infrastructure platform built on standard servers, storage, switches, and other devices. Furthermore, through virtualization technology, infrastructure hardware resources can be pooled and virtualized, providing virtual resources to upper-layer applications, achieving application-hardware decoupling. This allows each application to quickly add virtual resources to rapidly expand system capacity or quickly reduce virtual resources to shrink system capacity, greatly improving network resilience. In addition, by using general-purpose COTS servers to form a shared resource pool, newly developed services do not require separate hardware deployment, significantly shortening the time to market for new services.
[0055] The foundation of NFV technology includes cloud computing and virtualization technologies. General-purpose COTS (Compute-on-Demand) hardware devices such as computing, storage, and networking equipment can be decomposed into various virtual resources for use by upper-layer applications through virtualization. Virtualization technology decouples applications from hardware, significantly increasing the speed of virtual resource provisioning; cloud computing technology enables elastic scaling of applications, matching virtual resources with business loads, thus improving not only the utilization efficiency of virtual resources but also the system's response speed.
[0056] like Figure 1 As shown, Figure 1 This is a schematic diagram of an NFV system architecture. This NFV system can be used in various networks, such as a data center network, carrier network, or local area network. The NFV system includes an NFV management and orchestration system (NFV MANO) 101, an NFV infrastructure layer (NFVI) 102, multiple virtual network functions (VNFs) 103, multiple element management (EM) 104, and an operation-support system / business support system (OSS / BSS) 105. Detailed descriptions of each module are as follows.
[0057] NFV MANO 101: Used to perform monitoring and management of NFVI 102 and VNF 103.
[0058] The NFV MANO 101 includes an NFV orchestrator (NFVO), one or more VNF managers (VNFMs), and a virtualized infrastructure manager (VIM). Details are as follows:
[0059] (1) NFVO: It can implement network services (such as L2 and L3 VPN services) on NFVI102, and can also execute resource-related requests from one or more VNFMs, send configuration information to VNFMs, and collect status information of VNF 103. NFVO can communicate with VIM to allocate and / or reserve resources and exchange configuration and status information of virtualization hardware resources.
[0060] (2) VNFM: It can perform various management functions. For example, it can manage one or more VNF 103s, including but not limited to instantiation, updating, querying, scaling and / or terminating VNF 103s.
[0061] (3) VIM: It can perform resource management functions. For example, it manages the allocation of infrastructure resources (e.g., adding resources to virtual containers) and operational functions (e.g., collecting NFVI fault information). VNFM and VIM can communicate with each other to allocate resources and exchange configuration and status information of virtualized hardware resources.
[0062] NFVI 102: Includes hardware resources, software resources, or a combination of both to complete the deployment of a virtualized environment. That is to say, the hardware resource layer and the virtualization layer are used to provide virtualization resources, such as virtual machines and other forms of virtual containers, for VNF 103.
[0063] NFVI 102 comprises a virtual resource layer, a virtualization layer, and a hardware resource layer. The virtual resource layer includes virtual computing, virtual storage, and virtual networking. The hardware resource layer includes computing hardware, storage hardware, and network hardware. Details are as follows:
[0064] (1) Virtual computing and virtual storage: can be provided to VNF103 in the form of virtual machines and / or other virtual containers. For example, one or more VNF 103s can be deployed on a virtual machine.
[0065] (2) Virtual network: can include virtual switch, which is used to provide connection between virtual machines and other virtual machines.
[0066] (3) Virtualization layer: It can abstract hardware resources from the physical layer and decouple VNF 103 to provide virtualization resources to VNF 103. The virtualization layer can also abstract network hardware to form a virtual network.
[0067] (4) Computing hardware: This can be commercially available hardware and / or user-customized hardware used to provide processing and computing resources.
[0068] (5) Storage hardware: This can be storage capacity provided within the network or storage capacity residing in the storage hardware itself (local storage located within the server).
[0069] (6) Network hardware: The transport network in the network hardware can be virtualized using a centralized control plane and a separate forwarding plane (e.g., software-defined networking, SDN).
[0070] Optionally, computing and storage hardware resources can also be pooled together.
[0071] Optionally, the network hardware may also be switches, routers, and / or any other network devices configured to have switching capabilities.
[0072] Optionally, the network hardware can also span multiple domains and may include multiple networks interconnected by one or more transport networks.
[0073] VNF 103: Can be configured to virtualize at least one network function performed through a physical network device.
[0074] For example, VNF 103 can be configured to provide the functions available to different entities in an Internet Protocol Multimedia Subsystem (IMS) network, such as proxy-call session control function (P-CSCF), serving-call session control function (S-CSCF), or network functions of a home subscriber server (HSS).
[0075] EM 104: With configuration, one or more VNF103s can be managed.
[0076] VNFM can interact with VNF 103 and EM 104 to manage the lifecycle of VNF 103 and exchange configuration and status information.
[0077] OSS / BSS105 mainly consists of network management, system management, billing, sales, accounting and customer service, and the systems are organically integrated through a unified information bus.
[0078] It should be noted that, Figure 1 This is merely an illustrative structural diagram of an NFV system; in practical applications, it can be modified according to specific circumstances. Figure 1 The NFV system will undergo corresponding transformations. Furthermore, it should be noted that the technical solution provided in this application can also be applied to... Figure 1 The NFV system shown includes modules such as NFVI, VIM, and / or VNFM.
[0079] During server (or NFV system) maintenance, physical memory may experience a CE storm failure. A CE storm failure can cause increased CPU utilization and decreased performance, severely impacting the normal scheduling of business applications and potentially causing damage to business traffic control and KPIs. When such failures occur, due to a lack of effective detection methods and direct evidence, the root cause often remains unidentified for a considerable period.
[0080] Currently, hardware feature detection is mainly used to determine whether physical memory has experienced a CE storm. Hardware feature detection involves statistically analyzing the temporal and / or spatial distribution of CE errors in physical memory using an out-of-band BMC or BIOS firmware program. When a preset threshold is reached, a CE storm is considered to have occurred. However, hardware feature detection has the following drawbacks: (1) It simply counts the distribution of CE errors without considering whether these CE errors affect CPU performance or the extent of that impact; (2) The preset threshold is set based on expert experience and there is no unified industry standard. In practical applications, the threshold may be set too strictly or too leniently, leading to missed or incorrect detections.
[0081] In addition, there are some improvement methods. For example, adding a judgment on increased CPU utilization to the hardware feature detection conditions can reduce false positives to some extent. However, there are many reasons for increased CPU utilization (e.g., sudden business operations, human operation, software program errors (bugs) etc.), and logically, these possible causes cannot be ruled out.
[0082] To address the aforementioned technical problems, the embodiments of this application provide the following solutions.
[0083] like Figure 2 As shown, Figure 2 This is a flowchart illustrating a communication method provided in an embodiment of this application. The communication method includes, but is not limited to, the following steps:
[0084] S201: Obtain first data and first bandwidth. The first data includes at least one of a first duration and a second duration. The first duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by a CPU node, from issuing a physical memory load request to obtaining a physical memory data response. The second duration is the average duration of CPU pipeline execution during a single execution of a load instruction by a CPU node, from issuing a physical memory load request to obtaining a physical memory data response. The first bandwidth is the bandwidth of all memory controllers in the CPU node reading physical memory.
[0085] The units for the first and second durations are CPU cycles; the unit for the first bandwidth is bytes per second (byte / s); the first duration, second duration, and first bandwidth are comprehensive indicators of the performance monitoring unit (PMU), which can be calculated from the PMU's raw metrics. In CPU pipeline technology, various instruction-level parallelism techniques (e.g., out-of-order execution, speculative execution, branch prediction, hyper-threading, etc.) are used to improve pipeline efficiency and reduce pauses. When the degree of instruction-level parallelism is low, the first and second durations may be very close; conversely, when the degree of instruction-level parallelism is high, the first duration will be shorter than the second duration because the CPU pipeline can execute other instructions to hide some of the latency in memory accesses.
[0086] In one possible implementation, the first entity / second entity can directly obtain the first duration and / or the second duration, as well as the first bandwidth.
[0087] In another possible implementation, the first entity / second entity can first obtain the total duration of CPU pipeline pauses during the process of multiple executions of load instructions by the CPU node from issuing a physical memory load request to obtaining a physical memory data response, and / or the total duration of CPU pipeline pauses during the process of multiple executions of load instructions by the CPU node from issuing a physical memory load request to obtaining a physical memory data response, the number of load instructions, and the number of bytes read from physical memory by all memory controllers in the CPU node; then, the first entity / second entity divides the total duration of CPU pipeline pauses by the number of load instructions to obtain a first duration, and / or divides the total duration of CPU pipeline pauses by the number of load instructions to obtain a second duration, and divides the number of bytes by a preset measurement interval to obtain a first bandwidth.
[0088] Among them, the total CPU pipeline duration, the number of loaded instructions and the number of bytes are all raw PMU indicators. The preset measurement interval is pre-configured and is not a raw PMU indicator. The unit of the preset measurement interval can be seconds, microseconds, etc., and this application does not limit it.
[0089] It should be noted that each of the PMU raw metrics mentioned above is aggregated and measured on a CPU node basis. A CPU node may include multiple CPU core units, which share several memory controllers.
[0090] For the COTS server in the telecom cloud, the raw PMU metrics mentioned above can be measured using the performance monitoring counter (PMC) used to implement the event counter function.
[0091] For example, the measurement methods for the 10 common CPU generations and corresponding PMU raw metrics of telecom cloud COTS servers are shown in Table 1. Specifically, the architectures of the above 10 CPU generations include x86 and ARM V8. The suppliers for x86 include Intel and Hygon / AMD, and the supplier for ARM V8 includes HiSilicon. The corresponding Intel CPU generations include Xeon SP4 Sapphire Rapids, Xeon SP3 Ice Lake, Xeon SP2 Cascade Lake, Xeon SP1 Skylake, Xeon E5 V4 Broadwell-EP, Xeon E5 V3 Haswell-EP, and Xeon E5 V2 Ivy Bridge-EP. The number of load instructions for these seven CPU generations is measured using LLC-load-misses or sum(node-loads, node-load-misses), and the number of bytes is measured using sum(uncore_imc_# / cas_count_read / )*64B. The total CPU pipeline pause time for the four CPU generations (Xeon SP4 Sapphire Rapids, Xeon SP3 Ice Lake, Xeon SP2 Cascade Lake, and Xeon SP1 Skylake) is measured using cycle_activity.stalls_l3_miss, and the total CPU pipeline time is measured using cycle_activity.cycles_l3_miss. The Xeon E5 V4 Broadwell-EP, Xeon E5 V2 Ivy Bridge-EP, Xeon SP3 Ice Lake, Xeon SP2 Cascade Lake, Xeon SP1 Skylake, Xeon E5 V4 Broadwell-EP, Xeon E5 V3 Ice Lake, Xeon SP2 Cascade Lake, Xeon SP1 Skylake, Xeon SP2 Ice Lake, Xeon SP2 Cascade Lake, Xeon SP2 Ice Lake, Xeon SP3 Ice Lake, Xeon SP2 Cascade Lake, Xeon SP2 Ice ... The total CPU pipeline pause time for the three CPU generations, Haswell-EP, Xeon E5 V2 IvyBridge-EP, was obtained by measuring cycle_activity.stalls_ldm_pending, and the total CPU pipeline time was obtained by measuring cycle_activity.cycles_ldm_pending.
[0092] The corresponding CPU generations for Hygon / AMD include ZEN 1. ZEN 1 does not support measuring the total duration of CPU pipeline pauses. The total CPU pipeline duration and the number of loaded instructions for ZEN 1 are obtained by measuring Average L3 Read Miss Latency (in core clocks), and the number of bytes is obtained by measuring Combined DRAM B / bytes of all channels on an NPS1 node (die). Here, Average L3 Read Miss Latency (in core clocks) refers to L3Event[0xFF0F000000400090]*16 / L3Event[0xFF0F000000401F9A]; and Combined DRAM B / bytes of all channels on an NPS1 node (die) refers to... node(die) refers to (DfEvent[0x0000000000403807]+DfEvent[0x0000000000403847]+DfEvent[0x0000000000403887]+DfEvent[0x00000000004038 C7]+DfEvent[0x0000000100403807]+DfEvent[0x0000000100403847]+DfEvent[0x0000000100403887]+DfEvent[0x00000001004038C7])*64B.
[0093] HiSilicon corresponds to the Kunpeng-920C and Kunpeng-920 CPU generations. The total CPU pipeline duration for Kunpeng-920C and Kunpeng-920 is not supported for measurement. The number of load instructions for these two CPU generations is measured using LL_CACHE_RD(r0036) + LL_CACHE_MISS_RD(r0037), and the number of bytes is measured using sum(hisi_sccl#_ddrc# / flux_rd / )*32B. The total pipeline pause duration for the Kunpeng-920C CPU is measured using MEM_STALL_L2MISS(r7008), and for the Kunpeng-920 CPU, it is measured using MEM_STALL_L2MISS(r7007).
[0094] Table 1
[0095]
[0096]
[0097] In Table 1, the Kunpeng-920 and 920C CPUs did not use the MEM_STALL_L3MISS measurement method to determine the total CPU pipeline pause time; instead, they used the MEM_STALL_L2MISS measurement method as an approximation. Although the MEM_STALL_L3MISS measurement includes L3 cache latency, the latency variation of the L3 cache is negligible compared to the latency variation introduced by the CE storm failure.
[0098] Optionally, for CPU generations not covered in Table 1, the measurement methods for the corresponding PMU raw metrics can be obtained by consulting the technical reference manual (TRM) provided by the CPU vendor.
[0099] S202: Based on the first data and the first bandwidth, determine whether a CE storm failure has occurred in the local memory corresponding to the CPU node.
[0100] In this context, the local memory corresponding to a CPU node can be the local memory of a NUMA node. The CPU core unit and its local memory together form a NUMA node. The CPU prioritizes accessing the local memory of its NUMA node to obtain optimal performance.
[0101] Specifically, the first / second entity determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node in the following three ways: Method 1, determine whether the first duration and the first bandwidth meet the preset conditions; Method 2, determine whether the second duration and the first bandwidth meet the preset conditions; Method 3, determine whether the first duration, the second duration, and the first bandwidth meet the preset conditions.
[0102] The specific process for determining whether a CE storm fault has occurred in the local memory corresponding to a CPU node can be found in [reference needed]. Figures 3-6 The corresponding descriptions in the method embodiments shown are not detailed here.
[0103] In this embodiment, by acquiring the first data and the first bandwidth, the first entity / second entity can determine whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the first duration and / or the second duration and the first bandwidth in the first data. This enables the detection of whether a CE storm fault has occurred in the local memory corresponding to the CPU node without relying on hardware alarms or hardware interrupt notifications, thereby improving the accuracy of CE storm fault detection.
[0104] This application's embodiments use NFVI as the first entity, VIM as the second entity, VNFM as the third entity, a management data analytics function (MDAF) entity, a cloud-network collaborative fault handling system, VIM, OSS / BSS, or a physical infrastructure manager (PIM) as examples. Figures 3-6 The communication method shown is explained below. Figures 3-6 The CPU PMU metric acquisition module, CPU core L3 cache related event counters, CPU core pipeline related event counters, and CPU uncore memory controller related event counters can be deployed in NFVI. The processes involved in these modules can be understood as functions of NFVI. The memory CE storm fault determination module can be deployed in NFVI or VIM. The processes involved in this module can be understood as functions of NFVI or VIM. The memory CE storm fault comprehensive determination module can be deployed in VNFM, MDAF, cloud-network collaborative fault handling system, VIM, OSS / BSS, or PIM. The processes involved in this module can be understood as functions of VNFM, MDAF, cloud-network collaborative fault handling system, VIM, OSS / BSS, or PIM.
[0105] Optionally, the six names—CPU PMU indicator acquisition module, CPU core L3 cache related event counter, CPU core pipeline related event counter, CPU uncore memory controller related event counter, memory CE storm fault determination module, and memory CE storm fault comprehensive determination module—are used as examples and do not constitute a limitation on the embodiments of this application. With the development of communication technology and NFV technology, these six modules may adopt other names. For example, the CPU core L3 cache related event counter can also be described as a register used to implement the CPU core L3 cache related event counting function; the CPU core pipeline related event counter can also be described as a register used to implement the CPU core pipeline related event counting function; and so on, which will not be elaborated here.
[0106] like Figure 3 As shown, Figure 3 This is a flowchart illustrating another communication method provided in an embodiment of this application. The details are as follows.
[0107] In this scenario, the CPU PMU metric acquisition module is used for acquiring raw PMU metrics and calculating comprehensive PMU metrics, while the memory CE storm fault determination module is used for fault detection and judgment. This allows for the detection of whether a CE storm fault has occurred in the local memory corresponding to the CPU node without relying on hardware alarms or hardware interrupt notifications, thus improving the accuracy of CE storm fault detection. The steps in this embodiment include at least:
[0108] S301: The CPU PMU indicator acquisition module obtains the number of load instructions from the CPU core L3 cache related event counter.
[0109] Here, the CPU core L3 cache related event counter can be the PMC used to implement the CPU core L3 cache related event counter function; the number of load instructions can be the L3 cache miss load instruction count.
[0110] S302: The CPU PMU metrics acquisition module obtains the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline from the CPU core pipeline-related event counter.
[0111] Here, the CPU core pipeline-related event counter can be the PMC used to implement the CPU core pipeline-related event counter function; the total CPU pipeline pause time refers to the total CPU pipeline pause time during the process of multiple CPU node executions of load instructions from issuing a physical memory load request to obtaining physical memory data response. The total CPU pipeline pause time can be the L3 cache miss pipeline pause count; the total CPU pipeline time refers to the total CPU pipeline time during the process of multiple CPU node executions of load instructions from issuing a physical memory load request to obtaining physical memory data response. The total CPU pipeline time can be the L3 cache miss pipeline cycle count.
[0112] It should be noted that, since different CPU models support different PMC ranges, for the transmission of the two PMU raw metrics, namely the total duration of CPU pipeline pauses and the total duration of CPU pipeline, the total duration of CPU pipeline pauses is preferred, and the total duration of CPU pipeline is the second choice.
[0113] S303: The CPU PMU metrics acquisition module obtains the number of bytes from the CPU uncore memory controller related event counters.
[0114] Here, the CPU uncore memory controller related event counter can be the PMC used to implement the CPU uncore memory controller related event counter function; the number of bytes refers to the number of bytes of physical memory read by all memory controllers in the CPU node.
[0115] S304: The CPU PMU indicator acquisition module determines the first data based on the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline and the number of loaded instructions. The first data includes at least one of the first duration and the second duration.
[0116] In one possible implementation, after obtaining the total duration of CPU pipeline pauses and the number of loaded instructions, the CPU PMU metric acquisition module performs metric aggregation at the CPU node granularity, dividing the total duration of CPU pipeline pauses by the number of loaded instructions to obtain a first duration. The first duration satisfies the following condition:
[0117] mem load SPI=mem load instruction stalls / mem load instruction counts;
[0118] Here, mem load SPI represents the first duration, mem load instruction stalls represents the total duration of CPU pipeline stalls, and mem load instruction counts represents the number of load instructions. To reduce calculation errors, the total duration of CPU pipeline stalls and the number of load instructions need to be collected simultaneously.
[0119] In another possible implementation, after obtaining the total CPU pipeline duration and the number of loaded instructions, the CPU PMU metric acquisition module performs CPU node-level metric aggregation, dividing the total CPU pipeline duration by the number of loaded instructions to obtain a second duration. This second duration satisfies the following condition:
[0120] mem load CPI=mem load instruction cycles / mem load instruction counts;
[0121] Here, mem load CPI represents the second duration, mem load instruction cycles represents the total CPU pipeline duration, and mem load instruction counts represents the number of loaded instructions. To reduce calculation errors, the total CPU pipeline duration and the number of loaded instructions need to be collected simultaneously.
[0122] In another possible implementation, after the CPU PMU metric acquisition module obtains the total duration of CPU pipeline pauses, the total duration of CPU pipeline, and the number of loaded instructions, it performs metric aggregation at the CPU node level. It divides the total duration of CPU pipeline pauses by the number of loaded instructions to obtain the first duration, and divides the total duration of CPU pipeline by the number of loaded instructions to obtain the second duration.
[0123] S305: The CPU PMU indicator acquisition module determines the first bandwidth based on the number of bytes and the preset measurement interval.
[0124] The preset measurement interval is a pre-configured value and is not a raw PMU indicator.
[0125] Specifically, the CPU PMU metric acquisition module divides the number of bytes by a preset measurement interval to obtain the first bandwidth. The first bandwidth satisfies the following conditions:
[0126] mem load BW=mem read bytes counts / interval;
[0127] Where mem load BW represents the first bandwidth, mem read bytes counts represents the number of bytes, and interval represents the preset measurement interval.
[0128] S306: The CPU PMU indicator acquisition module sends the first data and the first bandwidth to the memory CE storm fault determination module.
[0129] In one possible implementation, the CPU PMU metric acquisition module deployed in NFVI can report the comprehensive PMU metrics, such as the first duration and / or the second duration and the first bandwidth, to the memory CE storm fault determination module deployed in VIM through the external interface between NFVI and VIM.
[0130] In another possible implementation, the CPU PMU metric acquisition module deployed in NFVI can report the comprehensive PMU metrics, such as the first duration and / or the second duration and the first bandwidth, to the memory CE storm fault determination module deployed in NFVI through an internal interface.
[0131] Among them, the first duration, the second duration, and the first bandwidth, these PMU comprehensive indicators, can be transmitted simultaneously or separately, and this application does not limit them in this regard.
[0132] S307: The memory CE storm fault determination module determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the first data and the first bandwidth.
[0133] Specifically, the memory CE storm fault determination module can determine whether a CE storm fault has occurred in the local memory corresponding to a CPU node in the following three ways:
[0134] Method 1: Determine whether the first duration and first bandwidth meet preset conditions. Specifically, if the first duration is greater than or equal to the first threshold within a preset time period and the first bandwidth is less than or equal to the third threshold within a preset time period, it is determined that a CE storm failure has occurred in the local memory; if the first duration is less than the first threshold within a preset time period, or the first bandwidth is greater than the third threshold within a preset time period, it is determined that a CE storm failure has not occurred in the local memory.
[0135] Here, the preset time period is pre-set; the first duration and the first bandwidth can be the results of the current measurement cycle; the first threshold can be n times the result of the previous measurement cycle, and the third threshold can be m times the result of the previous measurement cycle, where n is greater than or equal to 1, and m is greater than 0 and less than or equal to 1. Alternatively, the first threshold can also be n times the average of the results of the previous few measurement cycles, and the third threshold can also be m times the average of the results of the previous few measurement cycles. This application does not limit these possibilities. Furthermore, the current measurement cycle and the previous / previous measurement cycles can be continuous or discontinuous; the previous few measurement cycles can also be continuous or discontinuous. This application does not limit these possibilities either.
[0136] For example, the preset time period is 3 minutes, the current measurement period is time period 1, and the previous measurement period is time period 0; if the first duration measured in time period 1 is greater than or equal to n times the first duration measured in time period 0 within 3 minutes, and the first bandwidth measured in time period 1 is less than or equal to m times the first bandwidth measured in time period 0 within 3 minutes, it indicates that the result of time period 1 has changed abruptly compared to the result of time period 0, and it can be determined that a CE storm failure occurred in the local memory corresponding to the CPU node in time period 1.
[0137] For example, if the preset time period is 3 minutes, the current measurement period is time period 2, and the previous measurement periods of the current measurement period include time period 0 and time period 1; if the first duration measured in time period 2 is greater than or equal to n times the average of the first duration measured in time period 0 and time period 1 within 3 minutes, and the first bandwidth measured in time period 2 is less than or equal to m times the average of the first bandwidth measured in time period 0 and time period 1 within 3 minutes, it indicates that the result of time period 2 has a sudden change compared to the average of the results of time period 0 and time period 1, and it can be determined that a CE storm failure has occurred in the local memory corresponding to the CPU node in time period 2.
[0138] For example, if the preset time period is 3 minutes, the current measurement period is time period 5, and the previous measurement periods of the current measurement period include time period 0 and time period 1; if the first duration measured in time period 5 is greater than or equal to n times the average of the first duration measured in time period 0 and time period 1 within 3 minutes, and the first bandwidth measured in time period 5 is less than or equal to m times the average of the first bandwidth measured in time period 0 and time period 1 within 3 minutes, it indicates that the result of time period 5 has a sudden change compared to the average of the results of time period 0 and time period 1, and it can be determined that a CE storm failure occurred in the local memory corresponding to the CPU node in time period 5.
[0139] Method 2: Determine whether the second duration and the first bandwidth meet preset conditions. If the second duration is greater than or equal to the second threshold within the preset time period, and the first bandwidth is less than or equal to the third threshold within the preset time period, it is determined that a CE storm failure has occurred in the local memory. If the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that a CE storm failure has not occurred in the local memory.
[0140] Here, the preset time period is pre-set. The second duration and the first bandwidth can be the results of the current measurement cycle. The second threshold can be k times the result of the previous measurement cycle, and the third threshold can be m times the result of the previous measurement cycle, where k is greater than or equal to 1, and m is greater than 0 and less than or equal to 1. Alternatively, the second threshold can also be k times the average of the results of the previous few measurement cycles, and the third threshold can also be m times the average of the results of the previous few measurement cycles. This application does not limit this. Furthermore, the current measurement cycle and the previous / previous measurement cycles can be continuous or discontinuous, and the previous few measurement cycles can also be continuous or discontinuous. This application does not limit this either.
[0141] Method 3: Determine whether the first duration, second duration, and first bandwidth meet preset conditions. If the first duration is greater than or equal to the first threshold within the preset time period, the second duration is greater than or equal to the second threshold within the preset time period, and the first bandwidth is less than or equal to the third threshold within the preset time period, it is determined that a CE storm failure has occurred in the local memory. If the first duration is less than the first threshold within the preset time period, the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that a CE storm failure has not occurred in the local memory.
[0142] Here, the preset time period is pre-set. The first duration, second duration, and first bandwidth can be the result of the current measurement cycle. The first threshold can be n times the result of the previous measurement cycle, the second threshold can be k times the result of the previous measurement cycle, and the third threshold can be m times the result of the previous measurement cycle, where n is greater than or equal to 1, k is greater than or equal to 1, and m is greater than 0 and less than or equal to 1. Alternatively, the first threshold can also be n times the average of the results of the previous few measurement cycles, the second threshold can also be k times the average of the results of the previous few measurement cycles, and the third threshold can also be m times the average of the results of the previous few measurement cycles. This application does not limit this. Furthermore, the current measurement cycle and the previous / previous measurement cycles can be continuous or discontinuous, and the previous few measurement cycles can also be continuous or discontinuous. This application does not limit this either.
[0143] Optionally, the memory CE storm fault determination module can also determine the single access response speed based on the first duration and / or the second duration in the first data, and determine the batch data transfer rate based on the first bandwidth. Then, the memory CE storm fault determination module determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the single access response speed and the batch data transfer rate.
[0144] Among them, the single access response speed and batch data transfer rate are used to reflect the degree of impact of reading physical memory on CPU performance. The degree of impact of reading physical memory on CPU performance can be used as a basic basis for detecting CE storm failure. In other words, CE storm failure will affect CPU performance.
[0145] For example, the impact of a CE storm on CPU performance is shown in Table 2. The CE storm is categorized into its symptoms and its underlying causes, while CPU performance is divided into micro and macro levels. Specifically, the symptoms of a CE storm include a large number of frequent CE errors and a surge in CPU utilization. The underlying cause of the problem is increased latency in accessing physical memory. The evaluation metric for the impact of memory latency on micro-level CPU performance is the second latency, which serves as indirect evidence of the affected CPU performance. Here, memory latency reflects the response speed of a single access, and will not be elaborated further.
[0146] The essence of a CPU utilization spike is frequent CPU pipeline pauses. The evaluation metric for the impact on micro-level CPU performance and memory latency is the first duration, which is direct evidence of impaired CPU performance. Similarly, the evaluation metric for the impact of numerous frequent CE errors and CPU utilization spikes on macro-level CPU performance and memory bandwidth is the first bandwidth, which is direct evidence of impaired CPU memory performance. Here, memory bandwidth reflects the batch data transfer rate, and the first bandwidth can also be used to exclude interference scenarios such as cross-NUMA access and CPU frequency modulation.
[0147] Table 2
[0148]
[0149] Optionally, if the memory CE storm fault determination module is deployed in NFVI, it can also determine the first detection result based on the first data and the first bandwidth. Furthermore, the memory CE storm fault determination module can send the first detection result to the VIM via the NFVI-VIM external interface. The first detection result indicates whether a CE storm fault has occurred or not in the local memory corresponding to the CPU node, and the VIM can present it as a device alarm.
[0150] It should be noted that, Figure 3 The illustrated embodiment calculates the comprehensive PMU index using a CPU PMU index acquisition module. Alternatively, the comprehensive PMU index can also be calculated using a memory CE storm fault determination module.
[0151] like Figure 4 As shown, Figure 4 This is a flowchart illustrating another communication method provided in an embodiment of this application. The details are as follows.
[0152] In this scenario, the CPU PMU metric acquisition module is used for raw PMU metric acquisition, and the memory CE storm fault determination module is used for comprehensive PMU metric calculation and fault detection and judgment. This allows for the detection of whether a CE storm fault has occurred in the local memory corresponding to the CPU node without relying on hardware alarms or hardware interrupt notifications, thus improving the accuracy of CE storm fault detection. The steps in this embodiment include at least:
[0153] S401: The CPU PMU metrics acquisition module obtains the number of load instructions from the CPU core L3 cache related event counter.
[0154] S402: The CPU PMU metrics acquisition module obtains the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline from the CPU core pipeline-related event counter.
[0155] S403: The CPU PMU metrics acquisition module obtains the number of bytes from the CPU uncore memory controller related event counters.
[0156] The specific implementation of steps S401 to S403 is the same as that of steps S301 to S303 in the previous embodiment. You can refer to steps S301 to S303, and they will not be repeated here.
[0157] S404: The number of load commands sent by the CPU PMU indicator acquisition module to the memory CE storm fault determination module, the total duration of CPU pipeline pauses and / or the total CPU pipeline duration and the number of bytes.
[0158] In one possible implementation, the CPU PMU metrics acquisition module deployed in NFVI can report raw PMU metrics such as the number of loaded instructions, the total duration of CPU pipeline pauses and / or the total duration of CPU pipelines and the number of bytes to the memory CE storm fault determination module deployed in VIM through the external interface between NFVI and VIM.
[0159] In another possible implementation, the CPU PMU metrics acquisition module deployed in NFVI can report raw PMU metrics such as the number of loaded instructions, the total duration of CPU pipeline pauses and / or the total duration of CPU pipelines and the number of bytes to the memory CE storm fault determination module deployed in NFVI through an internal interface.
[0160] Among them, the raw PMU metrics such as the number of load instructions, the total CPU pipeline pause time, the total CPU pipeline time, and the number of bytes can be transmitted simultaneously or separately, and this application does not limit this.
[0161] S405: The memory CE storm fault determination module determines first data based on the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline pauses and the number of loaded instructions. The first data includes at least one of the first duration and the second duration.
[0162] S406: The memory CE storm fault determination module determines the first bandwidth based on the number of bytes and the preset measurement interval.
[0163] The memory CE storm fault determination module in steps S405 to S406 is used to execute the various processes involving the CPU PMU indicator acquisition module in steps S304 to S305. The specific implementation method can refer to steps S304 to S305 in the previous embodiment, which will not be repeated here.
[0164] S407: The memory CE storm fault determination module determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the first data and the first bandwidth.
[0165] The specific implementation of step S407 is the same as that of step S307 in the previous embodiment, and can be referred to step S307, which will not be repeated here.
[0166] It should be noted that, Figures 3-4 The illustrated embodiment determines whether a CE storm fault has occurred in the local memory corresponding to a CPU node by judging the comprehensive indicators of the PMU. Furthermore, other independent dimension detection results can be superimposed for a comprehensive judgment.
[0167] like Figure 5 As shown, Figure 5 This is a flowchart illustrating another communication method provided in an embodiment of this application. The details are as follows.
[0168] In this scenario, the CPU PMU indicator acquisition module is used for acquiring raw PMU indicators and calculating comprehensive PMU indicators, the memory CE storm fault determination module is used for fault detection and judgment, and the memory CE storm fault comprehensive determination module is used to overlay the detection results of other independent dimensions for comprehensive judgment, thereby achieving mutual corroboration of features from different dimensions, which helps to improve the credibility of the results. The steps in this embodiment include at least:
[0169] S501: The CPU PMU metrics acquisition module obtains the number of load instructions from the CPU core L3 cache related event counter.
[0170] S502: The CPU PMU metrics acquisition module obtains the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline from the CPU core pipeline-related event counter.
[0171] S503: The CPU PMU metrics acquisition module obtains the number of bytes from the CPU uncore memory controller related event counters.
[0172] S504: The CPU PMU indicator acquisition module determines the first data based on the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline and the number of loaded instructions. The first data includes at least one of the first duration and the second duration.
[0173] S505: The CPU PMU indicator acquisition module determines the first bandwidth based on the number of bytes and the preset measurement interval.
[0174] S506: The CPU PMU indicator acquisition module sends the first data and the first bandwidth to the memory CE storm fault determination module.
[0175] The specific implementation methods of steps S501 to S506 are the same as those of steps S301 to S306 in the above embodiment. You can refer to steps S301 to S306, which will not be repeated here.
[0176] S507: The memory CE storm fault determination module determines the first detection result based on the first data and the first bandwidth.
[0177] Here, the first detection result is used to indicate whether a CE storm failure has occurred or not in the local memory corresponding to the CPU node.
[0178] Specifically, the memory CE storm fault determination module determines whether a CE storm fault has occurred or not in the local memory corresponding to the CPU node by judging whether the first duration and / or the second duration and the first bandwidth meet preset conditions. The specific judgment process is described in step S307, and will not be repeated here.
[0179] S508: The memory CE storm fault determination module sends the first detection result to the memory CE storm fault comprehensive determination module.
[0180] In one possible implementation, the memory CE storm fault determination module deployed in VIM can report the first detection result to the memory CE storm fault comprehensive determination module deployed in VIM through an internal interface.
[0181] In another possible implementation, the memory CE storm fault determination module deployed in NFVI can report the first detection result to the memory CE storm fault comprehensive determination module deployed in VIM through the external interface between NFVI and VIM.
[0182] S509: The memory CE storm fault comprehensive judgment module obtains the first information.
[0183] The first piece of information includes at least one of the following: hardware feature detection results, alarm results of abnormally high host CPU usage, or alarm results of abnormal business KPIs.
[0184] In one possible approach, the memory CE storm fault comprehensive judgment module can obtain hardware feature detection results from the out-of-band BMC or BIOS firmware, and / or obtain alarm results of abnormal increase in host CPU usage from the host CPU utilization monitoring entity, and / or obtain alarm results of abnormal business KPI from the business KPI monitoring entity.
[0185] Specifically, the out-of-band BMC or BIOS firmware can be deployed in the NFVI; the host CPU utilization monitoring entity can be deployed in the NFVI; and the business KPI monitoring entity can be deployed in the VNF.
[0186] S510: The memory CE storm fault comprehensive judgment module determines the second detection result based on the first information.
[0187] Specifically, the memory CE storm fault comprehensive judgment module can comprehensively judge the hardware feature detection results, the alarm results of abnormal increase in host CPU usage, and / or the alarm results of abnormal business KPI, so as to determine whether the local memory corresponding to the CPU node has experienced or not experienced a CE storm fault.
[0188] S511: The memory CE storm fault comprehensive judgment module determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the first detection result and the second detection result.
[0189] Specifically, the memory CE storm fault comprehensive judgment module can overlay the first and second detection results, and determine whether a CE storm fault has occurred in the local memory corresponding to the CPU node by associating the occurrence time with the host where the fault occurred. The specific judgment process mainly includes the following two scenarios:
[0190] Scenario 1: Coverage priority, any fault detected in any dimension is presented as a result. Specifically, when the first detection result is that a CE storm fault has occurred in the local memory, and / or the second detection result is that a CE storm fault has occurred in the local memory, it is determined that a CE storm fault has occurred in the local memory.
[0191] For example, if the first detection result is that the local memory has experienced a CE storm failure, and the second detection result is that the local memory has not experienced a CE storm failure, then it is determined that the local memory corresponding to the CPU node has experienced a CE storm failure.
[0192] Scenario 2: Accuracy takes precedence; failure to detect a fault in any dimension will not be presented as a result. Specifically, if the first detection result is that no CE storm fault has occurred in the local memory, and / or the second detection result is that no CE storm fault has occurred in the local memory, then it is determined that no CE storm fault has occurred in the local memory.
[0193] For example, if the first detection result is that the local memory has experienced a CE storm failure, and the second detection result is that the local memory has not experienced a CE storm failure, then it is determined that the local memory corresponding to the CPU node has not experienced a CE storm failure.
[0194] It should be noted that, Figure 5 The illustrated embodiment calculates the comprehensive PMU index using the CPU PMU index acquisition module before making a comprehensive judgment. Alternatively, the comprehensive PMU index can also be calculated using the memory CE storm fault determination module.
[0195] like Figure 6 As shown, Figure 6This is a flowchart illustrating another communication method provided in an embodiment of this application. The details are as follows.
[0196] In this scenario, the CPU PMU indicator acquisition module is used for acquiring raw PMU indicators, the memory CE storm fault determination module is used for calculating comprehensive PMU indicators and detecting and judging faults, and the memory CE storm fault comprehensive determination module is used to overlay the detection results of other independent dimensions for comprehensive judgment, thereby achieving mutual corroboration of features from different dimensions, which helps to improve the credibility of the results. The steps in this embodiment include at least:
[0197] S601: The CPU PMU indicator acquisition module obtains the number of load instructions from the CPU core L3 cache related event counter.
[0198] S602: The CPU PMU indicator acquisition module obtains the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline from the CPU core pipeline-related event counter.
[0199] S603: The CPU PMU metrics acquisition module obtains the number of bytes from the CPU uncore memory controller related event counters.
[0200] S604: The number of load commands sent by the CPU PMU indicator acquisition module to the memory CE storm fault determination module, the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline pauses and the number of bytes.
[0201] S605: The memory CE storm fault determination module determines first data based on the total duration of CPU pipeline pauses and / or the total duration of CPU pipeline pauses and the number of loaded instructions. The first data includes at least one of the first duration and the second duration.
[0202] S606: The memory CE storm fault determination module determines the first bandwidth based on the number of bytes and the preset measurement interval.
[0203] The specific implementation methods of steps S601 to S606 are the same as those of steps S401 to S406 in the above embodiment. You can refer to steps S401 to S406, and they will not be repeated here.
[0204] S607: The memory CE storm fault determination module determines the first detection result based on the first data and the first bandwidth.
[0205] S608: The memory CE storm fault determination module sends the first detection result to the memory CE storm fault comprehensive determination module.
[0206] S609: The memory CE storm fault comprehensive judgment module obtains the first information.
[0207] S610: The memory CE storm fault comprehensive judgment module determines the second detection result based on the first information.
[0208] S611: The memory CE storm fault comprehensive judgment module determines whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the first detection result and the second detection result.
[0209] The specific implementation of steps S608 to S611 is the same as that of steps S508 to S511 in the previous embodiment. You can refer to steps S508 to S511, and they will not be repeated here.
[0210] By acquiring the total duration of CPU pipeline pauses and the number of loaded instructions, the first / second entity calculates a first duration, and / or by acquiring the total duration of CPU pipeline pauses and the number of loaded instructions, the first / second entity calculates a second duration. By acquiring the number of bytes read from physical memory by all memory controllers in the CPU node and a preset measurement interval, the first / second entity calculates a first bandwidth. By determining whether the first duration and / or the second duration and the first bandwidth meet preset conditions, the first / second entity can determine whether a CE storm fault has occurred in the local memory corresponding to the CPU node based on the PMU comprehensive indicator judgment method. This achieves the ability to detect whether a CE storm fault has occurred in the local memory corresponding to the CPU node without relying on hardware alarms or hardware interrupt notifications, improving the accuracy of CE storm fault detection. In addition, the third entity can also overlay detection results from other dimensions for comprehensive judgment, thereby improving the reliability of the results.
[0211] The methods of the embodiments of this application have been described in detail above. The following is a description of the apparatus provided in the embodiments of this application.
[0212] like Figure 7 As shown, Figure 7 This is a schematic diagram of a communication device provided in an embodiment of this application. The communication device includes an acquisition module 701 and a processing module 702. Detailed descriptions of each module are as follows.
[0213] The acquisition module 701 is used to acquire first data and first bandwidth. The first data includes at least one of a first duration and a second duration. The first duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by a central processing unit (CPU) node, from issuing a physical memory load request to obtaining a physical memory data response. The second duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by a CPU node, from issuing a physical memory load request to obtaining a physical memory data response. The first bandwidth is the bandwidth of all memory controllers in the CPU node for reading physical memory.
[0214] The processing module 702 is used to determine, based on the first data and the first bandwidth, whether a correctable error CE storm failure has occurred in the local memory corresponding to the CPU node.
[0215] Optionally, the processing module 702 is further configured to determine the single access response speed based on the first data; determine the batch data transfer rate based on the first bandwidth; and determine whether a CE storm failure has occurred in the local memory based on the single access response speed and the batch data transfer rate.
[0216] Optionally, the first data includes a first duration; the processing module 702 is further configured to determine that a CE storm failure has occurred in the local memory when the first duration is greater than or equal to a first threshold within a preset time period and the first bandwidth is less than or equal to a third threshold within a preset time period; and to determine that a CE storm failure has not occurred in the local memory when the first duration is less than the first threshold within a preset time period or the first bandwidth is greater than the third threshold within a preset time period.
[0217] Optionally, the first data includes the second duration; the processing module 702 is further configured to determine that a CE storm failure has occurred in the local memory when the second duration is greater than or equal to the second threshold within a preset time period and the first bandwidth is less than or equal to the third threshold within a preset time period; and to determine that a CE storm failure has not occurred in the local memory when the second duration is less than the second threshold within a preset time period or the first bandwidth is greater than the third threshold within a preset time period.
[0218] Optionally, the first data includes a first duration and a second duration; the processing module 702 is further configured to determine that a CE storm failure has occurred in the local memory when the first duration is greater than or equal to a first threshold within a preset time period, the second duration is greater than or equal to a second threshold within a preset time period, and the first bandwidth is less than or equal to a third threshold within a preset time period; and to determine that a CE storm failure has not occurred in the local memory when the first duration is less than the first threshold within a preset time period, the second duration is less than the second threshold within a preset time period, or the first bandwidth is greater than the third threshold within a preset time period.
[0219] Optionally, the acquisition module 701 is further used to acquire the total duration of CPU pipeline pauses and the number of load instructions during the process of the CPU node executing load instructions multiple times from issuing a physical memory load request to obtaining a physical memory data response; the processing module 702 is further used to divide the total duration of CPU pipeline pauses by the number of load instructions to obtain a first duration.
[0220] Optionally, the acquisition module 701 is further used to acquire the total CPU pipeline time and the number of load instructions during the process of the CPU node executing load instructions multiple times from issuing a physical memory load request to obtaining a physical memory data response; the processing module 702 is further used to divide the total CPU pipeline time by the number of load instructions to obtain a second time.
[0221] Optionally, the acquisition module 701 is further configured to acquire the number of bytes read from physical memory by all memory controllers in the CPU node and the preset measurement interval; the processing module 702 is further configured to divide the number of bytes by the preset measurement interval to obtain the first bandwidth.
[0222] Optionally, the processing module 702 is further configured to determine a first measurement result based on the first data and the first bandwidth; determine a second detection result based on the first information, wherein the first information includes at least one of the following: hardware feature detection result, alarm result of abnormal increase in host CPU usage, and alarm result of abnormal business key performance indicator (KPI); and determine whether a CE storm failure has occurred in the local memory based on the first detection result and the second detection result.
[0223] Optionally, the processing module 702 is further configured to determine that a CE storm failure has occurred in the local memory when the first detection result is that a CE storm failure has occurred in the local memory, and / or the second detection result is that a CE storm failure has occurred in the local memory.
[0224] Optionally, the processing module 702 is further configured to determine that no CE storm failure has occurred in the local memory when the first detection result is that no CE storm failure has occurred in the local memory, and / or the second detection result is that no CE storm failure has occurred in the local memory.
[0225] It should be noted that the implementation of each module can also be referenced accordingly. Figures 2-6 The corresponding descriptions of the method embodiments shown above describe the methods and functions performed by the first entity, the second entity, and / or the third entity in the above embodiments.
[0226] Figure 8 This is a schematic diagram of a server provided in an embodiment of this application. The server can be a chip or processing system in a communication system or communication device, and can implement any of the methods and functions in any of the foregoing embodiments.
[0227] like Figure 8As shown, the server includes a processor 801 configured to perform the actions described in the above method embodiments. Optionally, the server also includes a transceiver 802. Optionally, the server also includes a memory 803 configured to store a computer program, which the processor 801 retrieves and runs to control the transceiver 802 to transmit and receive signals. Optionally, the server may also include an antenna configured to transmit uplink data or uplink control signaling output by the transceiver 802 via a wireless signal.
[0228] The processor 801, transceiver 802 and memory 803 can communicate with each other through internal connection paths to transmit control and / or data signals.
[0229] The processor 801 and the memory 803 can be combined into a single processing device, with the processor 801 configured to execute program code stored in the memory 803 to perform the aforementioned functions. In specific implementations, the memory 803 can be integrated into the processor 801 or be independent of the processor 801.
[0230] The transceiver 802 described above can also be referred to as a transceiver unit or transceiver module. The transceiver 802 may include a receiver (or receiver circuit) and a transmitter (or transmitter circuit). The receiver is configured to receive signals, and the transmitter is configured to transmit signals.
[0231] It should be understood that Figure 8 The server shown can achieve Figures 2-6 The various processes in the method embodiments shown are illustrated. The operations and / or functions of each module in the server are respectively for implementing the corresponding processes in the above method embodiments. For details, please refer to the descriptions in the above method embodiments; to avoid repetition, detailed descriptions are appropriately omitted here.
[0232] The processor 801 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 801 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, etc. The communication bus 804 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. The communication bus 804 is configured to enable communication between these components. In this embodiment, the transceiver 802 is configured to communicate with other node devices for signaling or data. The memory 803 may include volatile memory, such as nonvolatile random access memory (NVRAM), phase change RAM (PRAM), magnetoresistive RAM (MRAM), etc., and may also include non-volatile memory, such as at least one disk storage device, electrically erasable programmable read-only memory (EEPROM), flash memory devices, such as NOR flash memory or NAND flash memory, and semiconductor devices, such as solid-state disks (SSDs). The memory 803 may also be at least one storage device located remotely from the aforementioned processor 801. The memory 803 may also store a set of computer program code or configuration information. The processor 801 can also execute programs stored in the memory 803. The processor 801 can cooperate with the memory 803 and the transceiver 802 to perform any of the methods and functions involved in the above-mentioned embodiments.
[0233] This application also provides a communication system, which includes at least one first entity and at least one second entity, wherein the first entity and the second entity are used to implement the functions involved in any of the above embodiments.
[0234] This application also provides a chip system including a processor for supporting a communication system or communication device to implement the functions involved in any of the above embodiments, such as calculating a first duration, and / or calculating a second duration and calculating a first bandwidth.
[0235] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform... Figures 2-6 The method of any one of the embodiments shown.
[0236] This application also provides a computer-readable medium storing a computer program that, when run on a computer, causes the computer to perform... Figures 2-6 The method of any one of the embodiments shown.
[0237] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the communication device, the unit or module within the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0238] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0239] It should be understood that the "and / or" appearing in the embodiments of this application is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.
[0240] It should be understood that in the embodiments of this application, "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.
[0241] It should be understood that the symbol " / " appearing in the embodiments of this application can indicate that the preceding and following objects are in an "or" relationship. Additionally, the symbol " / " can also represent a division sign, i.e., performing a division operation. For example, A / B can mean A divided by B.
[0242] It should be understood that some or all of the steps in the embodiments of this application may be performed. These steps or operations are merely examples. In the embodiments of this application, other operations or variations of various operations may also be performed. Furthermore, the steps may be performed in different orders as presented in the embodiments of this application, and it is not necessary to perform all the operations in the embodiments of this application.
[0243] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. Any modifications, equivalent substitutions, or improvements made within the principles of this application should be included within the scope of protection of this application.
Claims
1. A communication method, characterized in that, include: Acquire first data and first bandwidth, wherein the first data includes at least one of a first duration and a second duration, wherein the first duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by a central processing unit (CPU) node from issuing a physical memory load request to obtaining a physical memory data response, and the second duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by the CPU node from issuing a physical memory load request to obtaining a physical memory data response, and the first bandwidth is the bandwidth of all memory controllers in the CPU node for reading physical memory; Based on the first data and the first bandwidth, determine whether a correctable error CE storm failure has occurred in the local memory corresponding to the CPU node.
2. The method as described in claim 1, characterized in that, The step of determining whether a correctable error CE storm failure has occurred in the local memory corresponding to the CPU node based on the first data and the first bandwidth includes: Based on the first data, determine the single access response speed; Determine the batch data transmission rate based on the first bandwidth; Based on the single access response speed and the batch data transfer rate, determine whether the local memory has experienced the CE storm failure.
3. The method as described in claim 1 or 2, characterized in that, The first data includes the first duration; determining whether a correctable error CE storm failure has occurred in the local memory corresponding to the CPU node based on the first data and the first bandwidth includes: When the first duration is greater than or equal to the first threshold within a preset time period, and the first bandwidth is less than or equal to the third threshold within the preset time period, it is determined that the local memory has experienced the CE storm failure. When the first duration is less than the first threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that the local memory has not experienced the CE storm failure.
4. The method as described in claim 1 or 2, characterized in that, The first data includes the second duration; determining whether a correctable error CE storm failure has occurred in the local memory corresponding to the CPU node based on the first data and the first bandwidth includes: When the second duration is greater than or equal to the second threshold within a preset time period, and the first bandwidth is less than or equal to the third threshold within the preset time period, it is determined that the local memory has experienced the CE storm failure. When the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that the local memory has not experienced the CE storm failure.
5. The method as described in claim 1 or 2, characterized in that, The first data includes the first duration and the second duration; the step of determining whether a correctable error CE storm failure has occurred in the local memory corresponding to the CPU node based on the first data and the first bandwidth includes: When the first duration is greater than or equal to the first threshold within a preset time period, the second duration is greater than or equal to the second threshold within the preset time period, and the first bandwidth is less than or equal to the third threshold within the preset time period, it is determined that the local memory has experienced the CE storm failure. When the first duration is less than the first threshold within the preset time period, the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period, it is determined that the local memory has not experienced the CE storm failure.
6. The method according to any one of claims 1-5, characterized in that, The acquisition of the first data includes: The total duration of CPU pipeline pauses and the number of load instructions are obtained during the process of the CPU node executing multiple load instructions, from issuing a physical memory load request to obtaining a physical memory data response. The first duration is obtained by dividing the total duration of the CPU pipeline pause by the number of load instructions.
7. The method according to any one of claims 1-5, characterized in that, The acquisition of the first data includes: The total CPU pipeline time and the number of load instructions are obtained during the process of the CPU node executing load instructions multiple times, from issuing a physical memory load request to obtaining a physical memory data response. The second duration is obtained by dividing the total CPU pipeline duration by the number of load instructions.
8. The method according to any one of claims 1-5, characterized in that, The acquisition of the first bandwidth includes: Obtain the number of bytes of physical memory read by all memory controllers in the CPU node and the preset measurement interval; The first bandwidth is obtained by dividing the number of bytes by the preset measurement interval.
9. The method according to any one of claims 1-8, characterized in that, The method further includes: Based on the first data and the first bandwidth, a first measurement result is determined; Based on the first information, determine the second detection result, wherein the first information includes at least one of the following: hardware feature detection result, alarm result of abnormal increase in host CPU usage, and alarm result of abnormal business key performance indicator (KPI). Based on the first detection result and the second detection result, determine whether the local memory has experienced the CE storm failure.
10. The method as described in claim 9, characterized in that, The method further includes: When the first detection result indicates that the local memory has experienced the CE storm failure, and / or the second detection result indicates that the local memory has experienced the CE storm failure, it is determined that the local memory has experienced the CE storm failure.
11. The method as described in claim 9, characterized in that, The method further includes: When the first detection result is that the local memory has not experienced the CE storm failure, and / or the second detection result is that the local memory has not experienced the CE storm failure, it is determined that the local memory has not experienced the CE storm failure.
12. A communication device, characterized in that, include: An acquisition module is used to acquire first data and a first bandwidth. The first data includes at least one of a first duration and a second duration. The first duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by a central processing unit (CPU) node, from issuing a physical memory load request to obtaining a physical memory data response. The second duration is the average duration of CPU pipeline pauses during a single execution of a load instruction by the CPU node, from issuing a physical memory load request to obtaining a physical memory data response. The first bandwidth is the bandwidth of all memory controllers in the CPU node for reading physical memory. The processing module is used to determine, based on the first data and the first bandwidth, whether a correctable error CE storm failure has occurred in the local memory corresponding to the CPU node.
13. The apparatus as claimed in claim 12, characterized in that, The processing module is further configured to determine the single access response speed based on the first data; determine the batch data transmission rate based on the first bandwidth; and determine whether the local memory has experienced the CE storm failure based on the single access response speed and the batch data transmission rate.
14. The apparatus as claimed in claim 12 or 13, characterized in that, The first data includes the first duration; The processing module is further configured to determine that the local memory has experienced the CE storm failure when the first duration is greater than or equal to the first threshold within a preset time period and the first bandwidth is less than or equal to the third threshold within the preset time period; and to determine that the local memory has not experienced the CE storm failure when the first duration is less than the first threshold within the preset time period or the first bandwidth is greater than the third threshold within the preset time period.
15. The apparatus as claimed in claim 12 or 13, characterized in that, The first data includes the second duration; The processing module is further configured to determine that the local memory has experienced the CE storm failure when the second duration is greater than or equal to the second threshold within a preset time period and the first bandwidth is less than or equal to the third threshold within the preset time period; and to determine that the local memory has not experienced the CE storm failure when the second duration is less than the second threshold within the preset time period or the first bandwidth is greater than the third threshold within the preset time period.
16. The apparatus as claimed in claim 12 or 13, characterized in that, The first data includes the first duration and the second duration; The processing module is further configured to determine that the local memory has experienced the CE storm failure when the first duration is greater than or equal to the first threshold within a preset time period, the second duration is greater than or equal to the second threshold within the preset time period, and the first bandwidth is less than or equal to the third threshold within the preset time period; and to determine that the local memory has not experienced the CE storm failure when the first duration is less than the first threshold within the preset time period, the second duration is less than the second threshold within the preset time period, or the first bandwidth is greater than the third threshold within the preset time period.
17. The apparatus according to any one of claims 12-16, characterized in that, The acquisition module is also used to acquire the total duration of CPU pipeline pauses and the number of loading instructions during the process of the CPU node executing multiple load instructions from issuing a physical memory load request to obtaining a physical memory data response. The processing module is further configured to divide the total duration of the CPU pipeline pause by the number of load instructions to obtain the first duration.
18. The apparatus according to any one of claims 12-16, characterized in that, The acquisition module is also used to acquire the total CPU pipeline time and the number of loading instructions during the process of the CPU node executing multiple load instructions from issuing a physical memory load request to obtaining a physical memory data response. The processing module is further configured to divide the total CPU pipeline duration by the number of load instructions to obtain the second duration.
19. The apparatus according to any one of claims 12-16, characterized in that, The acquisition module is also used to acquire the number of bytes of physical memory read by all memory controllers in the CPU node and the preset measurement interval; The processing module is further configured to divide the number of bytes by the preset measurement interval to obtain the first bandwidth.
20. The apparatus according to any one of claims 12-19, characterized in that, The processing module is further configured to determine a first measurement result based on the first data and the first bandwidth; and to determine a second detection result based on the first information, wherein the first information includes at least one of the following: hardware feature detection result, alarm result of abnormal increase in host CPU usage, and alarm result of abnormal business key performance indicator (KPI). Based on the first detection result and the second detection result, determine whether the local memory has experienced the CE storm failure.
21. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program that, when executed by a processor, causes the method as described in any one of claims 1-11 to be implemented.
22. A computer program product, characterized in that, The computer program product includes a computer program that, when run on a computer, causes the computer to perform the method according to any one of claims 1-11.