Multi-dimensional data collection method for heterogeneous computing-oriented smartnic network card

Abstract

The application provides a multi-dimensional data acquisition method for a SmartNIC network card for heterogeneous computing, and relates to the technical field of network management.The method comprises the following steps: taking the SmartNIC network card as a data hub, actively querying the running state, and obtaining M heterogeneous component state data streams; cross-component alignment segmentation is performed to construct K associated behavior portraits; performance pattern mining is performed to construct a cross-component behavior atlas; after interference pattern recognition, interference root cause quantitative analysis is performed according to the K component call chains to obtain the interference root component and the interference propagation quantitative link; and the closed-loop resource dynamic balancing of the SmartNIC network card is performed.The application solves the technical problem that the prior art usually focuses on the monitoring of a single computing resource, ignores the synergistic effect between multiple computing components in a heterogeneous computing environment, and leads to low resource scheduling and task scheduling efficiency, thereby affecting the overall performance of the system.

Description

Technical Field

[0001] This invention relates to the field of network management technology, and more specifically to a multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing. Background Technology

[0002] SmartNICs (Intelligent Network Interface Cards) are key components in networks, leveraging advanced hardware acceleration technologies and network functions to optimize data transmission, load balancing, and task scheduling. Particularly in heterogeneous computing systems, SmartNICs act as data hubs, enabling real-time monitoring and optimization of data flow and resource allocation among multiple heterogeneous computing components, making them an important tool for system optimization. However, existing technologies still face some challenges in practical applications, primarily in efficiently acquiring, integrating, and optimizing multidimensional data streams, especially in complex multi-component environments.

[0003] Specifically, existing technologies typically focus on monitoring individual computing resources, neglecting the synergistic effects between multiple computing components in a heterogeneous computing environment. Each computing component has different performance characteristics and resource usage patterns, and real-time collection and integration of their status data is a challenge. This deficiency prevents the system from accurately grasping the mutual influence of each computing component during task execution, and from identifying resource competition or performance bottlenecks between components. Ultimately, this leads to inefficient resource and task scheduling, thus affecting the overall system performance. Summary of the Invention

[0004] This application provides a multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing, aiming to solve the technical problem that existing technologies usually focus on monitoring a single computing resource, while ignoring the synergistic effect between multiple computing components in a heterogeneous computing environment, resulting in low efficiency in resource scheduling and task scheduling, and thus affecting the overall system performance.

[0005] This application discloses a multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing. The method includes: using the SmartNIC network card as a data hub, actively querying the running status of M interconnected heterogeneous computing components through a private management interface to obtain M heterogeneous component status data streams; aligning and segmenting the M heterogeneous component status data streams across computing components according to K lifecycle time windows of K task trajectories in historical task logs to construct K related behavior profiles; constructing a cross-component behavior graph by performing performance pattern mining on the K related behavior profiles; identifying interference patterns in the cross-component behavior graph, and performing interference root cause quantification analysis of the interference patterns according to the K component call chains of the K task trajectories to obtain interference root component and interference propagation quantification link; and performing closed-loop resource dynamic balancing of the SmartNIC network card starting from the interference root component and according to the interference propagation quantification link.

[0006] One or more technical solutions provided in this application have at least the following beneficial effects:

[0007] By using the SmartNIC network interface card as a data hub and actively querying the running status of M heterogeneous computing components through a private management interface, the system can obtain real-time status data streams from multiple computing components. This ensures a comprehensive understanding of the task execution status and resource usage across the entire heterogeneous computing environment, thereby enhancing the system's observability and real-time performance. Based on aligning and segmenting the status data streams of the M heterogeneous components across computing components, and using K task trajectories from historical task logs, K related behavioral profiles are constructed. This allows for the combination of status information from multiple computing components, forming a global perspective on task behavior analysis and providing a foundation for further optimization of task scheduling and resource allocation. Furthermore, by mining performance patterns from the K related behavioral profiles, a cross-component behavior graph is constructed. This graph visually displays the execution patterns of tasks across multiple components and the interactions between them. Interrelationship analysis effectively identifies collaboration and bottlenecks between components. By performing interference pattern recognition on cross-component behavior graphs and combining the component call chains of K task trajectories, the root cause of interference is accurately identified. Further, through quantitative analysis of the root cause of interference, the propagation chain and root component of the interference are identified, providing a clear diagnosis for performance issues during task execution and effectively reducing the impact of interference on the system. Starting from the root component of the interference, closed-loop dynamic resource balancing of the SmartNIC network card is performed based on the quantitative link of interference propagation. This can automatically adjust resource allocation according to the interference sources identified during task execution, ensuring that resources are evenly distributed and avoiding performance bottlenecks. The closed-loop mechanism of resource balancing ensures that resource configuration can be adaptively adjusted when interference occurs, preventing the overload of a single component from affecting the execution of the entire task and improving the performance stability of the system.

[0008] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0009] Figure 1 This application provides a schematic flowchart of a multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing.

[0010] Figure 2 This application provides a schematic diagram of the interference root cause quantification analysis process in the multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing, which is provided in the embodiments of this application. Detailed Implementation

[0011] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0012] like Figure 1 As shown in the embodiments of this application, a multi-dimensional data acquisition method for SmartNIC network cards oriented towards heterogeneous computing is provided, the method comprising:

[0013] A100: Uses the SmartNIC network card as a data hub, and actively queries the running status of M interconnected heterogeneous computing components through a private management interface to obtain the status data stream of the M heterogeneous components.

[0014] As the core of data flow, the SmartNIC (Smart Network Interface Card) can exchange data with multiple computing components through its network connection. Using the SmartNIC as a data hub means it is responsible for collecting and managing status data from different computing components. The SmartNIC connects to heterogeneous computing components through a dedicated management interface. This interface, based on a specific protocol or API, allows the SmartNIC to proactively access the operational status of the computing components. The advantage of this proprietary interface is that it provides higher control and data access capabilities, ensuring the real-time nature and accuracy of the data. Proactive status queries are performed, that is, requests are sent periodically or on demand to each heterogeneous computing component to obtain their operational status data. These computing components are different types of processing units, such as CPUs, GPUs, and FPGAs, and therefore their status data also differ, such as temperature, load, and memory usage. The query results are returned in the form of data streams. These data streams contain the operational status of each computing component at a specific point in time. Each data stream corresponds to a heterogeneous component and contains a series of status information.

[0015] A200: Based on the K lifecycle time windows of K task trajectories in the historical task log, the state data streams of the M heterogeneous components are aligned and segmented across computing components to construct K related behavior profiles.

[0016] The system records the execution trajectories of multiple historical tasks. Each historical task generates a lifecycle log from initiation to completion, and the task trajectory includes the state changes of the task across various computing components. The lifecycle of each task trajectory is divided into multiple time windows, which represent the task's behavior in different time periods. For example, if a task's execution spans multiple computing components, it needs to be segmented according to time windows. The division of K lifecycle time windows depends on the specific execution details of the task; different task trajectories have different time windows, which may overlap or be independent. Based on the K task lifecycle time windows, the state data streams of M heterogeneous components are aligned and segmented. Since task execution involves the collaboration of multiple computing components, this step ensures that the state data within each time window correctly corresponds to the task's execution trajectory. For example, if a task executes simultaneously on both CPU and GPU, the state data streams of CPU and GPU need to be aligned and merged within that time window. By analyzing the aligned heterogeneous component state data streams, K related behavioral profiles are constructed. Each related behavioral profile reflects the behavioral characteristics of a task during execution on different computing components, including computational performance, resource consumption, and task execution status.

[0017] A300: By mining performance patterns from the K related behavior profiles, a cross-component behavior graph is constructed.

[0018] Performance pattern mining identifies task execution patterns, including common patterns, abnormal patterns, and performance bottlenecks during task execution. This involves various data mining and machine learning techniques, such as cluster analysis and anomaly detection, with the goal of extracting performance patterns across different computing components from K behavioral profiles. Based on the mined performance patterns, a cross-component behavior graph is built. This graph displays the interaction behaviors and execution relationships between various computing components and can be used to analyze component collaboration patterns during task execution. The graph construction process involves treating computing components as nodes in the graph and representing their interactions as edges, such as data flows and call chains. The weight of each edge represents the strength, latency, or other performance metrics of the interaction between components.

[0019] A400: After performing interference pattern recognition on the cross-component behavior map, based on the K component call chains of the K task trajectories, perform interference root cause quantification analysis of the interference pattern to obtain the interference root component and interference propagation quantification link.

[0020] Interference pattern identification is performed on the constructed cross-component behavior graph. Interference patterns refer to the manifestations of performance degradation or resource contention during task execution due to various reasons. By identifying interference patterns in the graph, potential problems can be identified. These interference patterns involve the mutual influence or abnormal behavior between multiple components; for example, excessive resource consumption in one component leads to performance degradation in other components. The call chain refers to the calling relationships between computational components involved in task execution. Each task trajectory involves multiple components, and the call chain records the interactions and call order of these components. By analyzing the K component call chains in historical task trajectories, it is clear how the task depends on various computational components during execution. Through the call chains, bottlenecks or interference sources in the task execution process can be traced.

[0021] Interference root cause analysis involves in-depth analysis of identified interference patterns to pinpoint the source of the interference. This analysis quantitatively assesses the role of each component in the interference pattern, using metrics such as resource consumption, latency, and errors. In this quantitative analysis, a series of mathematical or statistical models are applied to calculate the influence of each component in the propagation of interference and to identify the root cause component. The root cause component is the computational component that initially causes problems during execution; for example, it might be an overloaded computing resource or a slow-responding component, leading to performance issues or latency during task execution.

[0022] After identifying the root cause component of the interference, the analysis focuses on how the interference propagates to other components. The interference propagation quantification path refers to the path along which the interference travels from the root component to other components. Each propagation path has a quantified propagation information, including the intensity of the interference, its propagation speed, and delay. This path helps identify which components in the entire system are most affected by the interference, as well as the extent of its spread.

[0023] A500: Starting from the interference root cause component, perform closed-loop dynamic resource balancing of the SmartNIC network card according to the interference propagation quantization link.

[0024] Using the root cause component of interference as the starting point for scheduling and resource balancing, locating the source of interference allows for more effective problem-solving and optimized resource allocation. Quantifying the interference propagation path provides a reference for resource balancing, showing how interference propagates from the source component to other components. By analyzing the interference propagation path, the components most in need of resource adjustment can be identified, ensuring that the impact of interference on system performance is minimized. The SmartNIC network interface card (NIC) serves as the core of closed-loop dynamic resource balancing, responsible for real-time resource adjustments based on the interference propagation path. Closed-loop resource balancing means that resource configuration can be automatically adjusted based on real-time analysis results. Dynamic balancing means that the SmartNIC continuously monitors the status of various heterogeneous components and adjusts resource allocation according to the interference propagation path and task execution requirements. Thus, when a component experiences overload or resource contention, the resources of other components can be automatically adjusted to ensure smooth task execution. Specific balancing methods include adjusting the load of computing components, adjusting network bandwidth, and rescheduling tasks.

[0025] Furthermore, such as Figure 2 As shown, after identifying interference patterns in the cross-component behavior graph, the root cause quantification analysis of the interference patterns is performed based on the K component call chains of the K task trajectories to obtain the root cause components of the interference and the interference propagation quantification links. The method includes:

[0026] A410: Traverse the graph edges of the cross-component behavior graph using a preset performance anomaly threshold to identify multiple weighted anomaly edges; A420: Perform spatiotemporal clustering analysis on the multiple weighted anomaly edges to obtain the interference pattern and the corresponding interference time window; A430: Based on the intersection time period of the interference time window and K life cycle time windows, segment K call chain segments from the K component call chains; A440: Starting from the associated components of the multiple weighted anomaly edges, perform reverse causal backtracking on the K call chain segments to locate the interference propagation chain, and take the starting point of the interference propagation chain as the interference root source component; A450: Extract the associated edge weights of the interference propagation chain from the cross-component behavior graph to construct the interference propagation quantization link.

[0027] A pre-defined performance anomaly threshold is used to determine whether the performance of a computing component is abnormal. This threshold can be based on various performance metrics, such as response time, processing load, and memory usage, and can be set based on historical data, industry standards, or experience. By traversing the edges of the cross-component behavior graph, the interactions between components are checked to see if they conform to normal performance patterns. If the performance metrics of certain interactions exceed the pre-defined thresholds, these interactions are marked as abnormal. Weighted abnormal edges refer to interaction paths exhibiting abnormal performance; these abnormal edges may be caused by overload of a computing component, resource contention, or other performance issues.

[0028] Spatiotemporal clustering analysis is performed on multiple weighted anomalous edges. This analysis combines temporal and spatial dimensions to identify performance anomaly patterns occurring between specific time periods and components. Specifically, this clustering method groups anomalous edges by time and location (i.e., the interaction relationships between components), thus identifying which interactions between components repeatedly exhibit anomalous behavior within certain time periods. Through spatiotemporal clustering analysis, interference patterns are identified—specific behavioral patterns where interactions between multiple computing components lead to system performance degradation within a specific time period. These interference patterns represent concentrated manifestations of performance problems, affecting the execution of multiple tasks or propagating among multiple components. After identifying interference patterns, the time periods during which these patterns occur are further analyzed, termed interference time windows. Interference time windows refer to the time range within which performance problems or anomalous interactions occur.

[0029] The interference time window refers to the period during which anomalies occur. K lifecycle time windows refer to the lifecycles of K tasks at different times during execution. By comparing the interference time windows and the lifecycle time windows, we can find their intersection. This intersection period represents the time during which the task execution is affected by interference; that is, the task's execution is impacted by performance issues during this period. Through this intersection analysis, we can more accurately identify the sources and impacts of interference on the task.

[0030] Once the intersection time period is determined, relevant call chain fragments are extracted from the K component call chains based on this time period. A call chain fragment refers to the execution process of a task across multiple computing components within a specific time period. The call chain fragments reflect the behavior and execution trajectory of the task between different components within the interference time window. Segmenting the call chain fragments helps to pinpoint which interactions between components are interfered with during task execution, or which abnormal performance of components affects the smooth execution of the task.

[0031] Reverse causal backtracking refers to starting from the manifestation of the interference pattern and tracing backward to find the root cause of the problem. This process is based on K call chain segments, that is, the execution trajectory of tasks between different computing components. By backtracking, it traces how the interference propagates from some components to others. During the reverse backtracking process, each node in the call chain segment is analyzed to identify how the interference expands through the task execution path, affecting other components. In this way, the starting point and propagation path of the problem can be accurately found. Through reverse backtracking, the interference propagation chain is identified. This is a path that starts from the interference source component and gradually affects other components. This path shows how the interference propagates from the source component to other components, helping to identify how the interference spreads in the system and locate the specific propagation link. Finally, by reverse backtracking, the starting point of the interference propagation chain is found, that is, the component that initially caused the interference, called the interference root component. This component is the source of the performance problem in the entire system, and it directly affects the subsequent propagation of interference and the overall stability of the system.

[0032] Analyzing cross-component behavior graphs, particularly extracting edge weights related to interference propagation chains, helps quantify the impact of interference propagation. These weights represent performance characteristics such as the strength, latency, and resource consumption of interactions between components. Using the extracted edge weights, a quantitative interference propagation chain is constructed, demonstrating how interference propagates through interactions between different components and quantifying the impact strength of each chain. This quantitative interference propagation chain provides a refined understanding of the interference propagation process, including information on key components, interaction relationships, impact range, and intensity. It helps analyze the overall picture of the interference problem and provides important basis for subsequent optimization. Quantification means not only identifying interference propagation chains but also accurately assessing the impact of each chain. Based on the quantitative data, more refined resource adjustments, optimized task scheduling, or isolation of interference sources can be implemented to effectively mitigate the impact of interference.

[0033] Furthermore, spatiotemporal clustering analysis is performed on the multiple weighted anomaly edges to obtain the interference pattern and the corresponding interference time window. The method includes:

[0034] A421: Based on the combination of multiple heterogeneous computing components corresponding to the multiple weighted abnormal edges, extract multiple sets of original task execution time information from the state data streams of the M heterogeneous components; A422: Based on the temporal overlap density of the multiple sets of original task execution time information and the topological proximity of the multiple weighted abnormal edges in the cross-component behavior graph, perform spatiotemporal joint clustering of the multiple weighted abnormal edges to obtain multiple abnormal edge aggregation sets; A423: Based on the heterogeneous computing component composition and component interaction relationship associated with the multiple abnormal edge aggregation sets, match multiple candidate modes in the interference mode feature library; A424: Quantify multiple spatiotemporal cohesion scores based on the composition of the original task execution time information associated with the multiple abnormal edge aggregation sets and the topological distance set of the abnormal edges; A425: Based on the descending order of the multiple spatiotemporal cohesion scores, locate the interference mode in the multiple candidate modes; A426: Based on the set of weighted abnormal edges corresponding to the interference mode, perform task execution union core region localization to obtain the interference time window.

[0035] By associating weighted outliers with their corresponding heterogeneous computing component combinations, this process involves analyzing the heterogeneous computing components involved in these weighted outliers. Each outlier connects the interaction of two or more computing components, which share resources or collaborate to complete tasks. Therefore, the component combinations containing outliers become key analytical targets in the task execution process. For each pair or more pairs of component interactions, task execution time information is extracted from the corresponding heterogeneous component state data stream. This time information includes the task's start time, end time, duration, and execution cycle. The raw task execution time information describes the task's execution on different components, reflecting characteristics such as execution time and resource consumption on each computing component. This time information provides a concrete time-series data foundation for subsequent analysis.

[0036] This analysis examines the temporal overlap density of task execution time information extracted from multiple weighted outlier edges. Temporal overlap density refers to the degree to which multiple tasks execute within the same time period, i.e., whether multiple tasks run simultaneously on the same or adjacent computing components. Task temporal overlap density helps identify tasks with high temporal overlap that may interfere with each other during execution, especially when they use the same resources. Higher overlap density indicates a greater likelihood of resource contention or interference between tasks.

[0037] Topological proximity refers to the proximity of interactions between multiple computational components in a cross-component behavior graph. Simply put, if two components are closely connected in the behavior graph, their topological proximity is high. This analysis examines the topological relationships between multiple anomalous edges to identify which computational components interact more closely in the behavior graph and which components have stronger dependencies on each other when performing tasks.

[0038] By combining temporal overlap density and topological proximity, spatiotemporal joint clustering is performed. This clustering method clusters anomalous edges based on both temporal and spatial dimensions, integrating task execution time and the interaction relationships of computational components to identify component combinations with similar interference patterns. The clustering result is a set of multiple anomalous edge aggregations, each representing a group of component interactions where tasks share similar execution time overlap and topological relationships. Spatiotemporal clustering identifies concentrated regions of interference patterns, providing crucial clues for subsequent interference analysis.

[0039] The process extracts the associated heterogeneous computing components and their interactions from multiple anomalous edge aggregations. Each anomalous edge aggregation reflects a set of components and their interactions. These components can be different types of computing units, and they exhibit certain common behaviors during task execution. A disturbance pattern feature library is used to search for candidate disturbance patterns that match these component combinations and interaction relationships. This library contains different types of known disturbance patterns, each a typical problem pattern summarized through analysis of historical task data and performance logs. By matching the component interaction information from multiple anomalous edge aggregations with patterns in the feature library, candidate patterns similar to the anomalous patterns in the current task execution are identified. These candidate patterns provide potential solutions for subsequent root cause analysis of disturbances.

[0040] Using the raw task execution time information from each anomalous edge cluster, which reflects the execution duration and overlap of tasks across different computing components, tasks within each cluster may overlap in time or exhibit similar execution patterns. Analyzing this task execution time data constructs temporal features associated with each cluster, such as task concurrency and similarity. This information helps quantify the execution efficiency and resource utilization of tasks across different components. Topological distance refers to the distance between components in the cross-component behavior graph. The distance in the topological structure reflects the degree of dependency and closeness of interaction between components; a smaller topological distance indicates closer interaction between two components. By analyzing the anomalous edge topological distance set, the distance between each pair of components in the anomalous interaction path is measured, and these distances are used to further identify the propagation links of interference.

[0041] Spatiotemporal cohesion refers to the spatiotemporal correlation or clustering of task execution, that is, the similarity of tasks within a specific time and space range. Combining task execution time information and topological distance, the spatiotemporal cohesion score of each anomalous edge cluster is quantified. A high spatiotemporal cohesion score indicates that task execution has strong clustering in both time and space, and they may interfere with each other due to shared resources or dependencies.

[0042] After obtaining the spatiotemporal cohesion scores of all anomalous edge clusters, they are sorted in descending order of score. This means that anomalous edge clusters with higher cohesion scores will be ranked higher, indicating that these clusters are more spatially and temporally compact and more likely to cause interference. After sorting the spatiotemporal cohesion scores, the best-matching interference pattern is located from multiple candidate interference patterns based on these scores. This process helps to focus on the most severe interference patterns rather than paying too much attention to less important patterns.

[0043] The system identifies a set of weighted anomalous edges associated with interference patterns. These anomalous edges are component interaction paths identified as interference sources in spatiotemporal clustering analysis. The task execution union core region refers to the area most concentratedly affected by interference during the execution of multiple tasks. By analyzing the intersection of task execution processes, the region with the most severe interference is determined, which is the period and region where resource contention is most intense or component load is excessively high. Finally, interference time windows are extracted from the task execution union core region. The interference time window refers to the period during task execution where the interference pattern significantly affects task performance.

[0044] Furthermore, based on the K lifecycle time windows of K task trajectories in the historical task logs, the state data streams of the M heterogeneous components are aligned and segmented across computing components to construct K related behavioral profiles. The method includes:

[0045] A210: Define the component call time window boundaries for the K task trajectories to obtain the K lifecycle time windows; A220: After temporally aligning the state data streams of the M heterogeneous components, within the K lifecycle time windows, perform data point indexing and splicing according to the order of the K component call chains to construct K cross-component task data time-series segments; A230: Extract structured features from the K cross-component task data time-series segments to output the K associated behavior profiles.

[0046] We analyze K task trajectories, each describing the execution process of a task across multiple heterogeneous computing components, including its start and end times on each component. The component call time window refers to the time range within which a task executes on each computing component. In this process, based on each task's execution trajectory, the boundaries of the time windows for task execution on different components are defined. The process of defining time windows is based on the task's execution duration and the switching points between components. Based on the call time windows of each component in the task trajectory, a complete lifecycle time window is determined for each task, describing the entire time range from start to finish.

[0047] The state data streams collected from M heterogeneous computing components are time-aligned. Each component's state data stream records its operational status over a period of time, such as load and memory usage. The purpose of time alignment is to ensure that the state data of all components are aligned on the same time scale, that is, to ensure that the state data of different components can correspond to specific points in time within the task's lifecycle. Through alignment, it can be ensured that the state data of all components can be compared and analyzed within the correct time period in subsequent steps.

[0048] K component call chains describe the execution order and dependencies of tasks across different computing components. Based on the component call chain of each task, data point indexing is performed within the lifecycle time window. Data point concatenation means integrating the state data of each component at a specific point in time, ensuring that this data is arranged according to the execution order of the task on different components. After data point concatenation, K cross-component task data time-series segments are constructed, each representing the execution process of a task across multiple components within the lifecycle time window.

[0049] Structured feature extraction is performed on K time-series data segments of cross-component tasks. The goal of this process is to extract features meaningful to task execution behavior from the raw time-series data. These features include the task's execution time on each component, resource usage, frequency of interaction between tasks, latency, etc. The result of structured feature extraction is to transform the raw time-series data into structured feature vectors that are easy to analyze and compare. Finally, based on the extracted features, K related behavioral profiles are output, each representing the execution pattern and behavioral characteristics of a task on different computing components.

[0050] Furthermore, the method involves extracting structured features from the K time-series segments of cross-component task data and outputting the K related behavioral profiles.

[0051] A231: Decompose the first cross-component task data time sequence segment to obtain M single-component behavior data time sequence segments; A232: Perform staged behavior pattern recognition on the M single-component behavior time sequence data segments to obtain M single-component behavior pattern feature vectors; A233: Based on the temporal relationship between the M single-component behavior data time sequence segments, perform temporal correlation feature statistics on the M single-component behavior pattern feature vectors to obtain M component comprehensive feature vectors; A234: Perform behavior pattern vectorization encoding on the M component comprehensive feature vectors to output the first associated behavior profile.

[0052] From the constructed K cross-component task data time series segments, the first cross-component task data time series segment is selected as the current analysis object. This segment is then decomposed into M single-component behavioral data time series segments, where M refers to the number of computing components participating in the task execution. Each single-component time series segment represents the independent behavioral pattern of a computing component during task execution. Each single-component behavioral data time series segment records the specific behavior of that component during task execution, such as execution time, resource consumption, and computational load. This decomposition allows for a shift from a global perspective (cross-component) to a local perspective (single-component), enabling independent analysis of the behavior of each component.

[0053] The process involves identifying phased behavioral patterns for each individual component's time-series behavior. The goal is to divide the behavioral data of each component into several phases and identify the behavioral patterns within each phase. For example, during a certain period, a computing component might exhibit different behavioral characteristics such as high load, resource contention, or idleness. By identifying phased behavioral patterns, the working state of the component is analyzed, such as peak load, idle periods, and peak fluctuations. This helps identify different operating states of the component during task execution. This process often relies on time-series analysis methods, such as clustering and pattern mining, to identify the characteristics of each phase.

[0054] After identifying the phased behavior patterns, a single-component behavior pattern feature vector is generated for each component. This feature vector contains key behavioral characteristics for each phase, such as resource utilization, computational load, response time, and task completion rate. Each single-component behavior pattern feature vector is a numerical representation reflecting the component's performance characteristics at different stages of task execution. These feature vectors transform the component's behavior from time-series data into structured features, which is helpful for subsequent performance analysis, optimization, and anomaly detection.

[0055] Analyzing the temporal relationships between time-series segments of behavioral data from each individual component reveals that these relationships refer to the overlapping and alternating patterns of task execution order and time on different computing components. By identifying these temporal relationships, we can clarify how each computing component collaborates with the others during task execution. When tasks are executed on different components, the components may wait for each other, share resources, or execute in parallel. Analyzing these temporal relationships helps us understand the execution dependencies and interactions of the components in the system.

[0056] Based on these temporal relationships, temporal correlation feature statistics are performed on the feature vectors of the behavioral patterns of M individual components. This process associates the feature vectors of each individual component with the feature vectors of other components, forming a holistic feature representation of multi-component collaborative execution. Temporal correlation feature statistics not only focus on individual component features but also incorporate the interaction and collaboration patterns between components. For example, if the load of component A increases during a certain time period, and the load of component B changes accordingly, this indicates a resource competition or data transmission dependency between them. Through this statistical approach, the temporal collaborative relationships of multiple components can be captured.

[0057] By statistically analyzing temporal correlation features, M comprehensive feature vectors of components are obtained. These vectors contain the temporal dependencies and collaboration features among all components, reflecting the comprehensive behavioral patterns of each component during task execution. These vectors provide the foundation for subsequent overall behavior analysis, performance evaluation, and optimization.

[0058] Behavioral pattern vectorization encoding is performed on the comprehensive feature vectors of M components. Vectorization encoding refers to converting these multi-dimensional feature vectors into numerical codes that reflect the behavioral patterns of each component during task execution. Behavioral pattern vectorization encoding maps temporally correlated feature vectors to a low-dimensional space or uses specific encoding algorithms, such as principal component analysis, to compress complex behavioral data into smaller, representative vectors. These encoded vectors effectively represent the overall execution pattern of the task while reducing redundant information and improving processing efficiency. Finally, the first correlated behavioral profile is output. This profile is a visual or structured representation of the task execution behavior, capturing the comprehensive execution pattern of the task across all participating heterogeneous computing components. This profile provides a clear visual representation of the task's execution process in a multi-component environment, as well as the roles and influences of each component in the collaboration.

[0059] Furthermore, by performing performance pattern mining on the K related behavioral profiles, a cross-component behavioral graph is constructed. The method includes:

[0060] A310: Based on the K related behavior profiles, standard performance features are extracted to obtain K task-level multi-dimensional performance vectors, wherein the task-level multi-dimensional performance vectors cover time-series dimension vector groups, resource dimension vector groups, and interaction dimension vector groups; A320: K-means clustering is used to cluster the component behavior patterns of the K task-level multi-dimensional performance vectors to obtain P task behavior pattern clusters; A330: Using M heterogeneous computing components as topology nodes, directed connections are established between nodes according to the call relationships of the K component call chains to construct an initial component call topology; A340: According to the component interaction rules of the P task behavior pattern clusters, edge relationship weights are assigned to the initial component call topology to complete the construction of the cross-component behavior graph.

[0061] Standard performance features are extracted based on K related behavioral profiles. These profiles are obtained by analyzing the execution process of a task across multiple heterogeneous computing components, including information such as the task's execution mode and the behavior of each computing component. Performance feature extraction extracts key performance indicators (KPIs) from these profiles, such as execution time, resource consumption, and computational load. These features help understand the overall performance of the task. After performance feature extraction, a task-level multi-dimensional performance vector is generated for each task. Each task-level performance vector contains multiple dimensions, reflecting various aspects of task execution.

[0062] The time-series vector group describes the execution of a task within different time periods, including time-series characteristics such as start time, end time, duration, waiting time, and response time. These characteristics help evaluate the task's performance over time, such as whether there is latency or resource contention. The resource-series vector group describes the computing resources consumed by the task during execution and the efficiency of these resources, such as CPU load, memory usage, and GPU usage. These characteristics help understand the task's resource consumption and utilization. The interaction-series vector group describes the interaction patterns of the task between multiple computing components, such as the calling relationships between different components, data transfer volume, and synchronous and parallel execution characteristics. The interaction-series features can reveal the task's behavior in cross-component collaboration, such as dependencies between components and data flow.

[0063] K-means clustering algorithm is used to cluster K task-level multi-dimensional performance vectors. K-means clustering is an unsupervised learning method used to divide a dataset into several clusters. Data within a cluster have high similarity, while there are significant differences between different clusters. After clustering, P task behavior pattern clusters are obtained. Each cluster represents a similar task behavior pattern, and the tasks in these clusters share commonalities in terms of timing, resource usage, and component interaction.

[0064] The M heterogeneous computing components participating in task execution are considered as topological nodes in a graph, with each component representing a node. Task execution and data transmission are manifested through the connections between these components. Based on the call relationships of the K component call chains, the execution order and dependencies of tasks among the computing components are analyzed. The component call chains describe how tasks are executed sequentially across different computing components, reflecting the dependency order of task execution. Based on the call relationships of the K component call chains, directed connections are established between the nodes in the component call topology. Each directed edge represents a relationship where one computing component passes tasks or data to another, and the direction of the edge reflects the order of task execution, i.e., the direction of data or computation flow. Through these directed connections, an initial component call topology is constructed, representing the execution dependencies and data transmission paths of tasks among the various computing components.

[0065] This analysis examines the component interaction patterns of P task behavior pattern clusters. These patterns describe the execution modes and data flow patterns of these tasks among various computing components. For example, some components frequently work together, or a task may depend on a specific computing component at a certain stage. Identifying these interaction patterns helps in understanding the collaborative relationships between components during task execution. Based on these component interaction patterns, the edges in the initial component call topology are weighted. This weighting assigns a weight to each edge, determined by multiple factors such as data transfer volume, computational load, latency, and task dependency. This weighting makes the relationships between components more nuanced and reflects the influence of each edge on system performance. After edge weighting, a cross-component behavior graph is constructed. This graph is a weighted directed graph where each node represents a computing component, and the edge weights represent the interaction strength, data transfer volume, and computational dependency of tasks between components.

[0066] Furthermore, starting from the interference root cause component, and based on the interference propagation quantization link, a closed-loop dynamic resource balancing method is performed on the SmartNIC network card, comprising:

[0067] A510: Calculate resource regulation weights based on the interference propagation quantization link to obtain a resource allocation balance link; A520: Starting from the interference root cause component, perform closed-loop dynamic resource balancing of the SmartNIC network card based on the resource allocation balance link.

[0068] Based on the interference propagation quantification chain, the resource allocation of each component during task execution is weighted. Resource allocation weight refers to the resource adjustment weight allocated to each component in the interference propagation chain. By calculating these weights, the proportion of resources each component should receive can be determined to effectively mitigate the impact of interference. By calculating the resource allocation weights of each component, a resource allocation balance chain is obtained. This chain represents how resources are allocated among components according to the interference propagation chain, making the overall system's resource usage more balanced and preventing some components from affecting the normal execution of other components due to excessive resource consumption.

[0069] Using the root cause component of interference as the starting point for resource balancing, this component is the initial computing component causing the interference. Typically, its excessive load or abnormal performance leads to a decline in the performance of other components in the system. By using this component as the starting point, targeted resource regulation and optimization can be implemented to ensure that the interference does not spread to other components. The resource allocation balancing link provides guidance on how to adjust resources among components. During this process, closed-loop dynamic resource balancing is performed based on information from the resource allocation balancing link. Closed-loop resource balancing means continuously monitoring the status and resource usage of components and dynamically adjusting resource allocation based on real-time data. This closed-loop mechanism can automatically adjust the resources of each component, ensuring that the impact of the interference source is minimized and avoiding performance bottlenecks caused by uneven resource allocation. In the dynamic resource balancing process, the SmartNIC network interface card (NIC) is used as the core of resource scheduling, responsible for coordinating the resource allocation among different computing components. The SmartNIC monitors the resource requirements and performance status of the entire system, adjusting bandwidth, latency, computing resources, etc., to ensure the smooth execution of tasks.

[0070] Furthermore, the method also includes:

[0071] A110: In the process of actively querying and obtaining the original data streams of M heterogeneous components by using the SmartNIC network card as a data hub, the SmartNIC network card is used to collect basic physical index streams using a direct-connect sensor module; A120: The original data streams of the M heterogeneous components are calibrated under observer state perception using the basic physical index streams to obtain the state data streams of the M heterogeneous components.

[0072] The SmartNIC acts as a data hub, actively querying the raw data streams of M computing components within a heterogeneous computing system. These data streams include the operational status, performance data, and task execution information of each computing component. The SmartNIC is equipped with a direct-connect sensor module, which directly collects basic physical metrics related to network transmission. These metrics include: network bandwidth utilization, reflecting bottlenecks or congestion in network transmission; data transmission latency, reflecting response times between tasks and components; the number of lost data packets, indicating network quality issues; and metrics related to network interface or connection quality, such as signal strength or error rate.

[0073] By utilizing the collected basic physical index streams, data calibration is performed on the raw data streams of M heterogeneous components. Since there are differences in performance and network transmission between different computing components, this calibration mechanism ensures the consistency and accuracy of data collected from each component and eliminates interference caused by network quality fluctuations or transmission delays. By combining the network's physical indices with the data streams of each component, the raw data streams are corrected, making the component status reflected in the data streams more accurate.

[0074] Observer state awareness refers to intelligently adjusting data calibration strategies by sensing changes in the state of network and computing components. For example, if network latency is high or bandwidth is insufficient, it indicates a problem with data transmission. Calibration compensates for this impact, ensuring that the state data obtained from computing components is as accurate as possible. This process guarantees that even in unstable network environments, the SmartNIC network card can accurately reflect the true state of each computing component.

[0075] After data calibration, M heterogeneous component status data streams are obtained. These data streams represent the running status of each computing component during task execution, such as CPU load, memory usage, GPU usage, etc. Moreover, these data have been calibrated through network and transmission to reflect more realistic and accurate component performance.

[0076] Furthermore, the comprehensive feature vector of each component includes event causal features, resource competition features, and efficiency coupling features.

[0077] Event causality features refer to the temporal sequence of events and their interrelationships among components during task execution. These features capture the causal relationships in task execution, i.e., how the behavior of one component causes or influences the behavior of other components. Resource contention features describe the situation where multiple components contend for shared resources during task execution. These features reflect the resource contention situation when the task is executed on different components and reveal resource bottlenecks. Efficiency coupling features represent the correlation of the execution efficiency of a task across multiple computing components; that is, whether the efficient execution of a task on some components is coupled with the execution efficiency of other components. Efficiency coupling features reflect the collaborative execution efficiency of the task across different components. By combining event causality features, resource contention features, and efficiency coupling features, a comprehensive feature vector is generated for each computing component. These feature vectors comprehensively describe the component's performance, resource usage, and synergistic effects during task execution, helping to identify potential problem areas and providing a basis for subsequent optimization.

[0078] Furthermore, the comprehensive feature vectors of the M components are subjected to low-dimensional vectorization encoding based on dimensionality reduction feature fusion to output the first associated behavior profile.

[0079] Dimensionality reduction and feature fusion are performed on the comprehensive feature vectors of M components. Dimensionality reduction refers to mapping a high-dimensional feature space to a low-dimensional space, thereby reducing redundant information and retaining the most representative features. Feature fusion means merging multiple features so that the comprehensive feature vector of each component can reflect the overall performance of task execution, rather than a single performance dimension. Through dimensionality reduction and feature fusion, the comprehensive feature vectors of the M components are transformed into low-dimensional vectors. Low-dimensional vectorization encoding retains key features and eliminates redundant information, making the feature vectors of each component more compact in dimensionality while preserving all important performance information. Finally, the first associated behavior profile is output, which is generated based on the dimensionality-reduced low-dimensional vectorization encoding. This profile contains the comprehensive behavioral patterns of all components in task execution, providing a global perspective on the entire task execution process.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing, characterized in that, The method includes: Using the SmartNIC network card as a data hub, the system actively queries the running status of M interconnected heterogeneous computing components through a private management interface to obtain the status data stream of the M heterogeneous components. Based on the K lifecycle time windows of K task trajectories in the historical task log, the state data streams of the M heterogeneous components are aligned and segmented across computing components to construct K related behavior profiles; By performing performance pattern mining on the K related behavior profiles, a cross-component behavior graph is constructed. After identifying interference patterns in the cross-component behavior graph, the root cause analysis of the interference patterns is performed based on the K component call chains of the K task trajectories to obtain the root cause components of the interference and the interference propagation quantification links. The interference mode refers to the manifestation of performance degradation or resource contention problems caused by certain reasons during task execution; The interference pattern recognition of the cross-component behavior map includes: The graph edges of the cross-component behavior graph are traversed using a preset performance anomaly threshold to identify multiple weighted anomaly edges; Spatiotemporal clustering analysis is performed on the multiple weighted abnormal edges to obtain the interference pattern and the corresponding interference time window; The step of performing spatiotemporal clustering analysis on the multiple weighted outlier edges to obtain the interference pattern and the corresponding interference time window includes: Based on the combination of multiple heterogeneous computing components corresponding to the multiple weighted abnormal edges, multiple sets of original task execution time information are extracted from the state data streams of the M heterogeneous components; Based on the temporal overlap density of the multiple sets of original task execution time information and the topological proximity of the multiple weighted abnormal edges in the cross-component behavior graph, spatiotemporal joint clustering of the multiple weighted abnormal edges is performed to obtain multiple abnormal edge aggregation sets. Based on the heterogeneous computing component composition and component interaction relationships associated with the multiple abnormal edge aggregation sets, multiple candidate modes are matched in the interference mode feature library; Multiple spatiotemporal cohesion scores are quantified based on the original task execution time information associated with the multiple abnormal edge aggregation sets and the abnormal edge topological distance sets. Based on the descending order of the multiple spatiotemporal cohesion scores, the interference mode is located among the multiple candidate modes; Based on the set of weighted abnormal edges corresponding to the interference mode, the core area of ​​the task execution union is located to obtain the interference time window; Starting from the interference root cause component, the SmartNIC network card performs closed-loop dynamic resource balancing based on the interference propagation quantization link.

2. The multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing as described in claim 1, characterized in that, After identifying interference patterns in the cross-component behavior graph, the root cause quantification analysis of the interference patterns is performed based on the K component call chains of the K task trajectories to obtain the root cause components of the interference and the interference propagation quantification links. The method includes: Based on the intersection time period of the interference time window and the K life cycle time windows, divide the K component call chains into K call chain segments; Starting from the associated components of the multiple weighted abnormal edges, reverse causal backtracking is performed on the K call chain segments to locate the interference propagation chain, and the starting point of the interference propagation chain is taken as the interference root source component. The associated edge weights of the interference propagation chain are extracted from the cross-component behavior graph to construct the interference propagation quantization link.

3. The multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing as described in claim 1, characterized in that, Based on K lifecycle time windows of K task trajectories in historical task logs, the state data streams of the M heterogeneous components are segmented across computing components to construct K related behavioral profiles. The method includes: The component call time window boundaries are defined for the K task trajectories to obtain the K lifecycle time windows; After aligning the state data streams of the M heterogeneous components in time sequence, within the K life cycle time windows, data point indexing and splicing are performed according to the order of the K component call chains to construct K cross-component task data time sequence segments; Structured features are extracted from the time-series segments of the K cross-component task data, and the K related behavioral profiles are output.

4. The multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing as described in claim 3, characterized in that, The method involves extracting structured features from the K time-series data segments of cross-component tasks and outputting the K related behavioral profiles. Decompose the time sequence segment of the first cross-component task data to obtain M time sequence segments of single-component behavior data; Stage-based behavior pattern recognition is performed on the M single-component behavior time-series data segments to obtain M single-component behavior pattern feature vectors; Based on the temporal relationship between the time-series segments of the M single-component behavioral data, the temporal correlation feature statistics of the feature vectors of the M single-component behavioral patterns are performed to obtain the comprehensive feature vectors of the M components; The behavior pattern vectorization encoding is performed on the comprehensive feature vectors of the M components to output the first associated behavior profile.

5. The multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing as described in claim 1, characterized in that, By performing performance pattern mining on the K related behavior profiles, a cross-component behavior graph is constructed. The method includes: Based on the K related behavior profiles, standard performance features are extracted to obtain K task-level multi-dimensional performance vectors, wherein the task-level multi-dimensional performance vectors cover the time-series dimension vector group, the resource dimension vector group, and the interaction dimension vector group. K-means clustering is used to cluster the component behavior patterns of K task-level multi-dimensional performance vectors to obtain P task behavior pattern clusters; Using M heterogeneous computing components as topology nodes, directed connections are established between nodes based on the calling relationships of the K component calling chains to construct an initial component calling topology; Based on the component interaction rules of the P task behavior pattern clusters, the initial component call topology is weighted by edge relationships to complete the construction of the cross-component behavior graph.

6. The multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing as described in claim 1, characterized in that, Starting from the aforementioned interference root cause component, and based on the interference propagation quantization link, a closed-loop dynamic resource balancing method is performed on the SmartNIC network interface card, comprising: Based on the interference propagation quantization link, resource regulation weights are calculated to obtain a resource allocation balanced link; Starting from the interference root cause component, the SmartNIC network card performs closed-loop dynamic resource balancing based on the resource allocation balancing link.

7. The method for multi-dimensional data acquisition of SmartNIC network cards for heterogeneous computing as described in claim 1, characterized in that, The method further includes: In the process of actively querying and obtaining the original data streams of M heterogeneous components by using the SmartNIC network card as a data hub, the basic physical index stream of the SmartNIC network card is collected by the direct connection sensing module. The original data streams of the M heterogeneous components are calibrated under observer state perception using the basic physical index stream to obtain the state data streams of the M heterogeneous components.

8. The multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing as described in claim 4, characterized in that, Each component's integrated feature vector includes event causal features, resource competition features, and efficiency coupling features.

9. The multi-dimensional data acquisition method for SmartNIC network cards for heterogeneous computing as described in claim 4, characterized in that, The comprehensive feature vectors of the M components are subjected to low-dimensional vectorization encoding based on dimensionality reduction feature fusion to output the first associated behavior profile.

Images