Resource monitoring method and related equipment

Abstract

The invention discloses a resource monitoring method and related equipment, and relates to the technical field of computer monitoring. The method comprises the steps of obtaining a job identifier of a target job in the electronic design automation software and a process identifier of at least one associated process associated with the target job, and establishing a mapping relation table between the target job and the at least one associated process based on the job identifier and the process identifier of the associated process; on the basis of the mapping relation table, monitoring various resource index information corresponding to at least part of the associated processes; and based on the multiple resource index information corresponding to at least part of the associated processes, obtaining resource monitoring information of the target job by establishing an association relationship between the multiple resource index information and the target job. According to the embodiment of the invention, the process-level resource index monitoring of the operation can be effectively realized.

Description

Technical Field

[0001] This application belongs to the field of computer monitoring technology, and in particular relates to a resource monitoring method and related equipment. Background Technology

[0002] With the rapid development of the chip industry, EDA (Electronic Design Automation) software has become a core tool for chip design, simulation, and verification. Its stability and resource utilization efficiency directly affect the chip development cycle and cost. In practical applications, EDA software jobs exhibit significant characteristics such as high complexity, long execution times, and resource intensity. For example, a single simulation task may last for hours or even days, and the dynamic demands on hardware resources vary greatly during execution, posing a significant challenge to resource management. Currently, the chip design industry has an increasingly urgent need for monitoring EDA software.

[0003] However, existing monitoring technologies have many shortcomings in EDA scenarios, making it difficult to meet the fine-grained monitoring needs at the process level in EDA scenarios. This leads to frequent problems such as resource waste and difficulty in troubleshooting operational anomalies, which seriously affect chip R&D efficiency and R&D cost control. Summary of the Invention

[0004] This application provides a resource monitoring method and related equipment, which can effectively monitor process-level resource indicators of a job.

[0005] In a first aspect, embodiments of this application provide a resource monitoring method applied to a resource monitoring platform, the resource monitoring method comprising: Obtain the job identifier of the target job in the electronic design automation software, and the process identifier of at least one associated process associated with the target job, and establish a mapping table between the target job and at least one associated process based on the job identifier and the process identifier of the associated process. Based on the mapping table, monitor multiple resource metrics information corresponding to at least some of the associated processes; Based on multiple resource indicator information corresponding to at least some of the associated processes, resource monitoring information of the target job is obtained by establishing the correlation between multiple resource indicator information and the target job.

[0006] In some possible implementations, the target job includes multiple task nodes, each task node corresponding to one of at least one associated process; any task node is configured to be executed by its corresponding associated process; after establishing a mapping table between the target job and at least one process, the resource monitoring method further includes: The execution information of corresponding task nodes on at least some of the related processes is monitored through the Hook mechanism; The mapping relationship table is dynamically updated based on the execution information of the corresponding task nodes in the associated process; Based on the mapping table, monitor at least some of the associated processes for various resource metrics, including: Based on the updated mapping table, monitor at least some of the associated processes in the updated mapping table for various resource metrics.

[0007] In some possible implementations, before monitoring multiple resource metrics for at least one process based on a mapping table, the resource monitoring method further includes: Execute the first indicator extension instruction to add a logical relationship between the associated process and at least one GPU resource indicator, and / or execute the second indicator extension instruction to add a logical relationship between the associated process and I / O resource indicators; Based on the mapping table, monitor at least some of the associated processes for various resource metrics, including: Based on the mapping table, monitor at least one GPU resource metric corresponding to at least some of the associated processes, and / or monitor at least some I / O resource metric corresponding to the associated processes.

[0008] In some possible implementations, obtaining the job identifier of the target job in the electronic design automation software, and the process identifier of at least one associated process associated with the target job, includes: Monitor the job startup logs in electronic design automation software; Based on the job startup log, extract the job identifier of the target job and the process identifier of at least one associated process of the target job.

[0009] In some possible implementations, before monitoring multiple resource metrics information corresponding to at least some of the associated processes based on the mapping table, the resource monitoring method further includes: The program executes extended writing instructions to convert the mapping table into a format supported by the monitoring platform, and sets the converted mapping table as a monitoring metric of the monitoring platform.

[0010] In some possible implementations, after obtaining resource monitoring information for the target job by establishing a correlation between the various resource indicator information and the target job based on at least some of the associated processes, the resource monitoring method further includes: In response to the resource visualization command of the target job, query the resource monitoring information of the target job; By calling up a visual dashboard, resource monitoring information for the target task can be displayed.

[0011] In some possible implementations, after obtaining resource monitoring information for the target job by establishing a correlation between the various resource indicator information and the target job based on at least some of the associated processes, the resource monitoring method further includes: Based on preset alarm rules, detect whether there are any abnormalities in the resource monitoring information of the target job; If an anomaly is detected in the resource monitoring information, an alarm message is generated for the target job. The alarm message is used to indicate the type of abnormal resource in the target job.

[0012] In some possible implementations, after generating an alarm message corresponding to the target job upon detecting an anomaly in the resource monitoring information, and after the alarm message indicates the type of abnormal resource in the target job, the resource monitoring method further includes: Based on the target abnormal resource type, determine the target resource allocation strategy corresponding to the target abnormal resource type from multiple resource allocation strategies corresponding to different abnormal resource types; Implement the target resource allocation strategy to adjust resource allocation.

[0013] Based on the same inventive concept, in a second aspect, embodiments of this application provide a resource monitoring platform for executing the resource monitoring method provided in any embodiment of the first aspect of this application. The resource monitoring platform includes: The data acquisition layer is used to acquire the job identifier of the target job in the electronic design automation software, as well as the process identifier of at least one associated process associated with the target job, and to establish a mapping relationship table between the target job and at least one associated process based on the job identifier and the process identifier of the associated process. A unified association layer is used to monitor multiple resource indicator information corresponding to at least some of the associated processes based on a mapping relationship table; and it is also used to obtain resource monitoring information of the target job by establishing an association relationship between multiple resource indicator information and the target job based on multiple resource indicator information corresponding to at least some of the associated processes. The visualization analysis layer is used to visualize and analyze the resource monitoring information of the target operation.

[0014] Based on the same inventive concept, in a third aspect, embodiments of this application provide a resource monitoring device applied to a resource monitoring platform, the resource monitoring device comprising: The acquisition module is used to acquire the job identifier of the target job in the electronic design automation software, as well as the process identifier of at least one associated process associated with the target job, and to establish a mapping relationship table between the target job and at least one associated process based on the job identifier and the process identifier of the associated process. The monitoring module is used to monitor various resource metrics information corresponding to at least some of the associated processes based on the mapping relationship table; A module is established to obtain resource monitoring information for a target job by establishing a correlation between the various resource indicators and the target job, based on multiple resource indicator information corresponding to at least some of the associated processes.

[0015] Fourthly, embodiments of this application provide a resource monitoring device, which includes: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the resource monitoring method provided in any of the embodiments of this application described above.

[0016] Fifthly, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement the resource monitoring method provided in any of the embodiments of this application described above.

[0017] Sixthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform a resource monitoring method as provided in any of the embodiments of this application described above.

[0018] This application provides a resource monitoring method and related equipment. By acquiring the job identifier of a target job in electronic design automation software and the process identifiers of at least one associated process, a mapping table is established between the target job and at least one associated process based on the job identifier and the process identifiers of the associated processes. Based on this mapping table, various resource indicator information corresponding to at least some of the associated processes is monitored. Thus, by establishing the association between various resource indicator information and the target job based on the various resource indicator information corresponding to at least some of the associated processes, resource monitoring information for the target job is obtained.

[0019] As described above, the resource monitoring method and related equipment of this application embodiment take the target job as the monitoring object, establish a mapping relationship table between the target job and related processes, and collect process-level resource indicator information based on the mapping relationship table. Finally, by summarizing the process-level resource indicator information of related processes and establishing the association relationship between various resource indicator information and the target job, the resource monitoring information of the target job is obtained. Compared with related technologies, the embodiments of this application help to achieve refined resource monitoring of the target job, can locate the resource consumption of a single job, help to restore the complete resource consumption chain of the job, and facilitate subsequent problem investigation and resource optimization. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a resource monitoring method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the architecture of a resource monitoring platform provided in one embodiment of this application; Figure 3 This is a schematic diagram of the architecture of a resource monitoring platform provided in another embodiment of this application; Figure 4 This is a schematic diagram of the structure of a resource monitoring device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a resource monitoring device provided in one embodiment of this application. Detailed Implementation

[0022] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0023] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0024] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0025] As described in the background section, the chip design industry has an increasingly urgent need for monitoring EDA software, mainly facing the following core issues: (1) How to track the resource consumption details of a single job at the process level in real time and accurately. For example, in complex chip design simulation tasks, the demand for different resources may vary significantly at different stages. If real-time monitoring is not possible, it may lead to unreasonable resource allocation and affect the progress of the task. (2) How to accurately assess the hardware configuration required for job operation to ensure that the task can be executed efficiently. In some chip designs, the simulation task may require a large amount of hardware resources. If it is not accurately assessed, it may lead to insufficient hardware resources, the task cannot be completed on time, or excessive hardware resources, resulting in resource waste. (3) How to use historical data to optimize resource allocation strategies and improve resource utilization efficiency. For example, by analyzing the resource usage of historical jobs, the resource demand of future jobs can be predicted, thereby allocating resources in advance and avoiding the queuing of tasks due to insufficient resources, which would prolong the R&D cycle.

[0026] However, existing monitoring technologies have many shortcomings in EDA scenarios, making it difficult to meet the needs of granular monitoring at the job process level. Specifically, industry monitoring of servers and general software mainly relies on general monitoring tool combinations, commercial monitoring platforms (such as Datadog and NewRelic), or monitoring platforms provided by EDA vendors. The typical monitoring logic of these tools is to collect metrics periodically, aggregate data according to server or application dimensions, and display macro trends (such as the total CPU (Central Processing Unit) utilization of a server). They cannot accurately correlate processes with EDA, and fail to achieve accurate tracking of the entire lifecycle and multi-dimensional resources of a single EDA job process.

[0027] For example, commercial monitoring platforms such as Datadog and NewRelic provide agent-based monitoring of servers and applications, primarily targeting general scenarios like web applications and databases, but are not optimized for the specific characteristics of EDA software operations. Another example is some EDA tools (such as certain software from Synopsys and Cadence) that have built-in simple monitoring modules that can record job start times, end times, and total resource consumption, but lack real-time capabilities and cannot correlate with underlying hardware details.

[0028] Therefore, the main defects of the existing monitoring solutions include: (1) lack of refined monitoring: the process cannot be accurately associated with EDA jobs, resulting in the inability to locate the resource consumption details of a single job, affecting the accuracy of resource allocation. (2) incomplete resource dimensions: there is insufficient monitoring of key resources for EDA scenarios, and it is impossible to distinguish the key resource allocation of different jobs, resulting in the inability to fully consider the dynamic needs of key resources when allocating resources. (3) poor adaptability: EDA software jobs have dynamic process characteristics, such as the main process starting multiple sub-processes to cooperate in computing. Existing tools are difficult to identify the relationship between processes, resulting in fragmented indicators, and it is impossible to restore the complete resource consumption chain of the job, affecting the optimization of resource allocation strategies. (4) data silo problem: the monitoring data of EDA built-in and the underlying hardware monitoring data are not connected. For example, the job start time, end time and total resource consumption data recorded by EDA software are independent of the indicator data of the underlying hardware monitoring, and cannot be comprehensively analyzed, which is not conducive to problem investigation and resource optimization.

[0029] The inventors discovered that the core reason for these defects is that the monitoring solutions mentioned above all use "servers" or "general applications" as the monitoring objects, which makes it difficult to meet the fine-grained monitoring needs of EDA scenarios at the process level, resulting in resource waste and difficulty in troubleshooting job anomalies.

[0030] In view of the above, in order to solve the problems of the prior art, embodiments of this application provide a resource monitoring method and related equipment. It should be noted that the embodiments provided in this application are not intended to limit the scope of this application.

[0031] The resource monitoring method provided in the embodiments of this application will be introduced first below.

[0032] Figure 1 A flowchart illustrating a resource monitoring method provided in one embodiment of this application is shown. This resource monitoring method is applied to a resource monitoring platform. In some examples, the resource monitoring platform can employ a combination of general monitoring tools, such as Prometheus as the core for time-series data acquisition and storage, coupled with the Grafana visualization platform for visualization, and further supplemented by Node Exporter, Process Exporter, etc., to collect resource metrics.

[0033] Specifically, such as Figure 1 As shown, this resource monitoring method includes the following steps: S110, obtain the job identifier of the target job in the electronic design automation software, and the process identifier of at least one associated process associated with the target job, and establish a mapping relationship table between the target job and at least one associated process based on the job identifier and the process identifier of the associated process. S120, based on the mapping relationship table, monitors multiple resource indicator information corresponding to at least some of the associated processes; S130, based on multiple resource indicator information corresponding to at least some associated processes, resource monitoring information of the target job is obtained by establishing the association between multiple resource indicator information and the target job.

[0034] This application provides a resource monitoring method that obtains the job identifier of a target job in electronic design automation software, as well as the process identifiers of at least one associated process. Based on the job identifier and the process identifiers of the associated processes, a mapping table is established between the target job and at least one associated process. Based on this mapping table, various resource indicator information corresponding to at least some of the associated processes is monitored. Thus, by establishing the association between various resource indicator information and the target job, resource monitoring information for the target job is obtained based on the various resource indicator information corresponding to at least some of the associated processes.

[0035] As described above, the resource monitoring method of this application embodiment takes the target job as the monitoring object, establishes a mapping relationship table between the target job and related processes, and collects process-level resource indicator information based on the mapping relationship table. Finally, by summarizing the process-level resource indicator information of related processes and establishing the association relationship between various resource indicator information and the target job, the resource monitoring information of the target job is obtained. Compared with related technologies, the embodiments of this application help to achieve refined resource monitoring of the target job, can locate the resource consumption of a single job, help to restore the complete resource consumption chain of the job, and facilitate subsequent problem investigation and resource optimization.

[0036] The specific implementation methods of steps 110 to 130 above are described in detail below.

[0037] In S110, target jobs include, for example, chip design, simulation, and verification tasks initiated in EDA software such as Synopsys VCS and Cadence Innovus.

[0038] A target job may contain multiple task nodes, each executed by a specific process. These processes that execute the task nodes of the target job are called associated processes of the target job, and they work together to complete the target job.

[0039] The job identifier for the target job can be, for example, a job ID (Identifier), the user ID that created the target job, or the job name of the target job; there is no strict limitation here. For example, the job identifier is: Job ID: 12345.

[0040] The process identifier of the aforementioned associated process can refer to the PID (Process ID) of the associated process. The PID is a unique numerical identifier assigned to each running process by the operating system, thus allowing for reliable and unique identification and management of processes. For example, the process identifier of an associated process might be: Process ID: 67890.

[0041] In practice, some examples involve listening to the job startup logs of the EDA software to obtain the job identifier of the target job and the process identifier of at least one associated process of the target job.

[0042] In other examples, the job identifier of the target job and the process identifier of at least one associated process of the target job can be obtained through the API (Application Programming Interface) of the EDA software.

[0043] Next, after obtaining the job identifier and the process identifier of the associated process, a mapping table is established between the target job and at least one associated process. In some examples, a dictionary (or hash table) is used to store the mapping relationship between the job identifier and the process identifier of the associated process, resulting in a mapping table between the target job and at least one associated process.

[0044] In S120, during implementation, based on the mapping table, various resource metrics are monitored for at least some of the associated processes. These resource metrics include CPU utilization, memory usage, and network traffic. Furthermore, considering the critical resources of EDA operations, these resource metrics can also include process-level GPU utilization and video memory usage.

[0045] For example, in Prometheus, resource metrics are collected by combining various exporters. For instance, a Node Exporter is used to collect system-level resource metrics, while a Process Exporter is used to collect resource metrics for specific processes. The collected resource metric information needs to be stored for subsequent analysis and visualization. This data can be stored using Prometheus's time-series database.

[0046] In S130, the collected process-level resource metrics information is associated with the target job. This step can use the previously established mapping table to associate the resource metrics information of each process with the corresponding job identifier.

[0047] In this way, by establishing correlations between various resource metrics and target tasks, resource monitoring information for the target tasks can be obtained. Furthermore, the correlated resource monitoring information is stored in a persistent storage system, such as Prometheus, InfluxDB, or other time-series databases, ensuring that the resource monitoring data for the target tasks can be queried and analyzed.

[0048] In this step, by collecting resource indicator information of related processes and associating these indicators with the target job, resource monitoring data for the target job is obtained. This solves the problem of process ownership identification, helps to achieve refined resource monitoring of the target job, can locate the resource consumption of a single job, helps to restore the complete resource consumption chain of the job, and is beneficial for subsequent problem investigation and resource optimization.

[0049] It should be added that, in practical applications, for ultra-large-scale EDA clusters, Prometheus can be combined with Thanos when storing resource monitoring information for jobs, thereby facilitating long-term data storage and cross-cluster queries. This is not strictly limited here.

[0050] Optionally, in some embodiments of this application, the target job includes multiple task nodes, each of the multiple task nodes corresponding to one of at least one associated process; any task node is configured to be executed by the corresponding associated process; after establishing a mapping table between the target job and at least one process, the resource monitoring method further includes: The execution information of corresponding task nodes on at least some of the related processes is monitored through the Hook mechanism; The mapping relationship table is dynamically updated based on the execution information of the corresponding task nodes in the associated process; Based on the mapping table, monitor at least some of the associated processes for various resource metrics, including: Based on the updated mapping table, monitor at least some of the associated processes in the updated mapping table for various resource metrics.

[0051] In this embodiment, multiple processes in the target job can run in parallel or sequentially according to a certain task execution order. The related processes running may differ at different time stages. Therefore, the related processes that need to be monitored at different stages are also different.

[0052] Based on this, in order to update the related processes that need to be monitored in a timely manner at the current stage, this embodiment adopts a Hook mechanism to capture and respond to the execution information of task nodes in real time during the life cycle of the related processes, such as the creation, destruction, and status changes of task nodes, and dynamically update the mapping relationship table accordingly.

[0053] This allows the mapping table to reflect the real-time correspondence between task node execution progress and associated processes. Thus, based on the updated mapping table, monitoring the resource metrics of the associated processes that require key attention at the current stage ensures the accuracy and timeliness of monitoring data, avoiding the waste of monitoring resources by allocating additional resources to associated processes that have finished or have not yet created execution task nodes.

[0054] In this embodiment, by combining the Hook mechanism to dynamically update the mapping relationship table, the correspondence between task nodes and associated processes is reflected in real time, thereby ensuring the accuracy and reliability of resource monitoring. This enables refined resource monitoring of associated processes in the target job and reduces the occurrence of monitoring resource waste.

[0055] Optionally, in some embodiments of this application, before monitoring multiple resource indicator information corresponding to at least one process based on the mapping table, the resource monitoring method further includes: Execute the first indicator extension instruction to add a logical relationship between the associated process and at least one GPU resource indicator, and / or execute the second indicator extension instruction to add a logical relationship between the associated process and I / O resource indicators; Based on the mapping table, monitor at least some of the associated processes for various resource metrics, including: Based on the mapping table, monitor at least one GPU resource metric corresponding to at least some of the associated processes, and / or monitor at least some I / O resource metric corresponding to the associated processes.

[0056] Specifically, in some embodiments, existing monitoring schemes typically focus on system-level resource monitoring, such as CPU utilization, memory usage, and overall disk I / O. However, in EDA scenarios, these monitoring schemes often fail to meet the need for granular monitoring of critical resources. For example, EDA jobs involve a large number of parallel computing tasks, such as circuit simulation and signal integrity analysis. The parallel computing capabilities of GPUs can significantly accelerate these tasks and shorten the design cycle. However, there is a gap in the existing technology regarding process-level GPU resource monitoring.

[0057] Based on this, the first indicator extension instruction is executed to add a logical relationship between the associated process and at least one image processing unit GPU resource indicator. Based on the mapping table, monitor at least some of the associated processes for various resource metrics, including: Based on the mapping table, monitor at least one GPU resource metric corresponding to at least some of the associated processes.

[0058] The aforementioned GPU (Graphics Processing Unit) resource metrics include, for example, GPU memory usage and GPU core utilization. In some embodiments, GPU resource metrics may also include GPU frequency, memory bandwidth utilization, etc., without limitation.

[0059] In this embodiment, a logical relationship is established between associated processes and GPU resource metrics by executing a first metric extension metric, for example, by extending an existing monitoring tool (such as process-exporter). Thus, after reading the correspondence between the target job and associated processes from the mapping table, GPU resource metrics of the associated processes are periodically collected based on the mapping table, thereby achieving reliable monitoring of the GPU resource metrics of the associated processes.

[0060] In practical applications, as an example, based on secondary development of process-exporter, a new logic for "process-GPU resource metric (e.g., video memory) correlation" is added to monitor GPU resource metrics. Then, based on the mapping table, the GPU resource metrics corresponding to at least some of the associated processes are monitored by parsing the output of nvidia-smi. The nvidia-smi command is used to obtain the GPU's process-level video memory data. Monitoring metrics are generated based on the parsing results, such as process_gpu_memory_used{pid="12345",gpu_id="0"}5120 (unit: MB). Thus, by extending process-level GPU memory metric monitoring through secondary development based on nvidia-smi, the gap in existing monitoring solutions for critical EDA resources is filled.

[0061] Therefore, this embodiment can effectively monitor the GPU resource indicators of the associated processes of the target job, ensure the real-time and accuracy of the monitoring data, provide strong support for resource management and troubleshooting, help to discover potential performance bottlenecks and resource shortages in a timely manner, and thus facilitate rapid optimization of resource allocation and improve resource utilization efficiency.

[0062] In other embodiments, considering that EDA jobs involve a large number of data read and write operations, such as reading design files and writing simulation results, efficient I / O operations can improve data transmission efficiency and reduce waiting time. However, there is a gap in the existing technology for monitoring process-level I / O metrics.

[0063] Based on this, the second indicator extension instruction is executed to add a logical relationship between the associated process and the I / O resource indicator; Based on the mapping table, monitor at least some of the associated processes for various resource metrics, including: Based on the mapping table, monitor the I / O resource metrics corresponding to at least some of the related processes.

[0064] The aforementioned I / O resource metrics include, for example, the number of bytes read and written, and the number of I / O operations, but are not strictly limited here.

[0065] In this embodiment, a logical relationship is established between associated processes and I / O resource metrics by executing a second metric extension, for example, by extending existing monitoring tools (such as node-exporter). Thus, after reading the correspondence between the target job and associated processes from the mapping table, the I / O resource metrics of the associated processes are periodically collected based on the mapping table, thereby achieving reliable monitoring of the I / O resource metrics of the associated processes.

[0066] In practical applications, as an example, based on secondary development of node-exporter, new logic is added to monitor I / O resource metrics. Based on the mapping table and the / proc filesystem, I / O resource metrics for associated processes, such as read / write bytes, are obtained by reading the / proc / [pid] / io file. Corresponding I / O resource metrics are generated based on the read data. For example, process_io_read_bytes{pid="12345"}1048576 and process_io_write_bytes{pid="12345"}524288 are generated. Thus, through I / O metric monitoring extension and secondary development based on the / proc filesystem, the gap in existing monitoring solutions for critical EDA resources is filled.

[0067] Therefore, this embodiment can effectively monitor the I / O resource indicators of the associated processes of the target job, ensure the real-time and accuracy of the monitoring data, provide strong support for resource management and troubleshooting, promptly identify and resolve I / O-related issues, and reduce job failures caused by I / O errors.

[0068] It should be added that, in some other embodiments, eBPF (Extended Berkeley Packet Filter) technology or other tools can also be used to collect process-level GPU resource and I / O resource metrics. This can reduce the dependence on, for example, nvidia-smi and / proc file systems. The tools used for the process-level resource monitoring extensions described above in this application are not strictly limited.

[0069] It should be added that, in some optional embodiments, network traffic probes can be used to extend the implementation to process-level network traffic monitoring, so as to establish the underlying correlation between jobs, processes and hardware indicators (such as network fluctuations) in the future.

[0070] Optionally, in some embodiments of this application, obtaining the job identifier of the target job in the electronic design automation software, and the process identifier of at least one associated process associated with the target job, includes: Monitor the job startup logs in electronic design automation software; Based on the job startup log, extract the job identifier of the target job and the process identifier of at least one associated process of the target job.

[0071] In practice, the path of the job startup log file in the EDA software can be determined, and the job startup log can be obtained by accessing the job startup log file path to monitor changes in the log file.

[0072] Next, by parsing the job startup log, the job identifier of the target job and the process identifier of the associated process can be extracted. More specifically, by parsing the job startup log, information such as the startup time of the target job can also be extracted, which is helpful for subsequent problem localization and analysis.

[0073] In this embodiment, by monitoring the job startup logs of the EDA software and extracting the job identifier and process identifier, the target job can be reliably monitored and a mapping table can be created, which helps to improve the accuracy of the mapping table.

[0074] Optionally, in some embodiments of this application, before monitoring multiple resource indicator information corresponding to at least some associated processes based on the mapping relationship table, the resource monitoring method further includes: The program executes extended writing instructions to convert the mapping table into a format supported by the resource monitoring platform, and sets the converted mapping table as a monitoring indicator of the resource monitoring platform.

[0075] The aforementioned extension writing instructions can be implemented by pre-developing a custom exporter to convert the data in the mapping table into a Prometheus-supported format and expose it as monitoring metrics.

[0076] In practice, in order to fully realize reliable monitoring of the target job and its associated processes, after obtaining the above mapping table, the mapping table is converted into a format supported by the resource monitoring platform by executing the corresponding extended writing instructions, and then exposed as a monitoring indicator of the resource monitoring platform.

[0077] For example, in a resource monitoring platform that includes Prometheus, a custom exporter (eda-job-exporter) can be developed to convert the data in the mapping table into a format supported by Prometheus, such as eda_job_pid_mapping {job_id="J123" ,user="userA",pid="12345"}1. This converted mapping table can then be set as a monitoring metric for the monitoring platform, ensuring that Prometheus can periodically collect resource metrics for these related processes. When the mapping table is dynamically updated, the related processes for which Prometheus periodically collects metrics may also differ.

[0078] In this embodiment, the mapping relationship table is converted into a format supported by the resource monitoring platform by extending the writing instructions, and exposed as a monitoring indicator, so as to realize fine-grained monitoring of EDA operations, improve the real-time performance and accuracy of monitoring data, and avoid the resource monitoring platform's inadequate monitoring of the mapping relationship table.

[0079] Optionally, in some embodiments of this application, after obtaining resource monitoring information of the target job by establishing a correlation between the various resource indicator information and the target job based on at least some of the associated processes' corresponding multiple resource indicator information, the resource monitoring method further includes: In response to the resource visualization command of the target job, query the resource monitoring information of the target job; By calling up a visual dashboard, resource monitoring information for the target task can be displayed.

[0080] The resource visualization command mentioned above can be entered by the user or an automation tool. This resource visualization command specifies the target job to be queried; for example, the command may include the job identifier of the target job, such as job_12345.

[0081] In this embodiment, after receiving a resource visualization instruction from a user or automation tool, the system queries the resource monitoring platform for resource monitoring information related to the target job, such as CPU utilization, memory usage, GPU utilization, video memory usage, I / O resource metrics, etc.

[0082] Next, the retrieved resource monitoring information is passed to a visualization tool (such as Grafana) and displayed on a relevant dashboard, allowing users to intuitively view and analyze the resource usage of the target job, thereby enabling analysis and decision-making.

[0083] In some more specific embodiments, appropriate visualization dashboards are created or selected based on the resource monitoring requirements of the target job, displaying key resource metrics through corresponding charts and other means. When displaying these dashboards, the overall resource metric usage of the target job across different resource metric dimensions can be shown, or multi-dimensional resource metrics corresponding to each process within the target job can be displayed.

[0084] Furthermore, a unified indicator labeling system can be pre-defined, including core business labels, resource dimension labels, and time dimension labels, which can be expanded as needed. Core business labels include, for example, job_id (unique job identifier), user, task_type (EDA task type, such as simulation or routing), pid (process ID), or design phase. Resource dimension labels can include resource type, node hostname, GPU device ID, or job queue. Time dimension labels include, for example, job start time or duration bucket.

[0085] In this way, users can filter key requirement tags by combining different indicator tags, thereby enabling customized visualization content for multi-dimensional aggregation analysis. The visualization dashboard supports user-level and job-level resource analysis and optimization decisions, and provides services such as job runtime comparison analysis, resource allocation suggestions, bottleneck identification, and optimization suggestions.

[0086] It should be added that, in some feasible implementations, multi-dimensional aggregation analysis can employ multi-level grouping aggregation, hierarchical aggregation calculation, dynamic dimension switching, or real-time data stream processing. Multi-level grouping aggregation supports simultaneous cross-grouping by multiple dimensions such as user, project, tool type, and design stage. Hierarchical aggregation calculation supports layer-by-layer aggregation analysis from node level to process level to job level to user level. Dynamic dimension switching allows for dynamic adjustment of aggregation dimensions based on query requirements, supporting ad-hoc analysis. Real-time data stream processing can be based on the Prometheus streaming architecture to ensure the real-time nature of aggregation results.

[0087] In other feasible embodiments, the intelligent analysis function can be further optimized in depth, comprehensively improving resource management, cost control, and system performance through a multi-indicator correlation analysis engine. Core functions include establishing a bottleneck feature library to identify and classify different types of resource bottlenecks; causal reasoning analysis to identify the root causes of bottlenecks based on temporal relationships; propagation path analysis to track the propagation of bottlenecks and their impact range; and bottleneck severity assessment to classify bottlenecks according to their impact range and duration. These functions collectively support intelligent diagnostic recommendations, including bottleneck root cause localization, solution recommendation, optimization priority ranking, and expected effect prediction, helping users accurately locate problems and take effective measures.

[0088] In terms of capacity planning, the system provides business-plan-based forecasts that consider the resource requirements of the project pipeline while assessing technological development factors, such as the impact of new processes and tools on resource needs. Furthermore, it offers seasonal fluctuation forecasts, taking into account factors such as project cycles and holidays, and provides multi-scenario forecasts, including baseline, optimistic, and pessimistic scenarios. Based on these forecasts, the system generates planning recommendations covering hardware procurement, configuration optimization, and budget allocation, and provides early warnings and mitigation suggestions for capacity shortage risks.

[0089] In terms of cost optimization, idle and inefficiently used resources are identified through resource utilization assessment and efficiency benchmark comparison, thereby proposing opportunities for efficiency improvement. The optimization recommendation engine further provides suggestions for job scheduling optimization, resource allocation optimization, tool usage optimization, and process improvement to ensure efficient resource utilization. The cost-benefit analysis module performs ROI (Return on Investment) analysis on optimization measures, prioritizes optimization measures based on implementation difficulty, and provides short-term, medium-term, and long-term cost optimization implementation paths, while tracking the actual effects of optimization measures.

[0090] In terms of system integration and user experience, it provides interactive analytics dashboards, supports data drill-down, filtering, and correlation analysis, as well as intelligent report generation, automatically generating multi-dimensional analysis reports on a regular basis. The alert push function ensures users receive important findings and warnings promptly, while mobile support allows users to conveniently view and interact with analysis results on mobile devices.

[0091] Finally, in terms of decision support, comprehensive support is provided to users through scenario simulation, solution comparison, impact assessment, and implementation tracking. This includes simulating the effects of resource adjustments and process changes, comparing different optimization solutions, evaluating the impact of planning decisions on resource utilization, cost, and efficiency, and tracking the effectiveness of decision implementation to ensure continuous improvement and optimization. These integrated functions help users make more accurate decisions in resource management, cost control, and system performance optimization.

[0092] Furthermore, in some optional embodiments, multi-dimensional visualization dashboards can be set up to help users quickly identify resource bottlenecks, optimize resource allocation, improve job scheduling efficiency, and perform performance optimization. These multi-dimensional visualization dashboards include, but are not limited to: a global resource overview dashboard, a job detail analysis dashboard, a business-perspective monitoring dashboard, and a performance optimization dashboard.

[0093] Specifically, the aforementioned global resource overview dashboard provides a heatmap of overall cluster resource utilization, helping users quickly identify resource bottlenecks. Furthermore, this dashboard can display the running status distribution of jobs in each queue, facilitating monitoring of job scheduling efficiency. Additionally, the dashboard can display the top 10 rankings of user resource consumption, supporting resource quota management to ensure reasonable resource allocation. Simultaneously, the dashboard can also display operational statistics and analysis of key EDA tools, helping users optimize tool configurations.

[0094] The aforementioned job details analysis dashboard can include multi-dimensional resource usage trend charts for individual jobs, providing detailed tables of process-level resource consumption to help users gain a deeper understanding of the resource usage of each job. Furthermore, the dashboard can also display timeline graphs of GPU memory usage and analytical charts of I / O operation modes to provide users with comprehensive job performance analysis.

[0095] The aforementioned business-perspective monitoring dashboard provides comparative analysis of users' historical task resource efficiency, helping them assess changes in resource utilization efficiency. This dashboard can also display and track project resource budget usage, ensuring the rational allocation of project resources. Furthermore, it can display tool usage preferences and performance analysis to help users optimize tool selection. Finally, it can display statistics on task success rates and failure reasons to provide users with improvement directions.

[0096] The aforementioned performance optimization dashboard can display comparative analysis of job execution times, provide resource allocation suggestions, and help users optimize resource configuration. This dashboard can also display bottleneck identification and optimization suggestions to ensure job execution efficiency. Furthermore, the dashboard can display cost-benefit analysis reports to help users evaluate the economic benefits of optimization measures, ensuring the scientific and rational nature of resource optimization.

[0097] These multi-dimensional dedicated monitoring dashboards allow users to fully grasp the system's operational status, quickly identify and resolve resource bottlenecks, optimize job scheduling and resource allocation, and improve operational efficiency.

[0098] Optionally, in some embodiments of this application, after obtaining resource monitoring information of the target job by establishing a correlation between the various resource indicator information and the target job based on at least some of the associated processes' corresponding multiple resource indicator information, the resource monitoring method further includes: Based on preset alarm rules, detect whether there are any abnormalities in the resource monitoring information of the target job; If an anomaly is detected in the resource monitoring information, an alarm message is generated for the target job. The alarm message is used to indicate the type of abnormal resource in the target job.

[0099] In practice, a series of alarm rules are defined in advance based on actual monitoring needs. These rules can be based on indicators such as resource utilization, resource usage, and performance status. Alarm rules include, but are not limited to, CPU utilization exceeding 80%, GPU memory usage exceeding 90%, and I / O operation speed exceeding 10MB / s, etc., but this embodiment does not impose strict limitations on these rules.

[0100] This allows the resource monitoring platform to automatically detect anomalies in resource monitoring information by configuring alarm rules. The platform periodically queries resource monitoring information and performs checks based on preset alarm rules. When anomalies are detected, an alarm is generated for the target job. The alarm includes the target job identifier and the type of abnormal resource, clearly indicating the type of abnormal resource for the target job, enabling operations personnel to quickly locate the problem.

[0101] Furthermore, after generating alarm information, it can be sent to a preset alarm receiver, such as email, SMS, or instant messaging tools, to quickly notify operations and maintenance personnel and help them quickly locate and resolve problems. Alternatively, after generating alarm information, resource allocation and optimization measures can be automatically performed based on the content contained in the alarm information.

[0102] In some optional examples, to ensure comprehensive monitoring of job execution, job status alarm rules are set up: by detecting job failure status in real time and immediately notifying relevant users, problems can be detected and handled promptly. Simultaneously, job execution time is monitored, triggering a job timeout alarm when it exceeds the expected duration, helping to optimize job scheduling and resource allocation. Furthermore, job queuing time is detected, issuing a long queue alarm when queuing time is excessive, indicating the need to optimize resource scheduling and improve system efficiency.

[0103] In addition, resource usage alarm rules can be set: when CPU, memory, and GPU memory usage exceed preset thresholds, a resource overload alarm will be triggered to prevent resource overload from causing system performance degradation. The system will also detect abnormal resource usage patterns, such as excessively low GPU utilization, triggering a resource utilization anomaly alarm to indicate potential configuration problems or performance bottlenecks. Furthermore, resource contention will be monitored, and a resource contention alarm will be issued when it is detected to prevent performance degradation caused by resource contention.

[0104] In addition, performance metric alarm rules are set up: Storage I / O performance is monitored, and an I / O performance degradation alarm is triggered when a performance bottleneck is detected to ensure data transmission efficiency. Network communication latency is detected, and a network latency increase alarm is triggered when latency increases to ensure the efficiency of distributed computing. Simultaneously, the execution efficiency of EDA tools is monitored, configuration problems are identified, and tool execution efficiency alarms are triggered to ensure the tools are running at their optimal state.

[0105] In addition, set up business KPI alert rules: From the perspective of project management and cost control, predict project completion time based on resource usage, and issue project schedule risk alerts when there are schedule risks to help adjust plans in advance. Monitor resource costs, and issue cost overrun alerts when costs exceed the budget to ensure that resource usage is within budget. Furthermore, identify tasks with low resource utilization efficiency, trigger resource inefficiency alerts, and prompt the need to optimize resource allocation or task configuration.

[0106] In other optional examples, intelligent routing mechanisms can be employed for alarm handling, automatically routing alarms to the relevant responsible parties based on alarm type and severity to ensure timely processing. Simultaneously, an alarm escalation mechanism is supported to ensure that critical alarms are prioritized. Furthermore, a complete closed-loop alarm handling mechanism is established, covering the entire process from alarm triggering to notification distribution, processing execution, effect verification, and knowledge accumulation. Detailed logs are recorded at each stage, supporting process traceability to ensure effective problem resolution. An alarm handling knowledge base is established to accumulate handling experience and improve processing efficiency.

[0107] In other optional examples, a continuous optimization mechanism can be set up to periodically analyze alarm data, optimize alarm thresholds and rules, and ensure that the alarm system can adapt to constantly changing system states and business needs. Alarm strategies are adjusted based on processing results, enabling automatic learning and optimization of alarm rules, thereby improving the intelligence level of the alarm system. Through these measures, this embodiment can effectively improve the system's monitoring and alarm capabilities, ensure the smooth execution of operations and the stable operation of the system, while reducing maintenance costs and improving overall efficiency.

[0108] Optionally, in some embodiments of this application, after generating alarm information corresponding to the target job when an anomaly is detected in the resource monitoring information, and the alarm information is used to indicate the abnormal resource type of the target job, the resource monitoring method further includes: Based on the target abnormal resource type, determine the target resource allocation strategy corresponding to the target abnormal resource type from multiple resource allocation strategies corresponding to different abnormal resource types; Implement the target resource allocation strategy to adjust resource allocation.

[0109] The aforementioned categories of abnormal resource types include GPU resource anomalies, CPU resource anomalies, memory resource anomalies, and I / O resource anomalies. Based on the specific resource metrics being monitored, these categories can be further subdivided into smaller subcategories. For example, using GPU resource anomalies as the main category, subcategories could include GPU utilization anomalies, VRAM usage anomalies, etc., without strict limitations.

[0110] In this embodiment, resource bottlenecks are accurately located based on different types of abnormal resources, and various handling strategies for different abnormal situations are pre-set based on historical data and past processing experience. Thus, after identifying the type of abnormal resource present in the current target job, the most suitable strategy is selected from multiple preset resource allocation strategies to handle the abnormal situation, thereby optimizing resource allocation and reducing unnecessary resource waste.

[0111] As specific examples, regarding the handling of resource anomalies: For GPU resource anomalies, if memory usage exceeds 90%, the resource allocation strategy can be to immediately issue an alert and recommend migrating to a node with more memory, while simultaneously checking for memory leaks or excessively large models. If GPU utilization is below 10%, the resource allocation strategy involves performance analysis to check for CPU bottlenecks and adjusting corresponding parameters (e.g., batch size) or data pipelines accordingly. When the GPU temperature is too high, the resource allocation strategy corresponds to automatically adjusting fan speed, reducing GPU frequency if necessary, or migrating jobs.

[0112] For abnormal CPU resource usage: If a single-core CPU is continuously under full load, resource allocation strategies include: checking for single-threaded bottlenecks and recommending the use of multi-threaded versions of tools. If there are excessive context switches, resource allocation strategies include: reducing unnecessary inter-process communication and adjusting process priorities in CPU scheduling (process nice values). In cases of unbalanced CPU load, resource allocation strategies include: reallocating computing tasks to achieve load balancing.

[0113] For abnormal memory resource usage: If memory usage continues to increase, resource allocation strategies include generating memory snapshots for analysis and setting memory usage threshold alerts. If memory swapping is frequent, resource allocation strategies include immediately allocating more physical memory or migrating to nodes with sufficient memory; simultaneously, automatically identifying memory leak patterns and issuing alerts.

[0114] For I / O resource anomalies: If a disk I / O bottleneck is detected, resource allocation strategies include migrating data to higher-performance storage and optimizing data access patterns. For network storage latency, resource allocation strategies include checking network connectivity and enabling data caching; when storage space is insufficient, automatically cleaning up temporary files and triggering a storage expansion process.

[0115] Furthermore, in some optional embodiments, an intelligent diagnostic engine can be introduced into the alarm process. This engine, based on a rule engine and machine learning algorithms, performs root cause analysis of anomalies, automatically generates handling suggestions and optimization solutions, and supports automated execution of handling actions. Moreover, a tiered handling mechanism can be adopted in the handling strategy. For example, Level 1 handling automatically adjusts resource allocation and job scheduling; Level 2 handling notifies users and administrators and provides handling suggestions; and Level 3 handling involves manual intervention for in-depth problem investigation and optimization.

[0116] In some alternative embodiments, the effectiveness of the processing measures can be further monitored, processing experience can be accumulated, processing strategies can be optimized, and a processing knowledge base can be established to quickly resolve similar problems. This helps to effectively improve the efficiency of resource management and the overall performance of the system, and ensures the smooth execution of operations.

[0117] Overall, this application provides a resource monitoring method that enables comprehensive monitoring of a single EDA job process, covering key resources such as CPU, GPU, memory, video memory, and I / O. This allows for precise identification of resource bottlenecks, such as slow operation due to insufficient GPU video memory. Furthermore, by establishing a relationship between "user-job-process," this method achieves user-level resource management, enabling the statistical analysis of resource usage efficiency for different users. This provides data support for resource quota allocation and effectively reduces resource waste.

[0118] Meanwhile, by integrating job and hardware metrics, this method can quickly analyze the impact of hardware anomalies on jobs, improving troubleshooting efficiency and significantly shortening troubleshooting time. Furthermore, by leveraging log parsing and hook technology, real-time dynamic association between EDA jobs and multiple processes (including child processes) is achieved, resolving the process ownership identification problem. Finally, this method also supports visual dashboards, providing user-level and job-level resource analysis and optimization decisions, including job runtime comparison analysis, resource allocation suggestions, and bottleneck identification and optimization suggestions, thereby helping users manage and optimize resource usage more efficiently.

[0119] Based on the resource monitoring method provided in the above embodiments, and with the same inventive concept, this application also provides a resource monitoring platform corresponding to the above resource monitoring method. The following describes... Figure 2 A detailed introduction to the resource monitoring platform is provided.

[0120] Figure 2 This illustration shows a schematic diagram of a resource monitoring platform provided in an embodiment of this application. The resource monitoring platform 200 is used to execute the resource monitoring method provided in the foregoing embodiments of this application. The resource monitoring platform 200 includes: The data acquisition layer 210 is used to acquire the job identifier of the target job in the electronic design automation software, as well as the process identifier of at least one associated process associated with the target job, and to establish a mapping relationship table between the target job and at least one associated process based on the job identifier and the process identifier of the associated process. The unified association layer 220 is used to monitor multiple resource indicator information corresponding to at least some of the associated processes based on the mapping relationship table; and is also used to obtain the resource monitoring information of the target job by establishing the association relationship between multiple resource indicator information and the target job based on the multiple resource indicator information corresponding to at least some of the associated processes. The visualization analysis layer 230 is used to perform visualization analysis on the resource monitoring information of the target operation.

[0121] The resource monitoring platform 200 of this application embodiment constructs the aforementioned data acquisition layer 210, unified association layer 220, and visualization analysis layer 230. The data acquisition layer 210 can interact with the EDA cluster 240, obtaining the job identifier of the target job and the process identifiers of at least one associated process from the EDA cluster 240. Based on the job identifier and the process identifiers of the associated processes, it establishes a mapping relationship table between the target job and the associated processes, with the target job as the monitoring object. The unified association layer 220 collects process-level resource indicator information according to the mapping relationship table. The unified association layer 220 obtains the resource monitoring information of the target job by summarizing the process-level resource indicator information of the associated processes and establishing association relationships between various resource indicator information and the target job. Finally, the visualization analysis layer 230 performs visualization analysis on the resource monitoring information of the target job. Compared to related technologies, this resource monitoring platform 200, based on the aforementioned data acquisition layer 210, unified association layer 220, and visualization analysis layer 230, helps to achieve refined resource monitoring of target operations, can locate the resource consumption of individual operations, and helps to restore the complete resource consumption chain of operations through visualization analysis, which is beneficial for problem investigation and resource optimization.

[0122] Specifically, please see Figure 3 , Figure 3 A schematic diagram of the resource monitoring platform provided in another embodiment of this application is shown. The aforementioned data acquisition layer 210 may include the following functional components: Node-exporter functional component 211, Process-exporter extended functional component 212, Nvidia-exporter enhanced functional component 213, network traffic probe functional component 214, and task tag injector 215. Node-exporter functional component 211 can be used to collect node metrics, Process-exporter extended functional component 212 can be used to collect process CPU / memory / I / O, Nvidia-exporter enhanced functional component 213 can be used to collect process GPU / video memory, and network traffic probe functional component 214 can be used to collect process-level traffic. Finally, these data are transmitted to task tag injector 215, where the data is tagged with corresponding metrics so that the monitoring components in the unified association layer 22 can perform metric monitoring.

[0123] In some specific embodiments, within the data acquisition layer 210, the path to the job startup log file in the EDA cluster 240 is determined. By accessing this path, the job startup log is obtained to monitor changes in the log file. Next, by parsing the job startup log, the job identifier of the target job and the process identifier of the associated process can be extracted. More specifically, by parsing the job startup log, information such as the target job's startup time can also be extracted, which is helpful for subsequent problem localization and analysis.

[0124] Next, in the aforementioned data acquisition layer 210, a custom exporter (eda-job-exporter) is developed to convert the data in the mapping table into a format supported by Prometheus, such as eda_job_pid_mapping {job_id="J123" ,user="userA",pid="12345"}1.

[0125] Furthermore, in the aforementioned data acquisition layer 210, secondary development is performed based on the process-exporter expansion component 212 to add "process-GPU resource metric (e.g., video memory) correlation" logic to monitor GPU resource metrics. Next, based on the mapping table, the GPU resource metrics corresponding to at least some associated processes are monitored by parsing the output of nvidia-smi. The nvidia-smi command is used to obtain GPU process-level video memory data. Monitoring metrics are generated based on the parsing results, such as process_gpu_memory_used{pid="12345",gpu_id="0"}5120 (unit: MB). Thus, through the extension of process-level GPU video memory metric monitoring, and based on the secondary development of nvidia-smi, the gap in existing monitoring solutions for critical EDA resources is filled.

[0126] Furthermore, within the aforementioned data acquisition layer 210, new logic is added to monitor I / O resource metrics through secondary development based on the Node-exporter functional component 211. Based on the mapping table and the / proc file system, I / O resource metrics for associated processes, such as the number of bytes read and written, are obtained by reading the / proc / [pid] / io file. Corresponding I / O resource metrics are then generated based on the read data. Thus, through the extension of I / O metric monitoring and secondary development based on the / proc file system, the gap in existing monitoring solutions for critical EDA resources is filled.

[0127] The unified association layer 220 mentioned above may include Prometheus 221. Prometheus 221 can be used to monitor various resource indicator information corresponding to associated processes based on a mapping table. The unified association layer 220 is also used to obtain and store resource monitoring information of the target job by establishing association relationships between various resource indicator information and the target job based on the various resource indicator information corresponding to at least some of the associated processes.

[0128] The aforementioned visualization analysis layer 230 may include a Grafana dashboard 231. The visualization analysis layer 230 can perform visual analysis of resource monitoring information for target jobs. By calling the visualization dashboard, it supports user-level and job-level resource analysis and optimization decisions, ensuring multi-dimensional aggregated analysis. It can also provide job runtime comparison analysis, resource allocation suggestions, bottleneck identification, and optimization suggestions. Thus, combined with Prometheus 221 and Grafana dashboard 231, data storage, querying, visualization, and alerting can be achieved, ensuring stable system operation and efficient management. For example, a job may run slowly due to insufficient GPU memory.

[0129] It should be added that the functions and technical effects of the aforementioned resource monitoring platform 200 can also be found in the relevant descriptions of the resource monitoring methods mentioned above, which will not be elaborated further here for the sake of brevity.

[0130] Based on the resource monitoring method provided in the above embodiments, and with the same inventive concept, this application also provides a resource monitoring device corresponding to the above resource monitoring method. The following describes... Figure 4 A detailed introduction to the resource monitoring device is provided.

[0131] Figure 4 A schematic diagram of the structure of a resource monitoring device provided in an embodiment of this application is shown, which is applied to a monitoring platform. Figure 4 The resource monitoring device 400 shown includes: The acquisition module 410 is used to acquire the job identifier of the target job in the electronic design automation software, and the process identifier of at least one associated process associated with the target job, and to establish a mapping relationship table between the target job and the at least one associated process based on the job identifier and the process identifier of the associated process. Monitoring module 420 is used to monitor multiple resource indicator information corresponding to at least some of the associated processes based on the mapping relationship table; The module 430 is used to obtain resource monitoring information of the target job by establishing a relationship between the multiple resource indicator information and the target job based on multiple resource indicator information corresponding to at least some of the associated processes.

[0132] This application provides a resource monitoring device that, by setting up corresponding functional modules, acquires the job identifier of a target job in electronic design automation software, as well as the process identifier of at least one associated process related to the target job. Based on the job identifier and the process identifiers of the associated processes, a mapping table is established between the target job and at least one associated process. Based on this mapping table, various resource indicator information corresponding to at least some of the associated processes is monitored. Thus, by establishing the association between various resource indicator information and the target job based on the various resource indicator information corresponding to at least some of the associated processes, resource monitoring information for the target job is obtained.

[0133] As described above, the resource monitoring device of this application embodiment takes the target job as the monitoring object, establishes a mapping relationship table between the target job and related processes, and collects process-level resource indicator information based on the mapping relationship table. Finally, by summarizing the process-level resource indicator information of related processes and establishing the association relationship between various resource indicator information and the target job, the resource monitoring information of the target job is obtained. Compared with related technologies, the embodiments of this application help to achieve refined resource monitoring of the target job, can locate the resource consumption of a single job, help to restore the complete resource consumption chain of the job, and facilitate subsequent problem investigation and resource optimization.

[0134] In some possible implementations, the target job includes multiple task nodes, each task node corresponding to one of at least one associated process; any task node is configured to be executed by its corresponding associated process; after establishing a mapping table between the target job and at least one process, the resource monitoring device further includes: The task monitoring module is used to monitor the execution information of corresponding task nodes on at least some of the associated processes through a Hook mechanism; The update module is used to dynamically update the mapping relationship table based on the execution information of the corresponding task nodes on the associated process; The above-mentioned mapping relationship table monitors at least some of the related processes and their corresponding resource metrics, including: Based on the updated mapping table, monitor at least some of the associated processes in the updated mapping table for various resource metrics.

[0135] In some possible implementations, before monitoring multiple resource metrics information corresponding to at least one process based on a mapping table, the resource monitoring device further includes: An execution module is configured to execute a first indicator extension instruction to add a logical relationship between the associated process and at least one image processing unit (GPU) resource indicator, and / or execute a second indicator extension instruction to add a logical relationship between the associated process and I / O resource indicators. The above-mentioned mapping relationship table monitors at least some of the related processes and their corresponding resource metrics, including: Based on the mapping table, monitor at least one GPU resource metric corresponding to at least some of the associated processes, and / or monitor at least some I / O resource metric corresponding to the associated processes.

[0136] In some possible implementations, obtaining the job identifier of the target job in the electronic design automation software, and the process identifier of at least one associated process associated with the target job, includes: The log monitoring module is used to monitor the job startup logs in electronic design automation software. The identifier extraction module is used to extract the job identifier of the target job and the process identifier of at least one associated process of the target job based on the job startup log.

[0137] In some possible implementations, before monitoring multiple resource metrics information corresponding to at least some of the associated processes based on the mapping table, the resource monitoring device further includes: The third execution module is used to execute extended writing instructions, convert the mapping relationship table into a format supported by the monitoring platform, and set the converted mapping relationship table as a monitoring indicator of the monitoring platform.

[0138] In some possible implementations, after obtaining resource monitoring information for the target job by establishing a correlation between the various resource indicator information and the target job based on at least some of the associated processes, the resource monitoring device further includes: The query module is used to query the resource monitoring information of the target job in response to the resource visualization command of the target job. The display module is used to show resource monitoring information of the target job by calling a visual dashboard.

[0139] In some possible implementations, after obtaining resource monitoring information for the target job by establishing a correlation between the various resource indicator information and the target job based on at least some of the associated processes, the resource monitoring device further includes: The detection module is used to detect whether there are any abnormalities in the resource monitoring information of the target job based on preset alarm rules; The generation module is used to generate alarm information corresponding to the target job when an anomaly is detected in the resource monitoring information. The alarm information is used to indicate the type of abnormal resource in the target job.

[0140] In some possible implementations, after generating alarm information corresponding to the target job upon detecting an anomaly in the resource monitoring information, and after the alarm information indicates the type of abnormal resource in the target job, the resource monitoring device further includes: The determination module is used to determine the target resource allocation strategy corresponding to the target abnormal resource type from multiple resource allocation strategies corresponding to different abnormal resource types based on the target abnormal resource type. The fourth execution module is used to execute the target resource allocation strategy to adjust resource allocation.

[0141] Based on the resource monitoring method provided in the above embodiments, and with the same inventive concept, this application also provides a resource monitoring device corresponding to the above resource monitoring method. The following describes... Figure 5 A detailed introduction to resource monitoring equipment is provided.

[0142] Please see below. Figure 5 , Figure 5 This is a schematic diagram of the structure of a resource monitoring device provided in one embodiment of this application.

[0143] The resource monitoring device may include a processor 501 and a memory 502 storing computer program instructions.

[0144] Specifically, the processor 501 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0145] Memory 502 may include mass storage for data or instructions. For example, and not limitingly, memory 502 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 502 may include removable or non-removable (or fixed) media. Where appropriate, memory 502 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 502 is non-volatile solid-state memory.

[0146] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0147] The processor 501 implements any of the resource monitoring methods described in the above embodiments by reading and executing computer program instructions stored in the memory 502.

[0148] In one example, the data resource monitoring device may also include a communication interface 503 and a bus 510. For example, Figure 5 As shown, the processor 501, memory 502, and communication interface 503 are connected through bus 510 and complete communication with each other.

[0149] The communication interface 503 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0150] Bus 510 includes hardware, software, or both, that couples components of a resource monitoring device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 510 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0151] The resource monitoring device executes the resource monitoring method in the embodiments of this application, thereby realizing the resource monitoring method described in the embodiments of this application.

[0152] Furthermore, in conjunction with the resource monitoring methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the resource monitoring methods in the above embodiments.

[0153] Based on the resource monitoring methods in the above embodiments, this application provides a computer program product. When the instructions in the computer program product are executed by the processor of an electronic device, the electronic device performs the resource monitoring method provided in any of the above embodiments of this application.

[0154] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0155] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0156] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0157] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0158] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method of resource monitoring, characterized by, The method is applied to a resource monitoring platform, and comprises the following steps: Obtaining a job identifier of a target job in electronic design automation software and a process identifier of at least one associated process associated with the target job, and establishing a mapping relationship table between the target job and the at least one associated process based on the job identifier and the process identifier of the associated process; Monitoring a plurality of resource index information corresponding to at least part of the associated processes based on the mapping relationship table; Based on the plurality of resource index information corresponding to at least part of the associated processes, the association between the plurality of resource index information and the target job is established to obtain the resource monitoring information of the target job.

2. The resource monitoring method of claim 1, wherein, The target job comprises a plurality of task nodes, and each task node in the plurality of task nodes corresponds to one associated process in the at least one associated process; any task node is configured to be executed by the corresponding associated process; After the mapping relationship table between the target job and the at least one process is established, the method further comprises the following steps: Monitoring the execution information of the corresponding task node on the at least part of the associated processes through a Hook mechanism; Based on the execution information of the corresponding task node on the associated process, the mapping relationship table is dynamically updated; The monitoring of the plurality of resource index information corresponding to at least part of the associated processes based on the mapping relationship table comprises: Based on the updated mapping relationship table, the plurality of resource index information corresponding to at least part of the associated processes in the updated mapping relationship table is monitored.

3. The resource monitoring method of claim 1, wherein, Before the plurality of resource index information corresponding to the at least one process is monitored based on the mapping relationship table, the method further comprises the following steps: Executing a first index extension instruction to add a logical relationship between the associated process and at least one GPU resource index, and / or executing a second index extension instruction to add a logical relationship between the associated process and an I / O resource index; The monitoring of the plurality of resource index information corresponding to at least part of the associated processes based on the mapping relationship table comprises: Based on the mapping relationship table, at least one GPU resource index corresponding to at least part of the associated processes is monitored, and / or I / O resource index corresponding to at least part of the associated processes is monitored.

4. The method of claim 1, wherein, The obtaining of the job identifier of the target job in the electronic design automation software and the process identifier of the at least one associated process associated with the target job comprises the following steps: Listening to a job start log in the electronic design automation software; Based on the job start log, the job identifier of the target job and the process identifier of the at least one associated process associated with the target job are extracted.

5. The method of claim 1, wherein, Before the plurality of resource index information corresponding to at least part of the associated processes is monitored based on the mapping relationship table, the method further comprises the following steps: Executing an extension writing instruction to convert the mapping relationship table into a format supported by the resource monitoring platform, and setting the mapping relationship table after the format conversion as a monitoring index of the resource monitoring platform.

6. The method of claim 1, wherein, After the resource monitoring information of the target job is obtained based on the multiple resource index information corresponding to at least part of the associated processes by establishing an association between the multiple resource index information and the target job, the method further comprises: In response to a resource visualization instruction of the target job, the resource monitoring information of the target job is queried; The resource monitoring information of the target job is displayed by calling a visualization board.

7. The method of claim 1, wherein, After the resource monitoring information of the target job is obtained based on the multiple resource index information corresponding to at least part of the associated processes by establishing an association between the multiple resource index information and the target job, the method further comprises: Based on a preset alarm rule, it is detected whether the resource monitoring information of the target job is abnormal; In the case where it is detected that the resource monitoring information is abnormal, alarm information corresponding to the target job is generated, and the alarm information is used to indicate a target abnormal resource type of the target job.

8. The resource monitoring method of claim 7, wherein, After the alarm information corresponding to the target job is generated in the case where it is detected that the resource monitoring information is abnormal, and the alarm information is used to indicate an abnormal resource type of the target job, the method further comprises: Based on the target abnormal resource type, a target resource allocation strategy corresponding to the target abnormal resource type is determined from multiple resource allocation strategies corresponding to different abnormal resource types; The target resource allocation strategy is executed to perform resource allocation adjustment.

9. A resource monitoring platform, characterized in that, The resource monitoring platform is used to execute the resource monitoring method according to any one of claims 1-8, and the resource monitoring platform comprises: A data collection layer is configured to obtain a job identifier of a target job in electronic design automation software and a process identifier of at least one associated process associated with the target job, and establish a mapping relationship table between the target job and the at least one associated process based on the job identifier and the process identifier of the associated process; A unified association layer is configured to monitor multiple resource index information corresponding to at least part of the associated processes based on the mapping relationship table, and obtain resource monitoring information of the target job based on the multiple resource index information corresponding to at least part of the associated processes by establishing an association between the multiple resource index information and the target job; A visualization analysis layer is configured to perform visualization analysis on the resource monitoring information of the target job.

10. A computer program product, characterised in that, The instructions in the computer program product are executed by the processor of the electronic device, and the electronic device executes the resource monitoring method according to any one of claims 1-8.

Images