NPU node scheduling method, device, equipment and medium

Abstract

The invention discloses an NPU node scheduling method and device, equipment and a medium. The method comprises the following steps: monitoring resource occupation information of a plurality of NPU nodes; determining a target NPU node based on the resource occupation information of the plurality of NPU nodes under the condition that a newly added task request end is detected; and binding the target NPU node with the newly added task request end, so that the target NPU node executes a to-be-processed task of the newly added task request end. Through the technical scheme of the invention, the problems of low resource utilization rate and non-uniform load of the NPU node are solved, dynamic allocation and load balance of tasks are realized by sensing the resource state of the NPU node in real time, and the resource utilization rate and stability of the NPU node are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of distributed computing and artificial intelligence hardware scheduling, and particularly to an NPU node scheduling method, apparatus, device and medium. Background Technology

[0002] The Neural Processing Unit (NPU) is a processor specifically designed for artificial intelligence algorithms, providing powerful computing capabilities to support the efficient operation of various AI algorithms. In a computing architecture containing multiple NPU nodes, the rational allocation of NPU node resources directly determines the overall computing power utilization and task processing efficiency.

[0003] In existing server stacking architectures, NPU node resource allocation typically employs either static binding or simple round-robin strategies. In static binding mode, tasks establish a fixed association with a specific NPU node during the initialization phase, and subsequent task processing relies entirely on that node. The simple round-robin strategy, on the other hand, distributes new tasks sequentially to each NPU node according to a preset order, achieving basic load balancing by evenly distributing the number of tasks.

[0004] Most current technologies prevent NPUs from being reused by other tasks when idle, and they cannot dynamically allocate tasks based on real-time load, which can easily lead to problems such as single-node overload. Summary of the Invention

[0005] This invention provides an NPU node scheduling method, apparatus, device, and medium to achieve dynamic task allocation and load balancing, thereby improving NPU node resource utilization and stability.

[0006] According to one aspect of the present invention, an NPU node scheduling method is provided, comprising:

[0007] Monitor resource usage information of multiple NPU nodes;

[0008] Upon detecting a new task request, the target NPU node is determined based on the resource usage information of the multiple NPU nodes;

[0009] The target NPU node is bound to the newly added task request client, so that the target NPU node executes the pending tasks of the newly added task request client.

[0010] According to another aspect of the present invention, an NPU node scheduling apparatus is provided, the apparatus comprising:

[0011] The resource usage monitoring module is used to monitor the resource usage information of multiple NPU nodes;

[0012] The target NPU node determination module is used to determine the target NPU node based on the resource usage information of multiple NPU nodes when a new task request is detected.

[0013] The NPU node binding module is used to bind the target NPU node to the newly added task request client, so that the target NPU node can execute the pending tasks of the newly added task request client.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to execute the NPU node scheduling method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the NPU node scheduling method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer program product is provided, which, when executed by a processor, implements the NPU node scheduling method as described in any of the embodiments of the present invention.

[0020] This invention addresses the issues of low NPU node resource utilization and uneven load by monitoring the resource usage information of multiple NPU nodes. Upon detecting a new task request, a target NPU node is determined based on the resource usage information of the multiple NPU nodes. The target NPU node is then bound to the new task request, enabling the target NPU node to execute the pending tasks of the new task request. This solves the problems of low NPU node resource utilization and uneven load distribution. By real-time sensing of the NPU node's resource status, dynamic task allocation and load balancing are achieved, improving NPU node resource utilization and stability.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of an NPU node scheduling method according to an embodiment of the present invention;

[0024] Figure 2 This is a flowchart of an NPU node scheduling method according to an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of the structure of an NPU node scheduling device according to an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0030] Figure 1 This is a flowchart illustrating an NPU node scheduling method provided in an embodiment of the present invention. This embodiment is applicable to intelligent computing scenarios containing multiple NPU nodes, where reasonable scheduling of artificial intelligence tasks and NPU nodes is required. This method can be executed by an NPU node scheduling device according to an embodiment of the present invention. This device can be implemented in software and / or hardware, and can be integrated into electronic devices such as computer equipment, servers, mobile terminals, or processors. Figure 1 As shown, the method specifically includes the following steps:

[0031] S110: Monitor resource usage information of multiple NPU nodes.

[0032] In this embodiment, an NPU node can be specifically understood as an independent computing unit that processes tasks initiated by a task requester. It integrates a neural network processor core and supporting local storage and communication modules, enabling it to load and run the processing model for the corresponding task. Multiple NPU nodes can be deployed on the same server. Resource occupancy information can be specifically understood as key data reflecting the current operating status and carrying capacity of the NPU node, specifically including but not limited to computing resources and storage resources. By monitoring this information in real time, the idle level of each node can be accurately determined, providing data support for the subsequent selection of target NPU nodes.

[0033] Specifically, monitoring the resource usage information of multiple NPU nodes involves establishing real-time data interaction channels with each NPU node through a preset communication protocol to ensure stable acquisition of node status data. Secondly, each NPU node reports its resource usage, activity status, and bound task request endpoints in real time according to a fixed monitoring period. Optionally, the communication protocol can be MQTT, a lightweight message transmission protocol based on a publish-subscribe model. In this protocol, the NPU node scheduler acts as a message broker, and each NPU node acts as a client. NPU nodes receive resource collection instructions from the NPU node scheduler by subscribing to specific topics and publish the collected resource data to the designated topics, achieving efficient transmission of instructions and data. Alternatively, the communication protocol can be gRPC, a high-performance, open-source remote procedure call framework based on the HTTP / 2 protocol. This framework supports bidirectional streaming communication between clients and servers and features high transmission efficiency and good serialization performance. Meanwhile, to ensure the reliability of the communication link, each NPU node reports information to the NPU node scheduling device at a preset time interval. If the NPU node scheduling device does not receive information from a node within a set timeout period, it determines that the node is abnormal and marks its status to ensure that node offline or link interruption problems can be detected in a timely manner. For example, the preset time interval can be 5 seconds.

[0034] S120. Upon detecting a new task request, determine the target NPU node based on the resource usage information of the multiple NPU nodes.

[0035] In this embodiment, the task requester can be understood as the initiator that needs to call NPU node resources to process a specific task. This can be a hardware device, such as a high-definition camera, a voice interaction terminal, or a smart sensor, or a software application or server. These requesters can send tasks as needed, triggering the NPU node scheduling process. A newly added task requester can be understood as a requester that has not been bound to an NPU node. When a new task requester is added, a suitable NPU node needs to be obtained to process the task. The target NPU node can be understood as an NPU node selected from multiple NPU nodes through a preset scheduling logic that is suitable for the newly added task requester.

[0036] Specifically, when a new task request is detected, the most suitable target NPU node for handling the new task request needs to be determined based on the resource usage information of multiple NPU nodes.

[0037] In some embodiments of this disclosure, determining a target NPU node based on the resource usage information of multiple NPU nodes includes: comparing the resource usage information of multiple NPU nodes and determining the NPU node corresponding to the minimum resource usage information as the target NPU node.

[0038] Specifically, when a new task request is detected, the NPU node with the lowest resource usage is determined based on the current resource usage information of all NPU nodes, and that node is directly designated as the target NPU node for processing the new task.

[0039] In some embodiments of this disclosure, determining a target NPU node based on the resource usage information of multiple NPU nodes includes: determining the comprehensive resource usage information of each server based on the dependency relationship between the NPU nodes and the server, wherein each server includes multiple NPU nodes; determining a target server based on the comprehensive resource usage information of the server; determining a target NPU node based on the idle status of multiple NPU nodes in the target server; or determining the NPU node corresponding to the minimum resource usage information in the target server as the target NPU node.

[0040] In this embodiment, NPU nodes are deployed on servers, and at least one NPU node can be deployed on the same server. Comprehensive resource utilization information can be specifically understood as a comprehensive set of information reflecting the overall resource and availability of a server containing multiple NPU nodes, formed by integrating the resources of all NPU nodes within it. Comprehensive resource utilization information may include, but is not limited to, the average resource utilization rate of all NPU nodes within the server and the percentage of NPU nodes in normal operating condition. Comprehensive resource utilization information can intuitively reflect the overall resource utilization of the server, providing a global reference for determining which server is more suitable for handling new tasks, avoiding the problem of global imbalance caused by local optima based solely on the status of a single NPU node. The target server can be specifically understood as, based on the logic of the NPU node-server dependency relationship, the server selected from multiple servers whose overall resource status is adapted to the requirements of the new task request.

[0041] Specifically, when a new task request is detected, a hierarchical filtering method is used to determine the target NPU node suitable for that new task request. Specifically, based on the dependency relationship between NPU nodes and servers, the resource usage information of multiple NPU nodes within each server is integrated and calculated to generate comprehensive resource usage information for each server. By comparing the comprehensive resource usage information of each server, the server with the lowest comprehensive resource usage information is selected. At least one NPU node set in the target server is further filtered: if an idle NPU node exists, it is directly determined as the target NPU node; if no idle node exists, the NPU node with the lowest resource usage within that server is selected as the target NPU node. Through one of these two paths, the target NPU node adapted to the new task request is finally determined. These two filtering logics can quickly match a resource-suitable target NPU node for a new task request, ensuring efficient task startup, avoiding excessive load on a single node, balancing resources at the server level, and improving the resource utilization and operational stability of the entire NPU node cluster.

[0042] S130. Bind the target NPU node to the newly added task request client so that the target NPU node executes the pending task of the newly added task request client.

[0043] In this embodiment, binding can be understood as establishing a unique and identifiable association between the target NPU node and the new task requesting client, forming a dedicated task data transmission and processing link. The identification information of the task requesting client is associated and stored with the identification information of the target NPU node to ensure that subsequent tasks sent by the requesting client are accurately transmitted to the target NPU node. Binding the task requesting client to the target NPU node ensures that the tasks to be processed can be executed stably and efficiently on the target node.

[0044] Specifically, after identifying the target NPU node, a binding record is created between the new task requester identifier and the target NPU node identifier. Once the binding is complete, the target NPU node will receive and process the pending tasks sent by the requester, and will maintain dedicated resource support for the requester during the binding period, ensuring the continuity and stability of task processing.

[0045] Optionally, when the requesting end sends the task data to be processed, it first submits it to the NPU node scheduling device. The NPU node scheduling device queries the target NPU node based on the binding relationship and then sends the task data to be processed to the bound NPU node. Alternatively, the NPU node scheduling device sends the communication address information of the target NPU node to the task requesting end, and the task requesting end can directly send the data to be processed to the target NPU node, reducing the number of intermediate links.

[0046] Optionally, Redis-based cache consistency management can be used for binding, storing the binding relationship with the task requester identifier as the key and the target NPU node identifier as the value, enabling fast read, write, and query of binding information. This key-value pair storage method ensures that the NPU node scheduler can instantly locate the corresponding NPU node using the task requester identifier when needed, while also guaranteeing the consistency and reliability of bound data across multiple scenarios, avoiding information conflicts or loss.

[0047] The technical solution of this embodiment can monitor the resource usage information of multiple NPU nodes. When a new task request is detected, the resource usage information of the multiple NPU nodes is compared, and a target NPU node is determined based on the resource usage information of the multiple NPU nodes. The target NPU node is then bound to the new task request, so that the target NPU node executes the pending task of the new task request. This technical solution can determine a suitable NPU node based on resource usage information, ensuring efficient task startup when a new task request is detected, realizing dynamic task allocation and load balancing, and improving the resource utilization and stability of NPU nodes.

[0048] Figure 2This is a flowchart of an NPU node scheduling method provided by an embodiment of the present invention. This embodiment refines the above-mentioned optional methods, such as... Figure 2 As shown, the method specifically includes the following steps:

[0049] S210: Monitor resource usage information of multiple NPU nodes.

[0050] S220. Upon detecting a new task request, determine the target NPU node based on the resource usage information of the multiple NPU nodes.

[0051] S230: Record the binding relationship between the NPU node and the task requester.

[0052] Specifically, the system records the identifier information of the target NPU node corresponding to the identifier information of the task requester, ensuring that subsequent tasks sent by the requesting client are accurately transmitted to the target NPU node. Optionally, Redis-based cache consistency management can be used for binding, storing the binding relationship with the task requester identifier as the key and the target NPU node identifier as the value, enabling fast reading, writing, and querying of binding information.

[0053] S240. If a new or modified task type is detected at the task request end, based on the binding relationship between the NPU node and the task request end, the target NPU node corresponding to the task request end is determined, and the new or modified task type is distributed to the target NPU node so that the target NPU node executes the pending task of at least one task type of the task request end.

[0054] In this embodiment, when an NPU node establishes a binding relationship with a task requester, the types of tasks to be processed on the task requester are limited. These task types can be set according to actual needs, and there must be at least one task type. When business requirements change, the task types on the task requester can be flexibly adjusted, allowing for the addition, modification, and deletion of task types. Through this dynamic adjustment mechanism, the processing capacity of the NPU node is always matched with the actual needs of the task requester.

[0055] Specifically, when a new task type is detected from a task requesting client, the NPU node scheduling device queries the target NPU node corresponding to the requesting client through the recorded binding relationship, and evaluates the resource requirements of the new task type against the remaining resources of the target NPU node. If the resources of the NPU node meet the requirements, the new task type is added to that NPU node; if the resources of the NPU node are insufficient, the process for the new task requesting client is repeated, and other suitable NPU nodes are selected to execute the new task type, while maintaining the original binding relationship unchanged.

[0056] When a modified task type is detected on the requesting end, the target NPU node is located based on the current binding relationship. The processing of the task type to be replaced on that NPU node is first terminated, and the corresponding resources are released. Then, the modified task type is installed. If the modified task type exceeds the resource limit of that node, the original binding relationship is released, and a suitable target NPU node is re-matched to the requesting end to ensure the modified task can execute normally. For example, the task type of an NPU node can be changed from image classification to image segmentation. The image classification task on that NPU node is first terminated, and then the image segmentation task is installed. Accurately determining the bound target NPU node for newly added or modified task types ensures both the continuity and focus of task processing, and allows for rapid resource adaptation based on existing binding relationships, improving task scheduling efficiency and processing stability.

[0057] When a task requester detects a task type to be deleted, the NPU node scheduling device queries the target NPU node corresponding to the requester through the recorded binding relationship, terminates the processing process related to the task type to be deleted on that node, updates the binding relationship record between the node and the requester, removes the deleted task type identifier, and ensures efficient utilization of NPU node resources.

[0058] It should be noted that instructions to add or modify task types can be sent from a remote location to the NPU node scheduling device. The NPU node scheduling device responds to these instructions, identifies the target NPU node, and updates the scheduling of the NPU node for the new or modified task type. For example, the remote location could be a mobile phone, computer, or tablet. For instance, if a smart monitoring system needs to add a new task type for device anomaly sound recognition, the remote location sends the instruction to the NPU node scheduling device. The NPU node scheduling device then distributes the task to the corresponding NPU node based on the binding relationship, enabling the node to execute the new device anomaly sound recognition task.

[0059] Optionally, based on the above embodiments, the method further includes: for a target NPU node that has been bound to a task request client, if the resource usage information of the target NPU node exceeds a set threshold or the target NPU node is abnormal, a new target NPU node is determined, and a binding relationship is established between the task request client and the new target NPU node, so that the new target NPU node executes at least one task type of pending task from the task request client.

[0060] In this embodiment, the threshold setting can be understood as a preset threshold for the target NPU node of the bound task request end, used to determine whether its resources are overloaded or whether it has normal processing capabilities. It is the core judgment criterion for triggering node rescheduling.

[0061] Specifically, setting a threshold can be... This refers to the upper limit threshold for resource usage. When the resource usage information of a target NPU node exceeds the threshold, a new target NPU node is determined, and a binding relationship is established between the task requester and the new target NPU node. This allows the new target NPU node to execute at least one task of the task requester's pending tasks. Alternatively, if a target NPU node is abnormal, a binding relationship is established between the task requester and the new target NPU node, allowing the new target NPU node to execute at least one task of the task requester's pending tasks. Specifically, if the NPU node scheduling device does not receive information from a node within a set timeout period, it determines that the node is in an abnormal state. For example, the preset timeout period can be 5 seconds. If the NPU node scheduling device does not receive information from a node after 5 seconds, it can determine that the node is in an abnormal state. Determining a new target NPU node and binding relationship when the target NPU node's resource usage information exceeds the set threshold or when the target NPU node is abnormal improves failure recovery capabilities, ensures the continuous and stable execution of pending tasks on the task requester, avoids business interruption due to a single node problem, and further enhances the overall reliability and fault tolerance of the system.

[0062] The frame rate of an NPU node refers to the number of task data frames that an NPU node can process per unit of time, usually measured in frames per second (fps). For example, an NPU node with a frame rate of 30fps processing an image recognition task means it can complete inference calculations for 30 frames of images per second. The frame rate directly reflects the performance of the NPU; a higher frame rate indicates higher efficiency in processing continuous data, supporting higher resolution or more complex real-time tasks. The frame rate interval refers to the time interval between two adjacent frames of data successfully processed by the NPU node, usually measured in milliseconds (ms), and is inversely related to the frame rate. For example, a frame rate interval of approximately 33ms corresponds to 30fps, meaning one frame of data is processed every 33ms. The frame rate interval more intuitively reflects the real-time nature of task processing; a smaller interval indicates lower data processing latency, better suited for scenarios with high real-time requirements. This refers to the basic resource requirements for task processing. Different tasks have different requirements for real-time performance and processing complexity, and will correspond to different values.

[0063] Optionally, the target NPU node includes a processing model for executing the task to be processed, and different processing models correspond to different task types; the method further includes: traversing the NPU nodes, determining a model loading node based on the resource usage information of multiple NPU nodes, loading the processing model corresponding to each task type to the model loading node, and broadcasting the model distribution information to all NPU nodes, so that the NPU node loads the required processing model from the model loading node.

[0064] In this embodiment, the processing model for the task to be processed can be specifically understood as a dedicated algorithm model that is pre-trained and compiled for different types of tasks and can run on NPU nodes. These models are the core of the NPU nodes' task execution, and are essentially packages containing specific algorithm logic, weight parameters, and computational processes, capable of efficiently completing task processing using the hardware acceleration capabilities of the NPU. For example, for image classification tasks, the processing model can be a deep learning model based on the ResNet architecture, used to extract image features and output classification results; for speech recognition tasks, it may be a combination of an acoustic model and a language model based on Transformer, used to convert audio signals into text. The model distribution information can be specifically understood as an information packet generated and broadcast to all NPU nodes after loading the processing models for each task type into the model loading node, used to guide the nodes to obtain the required model. The model distribution information may include the identifier of the model loading node storing the model and the task type corresponding to the model. The model loading node can be specifically understood as a specific NPU node selected from multiple NPU nodes, specifically used to store the processing models for the corresponding processing tasks, and plays a core role in the centralized storage and distribution of models.

[0065] Specifically, after determining the model loading node based on the resource usage information of multiple NPU nodes, the processing model corresponding to each task type is loaded onto the model loading node, and the model distribution information is broadcast to all NPU nodes, enabling NPU nodes to load the required processing model from the model loading node. Centralized model storage avoids resource waste caused by repeated loading across the entire cluster, while allowing each NPU node to quickly obtain the required model to accelerate task startup, simplifying model update management, and ensuring consistency of models used by NPU nodes.

[0066] Optionally, based on the above embodiments, the method further includes: traversing the NPU nodes and synchronizing alarm parameter configuration information to each NPU node, so that when the NPU node performs a task to be processed and meets the alarm conditions, it generates alarm information based on the alarm parameter configuration information.

[0067] In this embodiment, the alarm parameter configuration information can be understood as system-preset information used by NPU nodes to determine whether an abnormality has occurred in the task processing process and to trigger an alarm. This includes, but is not limited to, alarm triggering conditions, alarm level classification, and the identifier of the faulty NPU node. Alarm triggering conditions may include tasks processing delays exceeding a specified time, excessive resource consumption by the NPU node, etc., and alarm levels can be classified according to these triggering conditions.

[0068] Specifically, all NPU nodes are traversed, and alarm parameter configuration information is synchronized to each node through a unified configuration synchronization channel. This information includes, but is not limited to, alarm triggering conditions and alarm level classifications. For example, an alarm is triggered if the task processing delay exceeds 300ms three times consecutively, or if the NPU core temperature exceeds 85℃, a high-level alarm is triggered. After receiving the alarm parameter configuration information, each NPU node continuously collects its own operating status and task processing data while executing pending tasks. When a certain indicator is detected to meet the alarm condition, the node will immediately generate alarm information according to a preset format and send it to the corresponding receiving terminal through a specified path, thereby enabling timely detection and reporting of task execution anomalies and facilitating rapid system intervention.

[0069] The technical solution of this embodiment can monitor the resource usage information of multiple NPU nodes. When a new task request is detected, a target NPU node is determined based on the resource usage information of the multiple NPU nodes. Next, the binding relationship between the NPU nodes and the task request is recorded. When a new or modified task type is detected in the task request, the target NPU node corresponding to the task request is determined based on the binding relationship between the NPU nodes and the task request. The new or modified task type is then distributed to the target NPU node, enabling the target NPU node to execute at least one pending task of the task request. This technical solution can accurately match the target NPU node of a new task request, avoiding resource waste; simultaneously, it quickly adapts to the needs of adding or modifying task types through binding relationships, achieving dynamic binding, ensuring the targeting and continuity of task processing, and improving overall scheduling efficiency.

[0070] Based on the above embodiments, an optional example is provided, which can be used in a smart mine AI intelligent video analysis scenario. The task requesting end includes a camera, which can be deployed at locations such as the mine entrance, mine exit, or underground working area, depending on monitoring needs. The images / videos collected by the camera serve as data to be processed. Each camera's task to be processed includes at least one image processing task. For example, the tasks to be processed for cameras deployed at the mine entrance and exit may include face recognition and vehicle recognition, while the tasks to be processed for cameras deployed in the underground working area may include environmental perception tasks, such as identifying anomalies in the underground working area.

[0071] In this embodiment, the starting and stopping of cameras, and the addition, modification, and deletion of task types can be controlled by a remote device. The remote device sends a camera start command to the NPU node scheduling device as needed. This start command includes the device information of the camera to be started. When the NPU node scheduling device detects a new camera, for example, adding a camera at a mine entrance, it needs to compare the resource usage information of multiple NPU nodes and determine the NPU node corresponding to the minimum resource usage information as the target NPU node. Alternatively, based on the dependency relationship between NPU nodes and servers, it determines the comprehensive resource usage information of each server, determines the target server based on the comprehensive resource usage information of the servers, and determines the target NPU node based on the idle status of multiple NPU nodes in the target server. Alternatively, it determines the NPU node corresponding to the minimum resource usage information in the target server as the target NPU node. After determining the target NPU node, the target NPU node is bound to the newly added camera so that the target NPU node executes the image processing task of the newly added camera.

[0072] The remote device sends task type adjustment instructions to the NPU node scheduling device as needed. These instructions include task type addition instructions, task type modification instructions, and task type deletion instructions. The aforementioned task type adjustment instructions may include the camera's device information and task adjustment information.

[0073] The NPU node scheduling device analyzes the above-mentioned task type adjustment instructions. When a new task type is added to the camera, for example, when a new environmental anomaly detection task is added to camera A at the mine entrance, which identifies whether there are environmental anomalies such as water accumulation, falling rocks, or smoke in the mine entrance area through image recognition, the NPU node scheduling device determines the target NPU node corresponding to the camera based on the binding relationship between the NPU node and camera A, and distributes the new environmental anomaly detection task to the target NPU node so that the target NPU node can execute this task.

[0074] The NPU node scheduling device analyzes the above-mentioned task type adjustment instructions. When the task type of camera B is modified, for example, when the work area changes from the personnel exploration stage to the equipment-led mining stage, camera B deployed in the mine work area needs to change the personnel detection task to the equipment operation status detection task. Based on the binding relationship between the NPU node and camera B, the target NPU node corresponding to camera B is determined, and the modified task is distributed to the target NPU node so that the target NPU node can execute this task.

[0075] The NPU node scheduling device parses the above task type adjustment instructions. When deleting the task type of camera C, for example, when the mining operation is completed in the work area, the camera C deployed in the mining work area needs to delete the equipment operation status detection task. Based on the binding relationship between the NPU node and camera C, the target NPU node corresponding to camera C is determined, and the instruction to delete the task type is sent to the target NPU node so that the target NPU node terminates the execution of the task and releases resources.

[0076] Figure 3 This is a schematic diagram of an NPU node scheduling device provided in an embodiment of the present invention. This embodiment is applicable to intelligent computing scenarios containing multiple NPU nodes, where reasonable scheduling of artificial intelligence tasks and NPU nodes is required. The device can be implemented in software and / or hardware, and can be integrated into any device that provides NPU node scheduling functionality, such as… Figure 3 As shown, the NPU node scheduling device specifically includes: an occupancy information monitoring module 310, a target NPU node determination module 320, and an NPU node binding module 330.

[0077] The resource occupancy monitoring module 310 is used to monitor the resource occupancy information of multiple NPU nodes;

[0078] The target NPU node determination module 320 is used to determine the target NPU node based on the resource occupancy information of multiple NPU nodes when a new task request terminal is detected.

[0079] The NPU node binding module 330 is used to bind the target NPU node to the newly added task request client, so that the target NPU node executes the pending tasks of the newly added task request client.

[0080] The technical solution of this embodiment can monitor the resource usage information of multiple NPU nodes. When a new task request is detected, the resource usage information of the multiple NPU nodes is compared, and a target NPU node is determined based on the resource usage information of the multiple NPU nodes. Finally, the target NPU node is bound to the new task request, so that the target NPU node executes the pending task of the new task request. This technical solution can determine a suitable NPU node based on resource usage information, ensuring efficient task startup when a new task request is detected, realizing dynamic task allocation and load balancing, and improving the resource utilization and stability of NPU nodes.

[0081] Based on the above embodiments, optionally, the target NPU node determination module 320 is used to compare the resource usage information of multiple NPU nodes and determine the NPU node corresponding to the minimum resource usage information as the target NPU node; or, based on the subordinate relationship between the NPU node and the server, determine the comprehensive resource usage information of each server, wherein each server includes multiple NPU nodes; determine the target server based on the comprehensive resource usage information of the server, determine the target NPU node based on the idle status of multiple NPU nodes in the target server, or determine the NPU node corresponding to the minimum resource usage information in the target server as the target NPU node.

[0082] Based on the above embodiments, optionally, the NPU node binding module 330 includes a binding relationship determination unit and a task type distribution unit.

[0083] The binding relationship determination unit is used to record the binding relationship between the NPU node and the task requester;

[0084] The task type distribution unit is used to determine the target NPU node corresponding to the task requesting end based on the binding relationship between the NPU node and the task requesting end when a new or modified task type is detected at the task requesting end, and distribute the new or modified task type to the target NPU node so that the target NPU node can execute the pending task of at least one task type of the task requesting end.

[0085] Optionally, based on the above embodiments, the device further includes a target node re-determination module, used to re-determine a new target NPU node for a target NPU node that has been bound to a task request client, if the resource usage information of the target NPU node exceeds a set threshold or the target NPU node is abnormal, and establish a binding relationship between the task request client and the new target NPU node, so that the new target NPU node can execute at least one task type of pending task of the task request client.

[0086] Optionally, based on the above embodiments, the device further includes a model distribution information broadcasting module, which is used to traverse the NPU nodes, determine the model loading node based on the resource occupancy information of multiple NPU nodes, load the processing model corresponding to each task type to the model loading node, and broadcast the model distribution information to all NPU nodes so that the NPU nodes can load the required processing model from the model loading node.

[0087] Based on the above embodiments, optionally, the device further includes an alarm parameter configuration information synchronization module, used to traverse the NPU nodes and synchronize alarm parameter configuration information to each NPU node, so that when the NPU node meets the alarm conditions during the execution of the task to be processed, it generates alarm information based on the alarm parameter configuration information.

[0088] Based on the above embodiments, optionally, the task requesting end includes a camera device; the task to be processed by the camera device includes an image processing task.

[0089] The above-described products can perform the methods provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects for performing the methods.

[0090] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0091] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0092] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0093] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as NPU node scheduling methods.

[0094] In some embodiments, the NPU node scheduling method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the NPU node scheduling method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the NPU node scheduling method by any other suitable means (e.g., by means of firmware).

[0095] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0096] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0097] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0098] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0099] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0100] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0101] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0102] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the NPU node scheduling method according to any embodiment of the invention.

[0103] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An NPU node scheduling method, characterized in that, include: Monitor resource usage information of multiple NPU nodes; Upon detecting a new task request, the target NPU node is determined based on the resource usage information of the multiple NPU nodes; The target NPU node is bound to the newly added task request client, so that the target NPU node executes the pending tasks of the newly added task request client.

2. The method according to claim 1, characterized in that, The target NPU node is determined based on the resource usage information of multiple NPU nodes, including: By comparing the resource usage information of multiple NPU nodes, the NPU node corresponding to the minimum resource usage information is determined as the target NPU node; or, Based on the hierarchical relationship between NPU nodes and servers, the comprehensive resource usage information of each server is determined, wherein each server includes multiple NPU nodes; a target server is determined based on the comprehensive resource usage information of the server, and a target NPU node is determined based on the idle status of multiple NPU nodes in the target server, or the NPU node corresponding to the minimum resource usage information in the target server is determined as the target NPU node.

3. The method according to claim 1, characterized in that, The method further includes: recording the binding relationship between the NPU node and the task requesting end; Furthermore, if a new or modified task type is detected at the task request end, based on the binding relationship between the NPU node and the task request end, the target NPU node corresponding to the task request end is determined, and the new or modified task type is distributed to the target NPU node so that the target NPU node executes the pending task of at least one task type from the task request end.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: If the resource usage information of the target NPU node that is already bound to the task request client exceeds a set threshold or the target NPU node is abnormal, a new target NPU node is determined, and a binding relationship is established between the task request client and the new target NPU node, so that the new target NPU node can execute at least one task type of pending task from the task request client.

5. The method according to claim 1, characterized in that, The target NPU node includes a processing model for executing the task to be processed, and different processing models correspond to different task types. The method further includes: The NPU nodes are traversed, and the model loading node is determined based on the resource usage information of multiple NPU nodes. The processing model corresponding to each task type is loaded into the model loading node, and the model distribution information is broadcast to all NPU nodes so that the NPU nodes can load the required processing model from the model loading node.

6. The method according to claim 1, characterized in that, The method further includes: The NPU nodes are traversed, and alarm parameter configuration information is synchronized to each NPU node so that when the NPU node performs a task to be processed and the alarm conditions are met, alarm information is generated based on the alarm parameter configuration information.

7. The method according to claim 1, characterized in that, The task requesting end includes a camera device; the tasks to be processed by the camera device include image processing tasks.

8. An NPU node scheduling device, characterized in that, include: The resource usage monitoring module is used to monitor the resource usage information of multiple NPU nodes; The target NPU node determination module is used to determine the target NPU node based on the resource usage information of multiple NPU nodes when a new task request is detected. The NPU node binding module is used to bind the target NPU node to the newly added task request client, so that the target NPU node can execute the pending tasks of the newly added task request client.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the NPU node scheduling method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the NPU node scheduling method according to any one of claims 1-7.

Images