Method and device for calling NPU across devices, electronic device and storage medium
Through NPU virtual services and distributed hardware management services, NPU resources are scheduled across devices, and the delay problem under the combined end-cloud method is solved, NPU resource sharing between terminal devices is realized, and AI task processing efficiency and latency performance are improved.
Patent Information
- Application Number
- CN202411023710.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-07-29
AI Technical Summary
The existing end-cloud combination method relies on the network when processing neural network tasks, resulting in delays and cannot meet all working conditions, especially the NPU specifications of the end-side device cannot meet the high computing power requirements.
Through NPU virtual services, the NPU of the terminal devices in the network is converted into NPU service objects, and the NPU resource call across devices is realized through distributed hardware management services registration and scheduling, allowing the target terminal devices to select appropriate NPU service objects according to task requirements. Perform processing.
It realizes efficient utilization of NPUs in the network equipment, reduces the delay in processing tasks, and enables terminal devices that do not have NPUs or lack of NPU computing power to complete complex AI task processing with the help of NPUs of other devices.
Smart Images

Figure CN119031029A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of neural network processor technology, and in particular to a method, device, electronic device and storage medium for calling an NPU across devices. Background Art
[0002] The Neural Processing Unit (NPU) is specially designed to achieve low power consumption and parallel acceleration of artificial intelligence (AI) reasoning. It is specially optimized for matrix operations and plays an important role in multiple AI fields such as natural language processing and machine vision. Due to the rapid development of generative AI, more and more end-side devices are now equipped with NPUs and large models, giving devices AI capabilities.
[0003] Edge-side AI capabilities, such as large models and video stream processing, require relatively high NPU computing power. However, due to space, power consumption, and cost constraints, the NPU specifications of edge-side devices cannot meet these requirements.
[0004] The traditional solution is to combine the end and the cloud, that is, simple tasks are processed locally, and for more complex tasks, the data is sent to the cloud for processing, and the results are returned to the end side to the user. However, this strategy of combining the end and the cloud also has some disadvantages, such as dependence on the network and high latency, and cannot meet all working conditions. Summary of the invention
[0005] The present invention provides a method, an apparatus, an electronic device and a storage medium for calling NPU across devices, so as to enable a terminal device in a network to use the NPU of other devices and reasonably utilize NPU resources.
[0006] According to one aspect of the present invention, a method for calling an NPU across devices is provided, comprising:
[0007] Converting an NPU of a terminal device in a network into an NPU service object through an NPU virtual service, wherein the number of terminal devices in the network is at least two, and at least one of the terminal devices in the network has an NPU;
[0008] By calling the registration interface of the distributed hardware management service through the NPU virtual service, the NPU service object of each terminal device is registered to the distributed hardware management service;
[0009] When the target terminal in the network receives a task to be processed, the target information of each NPU service object in the network is queried through the distributed hardware management service, and a target NPU service object for processing the task to be processed is determined based on the target information.
[0010] According to another aspect of the present invention, there is provided an apparatus for calling an NPU across devices, comprising:
[0011] A conversion module, used to convert an NPU of a terminal device in a network into an NPU service object, wherein the number of terminal devices in the network is at least two, and at least one of the terminal devices in the network has an NPU;
[0012] A registration module, used to call the registration interface of the distributed hardware management service through the NPU virtual service, and register the NPU service object of each terminal device to the distributed hardware management service;
[0013] The calling module is used to query the target information of each NPU service object in the network through the distributed hardware management service when the target terminal in the network receives the task to be processed, and determine the target NPU service object for processing the task to be processed based on the target information.
[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
[0015] at least one processor;
[0016] and a memory communicatively connected to the at least one processor; wherein,
[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for calling NPU across devices described in any embodiment of the present invention.
[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for calling an NPU across devices described in any embodiment of the present invention when executed.
[0019] The technical solution of the embodiment of the present invention converts the NPU of the terminal device in the network into an NPU service object through the NPU virtual service, wherein the number of terminal devices in the network is at least two, and the terminal devices in the network are equipped with NPU; through the NPU virtual service, the registration interface of the distributed hardware management service is called to register the NPU service object of each terminal device to the distributed hardware management service; when the target terminal in the network receives the task to be processed, the target information of each NPU service object in the network is queried through the distributed hardware management service, and the target NPU service object for processing the task to be processed is determined based on the target information. The technical solution of the present invention solves the problem that when processing neural network tasks, the current end-cloud combination method relies on the network, has high latency, and cannot meet all working conditions. Through the distributed hardware management service, the task to be processed of the current terminal device is scheduled to the NPU service object of other terminal devices in the network, so that terminal devices without NPU or insufficient NPU computing power can use the NPU of other terminal devices, realizing efficient utilization of the NPU of the device in the network and reducing the latency of processing tasks.
[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0022] Figure 1 A flowchart of a method for calling NPU across devices provided in Embodiment 1 of the present invention;
[0023] Figure 2 An architecture diagram of a cross-device NPU calling system provided in Embodiment 2 of the present invention;
[0024] Figure 3 A flowchart of a method for calling NPU across devices provided in Embodiment 2 of the present invention;
[0025] Figure 4 A schematic diagram of the structure of an apparatus for calling NPU across devices provided in Embodiment 3 of the present invention;
[0026] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION
[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0029] Embodiment 1
[0030] Figure 1 This is a flowchart of a method for calling NPU across devices provided in the first embodiment of the present invention. This embodiment is applicable to the case where the current terminal device in the network calls the NPU resources of other terminal devices. The method can be executed by a device for calling NPU across devices. The device can be implemented in the form of hardware and / or software, and the device can be configured in the terminal device. Figure 1 As shown, the method includes:
[0031] S110. Convert the NPU of the terminal device in the network into an NPU service object through the NPU virtual service.
[0032] Among them, the NPU virtual service is used to convert the functions of the NPU hardware of the terminal device into a service form that can be called through the network, that is, the NPU service object; networking can be understood as building a communication network between terminal devices. The number of terminal devices in the networking can be two or more. At least one of the terminal devices in the networking has an NPU. The networking can include terminal devices with NPU and terminal devices without NPU.
[0033] Specifically, the NPU hardware resources of the terminal devices in the network can be abstracted through the NPU virtual service to obtain a service form that can be called through the network, that is, the NPU service object.
[0034] In an embodiment of the present invention, converting the NPU of a terminal device in a network into an NPU service object through an NPU virtual service includes: deploying an NPU virtual service in the terminal device in the network, abstracting the NPU hardware of the terminal device through the NPU virtual service to obtain the abstracted NPU function; defining a target interface, and encapsulating the target interface and the abstracted NPU function into the NPU service object.
[0035] Specifically, the NPU virtual service is deployed in the terminal devices within the network. The NPU virtual service can abstract the NPU hardware of the terminal devices within the network, that is, identify and extract the core functions and features of the NPU. Next, a set of standard interfaces, namely the target interface, is defined to shield the differences between different NPU hardware and manufacturer SDKs. Further, the abstracted NPU function and the defined standard interface are encapsulated into an NPU service object. The encapsulated NPU service object can be exposed to other terminal devices in the network through a soft bus or other private custom protocols. Other terminal devices can connect to this device through the network and call the NPU service object provided by it.
[0036] like Figure 2 As shown, it is an architecture diagram of a cross-device NPU calling system provided in Embodiment 2 of the present invention. The system includes two terminal devices, device 1 and device 2 each have an NPU, and distributed hardware management services and NPU virtual services (NPU service objects) are deployed on both devices 1 and 2.
[0037] S120. Calling a registration interface of a distributed hardware management service through the NPU virtual service to register the NPU service object of each terminal device to the distributed hardware management service.
[0038] Among them, the distributed hardware management service provides registration, deregistration and query interfaces. The registration interface is used to register NPU service objects, the deregistration interface is used to deregister NPU service objects, and the query interface is used to query the information of each NPU service object.
[0039] Specifically, when the terminal device starts, the registration interface of the distributed hardware management service is called through the NPU virtual service, and the NPU service object is registered to the distributed hardware management service through the interface, so that the distributed hardware management service can manage and allocate the NPU service object.
[0040] In the embodiment of the present invention, after registering the NPU service object of each terminal device to the distributed hardware management service, it also includes: updating the target information of each NPU service object in real time through the distributed hardware management service.
[0041] The target information includes at least one of computing power information and load information.
[0042] Specifically, the distributed hardware management service is responsible for managing all information of the NPU service object, such as computing power, device, current operating status, etc. The distributed hardware management service can obtain and update this information in real time, and provide a query interface for the local machine and other devices in the network to query and obtain.
[0043] S130. When a target terminal in the network receives a task to be processed, the target information of each NPU service object in the network is queried through the distributed hardware management service, and a target NPU service object for processing the task to be processed is determined based on the target information.
[0044] Among them, the target terminal can be a terminal device that receives the task to be processed, the task to be processed can be an AI task, and the target NPU service object can be an NPU service object used to execute the task to be processed.
[0045] Specifically, the client on the target terminal can query the target information of the NPU service objects of each terminal device in the network through the distributed hardware management service, and select a suitable NPU service object as the target NPU service object according to the task requirements and load conditions. Then, the target NPU service object is called to process the AI task.
[0046] Preferably, the communication mode between the terminal devices in the network includes at least one of soft bus, Bluetooth, Wi-Fi and Star Flash.
[0047] Specifically, the terminal devices in the network can call the NPU service objects of other terminals in the network through the above-mentioned communication method, without uploading the data related to the task to be processed to the cloud for processing through long-distance communication, thereby reducing network latency, improving task execution efficiency, and meeting the real-time requirements of AI task processing.
[0048] The technical solution of the embodiment of the present invention converts the NPU of the terminal device in the network into an NPU service object through the NPU virtual service, wherein the number of terminal devices in the network is at least two, and the terminal devices in the network are equipped with NPU; through the NPU virtual service, the registration interface of the distributed hardware management service is called to register the NPU service object of each terminal device to the distributed hardware management service; when the target terminal in the network receives the task to be processed, the target information of each NPU service object in the network is queried through the distributed hardware management service, and the target NPU service object for processing the task to be processed is determined based on the target information. The technical solution of the present invention solves the problem that when processing neural network tasks, the current end-cloud combination method relies on the network, has high latency, and cannot meet all working conditions. Through the distributed hardware management service, the task to be processed of the current terminal device is scheduled to the NPU service object of other terminal devices in the network, so that terminal devices without NPU or insufficient NPU computing power can use the NPU of other terminal devices, realizing efficient utilization of the NPU of the device in the network and reducing the latency of processing tasks.
[0049] Embodiment 2
[0050] Figure 3 This is a flowchart of a method for calling NPU across devices provided in the second embodiment of the present invention. This embodiment further refines the processing process of the task to be processed on the basis of the above embodiment. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above embodiment are not repeated here. Figure 3 As shown, the method includes:
[0051] S210: Convert the NPU of the terminal device in the network into an NPU service object through the NPU virtual service.
[0052] Therein, the number of terminal devices in the network is at least two, and at least one of the terminal devices in the network is equipped with an NPU.
[0053] S220. Calling a registration interface of a distributed hardware management service through the NPU virtual service to register the NPU service object of each terminal device to the distributed hardware management service.
[0054] S230: When a target terminal in the network receives a task to be processed, the target information of the NPU service object corresponding to each terminal device in the network is queried through the distributed hardware management service.
[0055] S240. Select a target NPU service object from the plurality of NPU service objects based on the target information.
[0056] Among them, the terminal device corresponding to the target NPU service object is the terminal to be used, and the terminal to be used can be a device different from the target terminal. For example, the target terminal does not have an NPU and can also complete the AI task through a cross-device method using the target NPU service object of the terminal to be used.
[0057] It is understandable that there may be multiple terminal devices equipped with NPU in the network. By converting the NPU hardware of each terminal device into an NPU service object, multiple NPU service objects can be obtained, and the resources corresponding to each NPU service object are also different. Therefore, according to different tasks, computing resources can be reasonably scheduled to maximize the use of examples. For example, according to the computing power required for the current task and the NPU load, select a suitable NPU service object as the target NPU service object to run the current task.
[0058] S250, determining the interface name and input parameters of the task to be processed, and passing in the interface name and the input parameters, so as to process the task to be processed through the target NPU service object and obtain a processing result of the task to be processed.
[0059] Specifically, the client on the target terminal queries the virtual NPU information of the device in the network through the distributed hardware management service, obtains one of the NPU service objects based on this information, passes in the interface name and input parameters, and calls the distributed service through the object. After receiving the interface name and input parameters, the target NPU service object will process the task according to the preset logic, and when the task processing is completed, the target NPU service object will return the processing result.
[0060] In an embodiment of the present invention, it also includes: when the target NPU service object is called for the first time, a target identifier is assigned to the client that calls the target NPU service object, and context information is saved based on the target identifier; when the client repeatedly calls the target NPU service object, the saved context information is obtained based on the target identifier, and the pending task is processed based on the context information.
[0061] The context information includes input data, intermediate results and final results.
[0062] Specifically, when the client calls the NPU service object for the first time, the system assigns it a unique sessionId, which is the target identifier. The server maintains the context of each call based on the sessionId, including input data, intermediate results, and final results. Through the sessionId, the server can maintain data consistency between multiple calls to ensure the accuracy and completeness of the calculation. The server provides multiple interfaces, multiple calls, and multi-threaded calls. In different calls, the server determines which group of calls it belongs to based on the passed in sessinId, so as to find the context saved in the previous call and obtain the intermediate data generated by the previous call, ensuring that the environment used is consistent and the data is complete and available. For each call, verify the identity of the caller to see if he is the creator of the context.
[0063] S260: Feedback the processing result to the target terminal.
[0064] Specifically, the processing result may be fed back to the client on the target terminal.
[0065] In this way, the devices in the network can use the NPU capabilities of other devices, reasonably schedule computing resources according to different tasks, and maximize the use of examples. For example, according to the computing power required for the current task and the NPU load, select the appropriate NPU resources to run the current task. Implementation: 1. Devices without NPU can also complete AI tasks with the help of cross-device; 2. Devices with insufficient computing power can complete complex AI task calculations with the help of distributed computing.
[0066] The technical solution of the embodiment of the present invention converts the NPU of the terminal device in the network into an NPU service object through the NPU virtual service, wherein the number of terminal devices in the network is at least two, and the terminal devices in the network are equipped with NPU; through the NPU virtual service, the registration interface of the distributed hardware management service is called to register the NPU service object of each terminal device to the distributed hardware management service; when the target terminal in the network receives the task to be processed, the target information of each NPU service object in the network is queried through the distributed hardware management service, and the target NPU service object for processing the task to be processed is determined based on the target information. The technical solution of the present invention solves the problem that when processing neural network tasks, the current end-cloud combination method relies on the network, has high latency, and cannot meet all working conditions. Through the distributed hardware management service, the task to be processed of the current terminal device is scheduled to the NPU service object of other terminal devices in the network, so that terminal devices without NPU or insufficient NPU computing power can use the NPU of other terminal devices, realizing efficient utilization of the NPU of the device in the network and reducing the latency of processing tasks.
[0067] Embodiment 3
[0068] Figure 4 This is a schematic diagram of the structure of an apparatus for calling NPU across devices provided in Embodiment 3 of the present invention. Figure 4 As shown, the device comprises:
[0069] A conversion module 310, configured to convert an NPU of a terminal device in a network into an NPU service object, wherein the number of terminal devices in the network is at least two, and at least one of the terminal devices in the network has an NPU;
[0070] A registration module 320 is used to call a registration interface of a distributed hardware management service through the NPU virtual service to register the NPU service object of each terminal device to the distributed hardware management service;
[0071] The calling module 330 is used to query the target information of each NPU service object in the network through the distributed hardware management service when the target terminal in the network receives the task to be processed, and determine the target NPU service object for processing the task to be processed based on the target information.
[0072] The technical solution of the embodiment of the present invention converts the NPU of the terminal device in the network into an NPU service object through the NPU virtual service, wherein the number of terminal devices in the network is at least two, and the terminal devices in the network are equipped with NPU; through the NPU virtual service, the registration interface of the distributed hardware management service is called to register the NPU service object of each terminal device to the distributed hardware management service; when the target terminal in the network receives the task to be processed, the target information of each NPU service object in the network is queried through the distributed hardware management service, and the target NPU service object for processing the task to be processed is determined based on the target information. The technical solution of the present invention solves the problem that when processing neural network tasks, the current end-cloud combination method relies on the network, has high latency, and cannot meet all working conditions. Through the distributed hardware management service, the task to be processed of the current terminal device is scheduled to the NPU service object of other terminal devices in the network, so that terminal devices without NPU or insufficient NPU computing power can use the NPU of other terminal devices, realizing efficient utilization of the NPU of the device in the network and reducing the latency of processing tasks.
[0073] Optionally, the conversion module 310 is specifically configured to:
[0074] Deploy an NPU virtual service in the terminal device in the network, and abstract the NPU hardware of the terminal device through the NPU virtual service to obtain the abstracted NPU function;
[0075] A target interface is defined, and the target interface and the abstracted NPU function are encapsulated into the NPU service object.
[0076] Optionally, the device also includes a more detailed status module, which is used to: after registering the NPU service object of each terminal device to the distributed hardware management service, update the target information of each NPU service object in real time through the distributed hardware management service, wherein the target information includes at least one of computing power information and load information.
[0077] Optionally, the calling module 330 is specifically used to:
[0078] Query the target information of the NPU service object corresponding to each terminal device in the network through the distributed hardware management service;
[0079] Based on the target information, a target NPU service object is selected from the multiple NPU service objects, wherein the terminal device corresponding to the target NPU service object is the terminal to be used.
[0080] Optionally, after determining the target NPU service object for processing the task to be processed based on the target information, the method further includes:
[0081] Determine the interface name and input parameters of the task to be processed, and pass in the interface name and the input parameters to process the task to be processed through the target NPU service object to obtain a processing result of the task to be processed;
[0082] The processing result is fed back to the target terminal.
[0083] Optionally, the device is also used for:
[0084] When the target NPU service object is called for the first time, a target identifier is allocated to a client that calls the target NPU service object, and context information is saved based on the target identifier, wherein the context information includes at least one of input data, intermediate results, and final results;
[0085] When the client repeatedly calls the target NPU service object, the saved context information is obtained based on the target identifier, and the to-be-processed task is processed based on the context information.
[0086] Optionally, the communication mode between the terminal devices in the network includes at least one of soft bus, Bluetooth, Wi Fi and Star Flash.
[0087] The apparatus for calling NPU across devices provided in the embodiment of the present invention can execute the method for calling NPU across devices provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.
[0088] Embodiment 4
[0089] Figure 5 A schematic diagram of the structure of an electronic device provided for Embodiment 4 of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0090] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0091] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0092] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for calling an NPU across devices.
[0093] In some embodiments, a method for calling an NPU across devices may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for calling an NPU across devices described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute a method for calling an NPU across devices in any other appropriate manner (e.g., by means of firmware).
[0094] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0095] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0096] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0097] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0098] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0099] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0100] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0101] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for calling NPU across devices, It is characterized in that include: Converting an NPU of a terminal device in a network into an NPU service object through an NPU virtual service, wherein the number of terminal devices in the network is at least two, and at least one of the terminal devices in the network has an NPU; By calling the registration interface of the distributed hardware management service through the NPU virtual service, the NPU service object of each terminal device is registered to the distributed hardware management service; When the target terminal in the network receives a task to be processed, the target information of each NPU service object in the network is queried through the distributed hardware management service, and a target NPU service object for processing the task to be processed is determined based on the target information.
2. The method according to claim 1, It is characterized in that The converting the NPU of the terminal device in the network into an NPU service object through the NPU virtual service includes: Deploy an NPU virtual service in the terminal device in the network, and abstract the NPU hardware of the terminal device through the NPU virtual service to obtain the abstracted NPU function; A target interface is defined, and the target interface and the abstracted NPU function are encapsulated into the NPU service object.
3. The method according to claim 1, It is characterized in that After registering the NPU service object of each terminal device to the distributed hardware management service, the method further includes: The target information of each NPU service object is updated in real time through the distributed hardware management service, wherein the target information includes at least one of computing power information and load information.
4. The method according to claim 1, It is characterized in that The querying target information of each NPU service object in the network through the distributed hardware management service, and determining a target NPU service object for processing the task to be processed based on the target information, includes: Query the target information of the NPU service object corresponding to each terminal device in the network through the distributed hardware management service; Based on the target information, a target NPU service object is selected from the multiple NPU service objects, wherein the terminal device corresponding to the target NPU service object is the terminal to be used.
5. The method according to claim 1, It is characterized in that After determining the target NPU service object for processing the task to be processed based on the target information, the method further includes: determining an interface name and input parameters of the task to be processed, and passing in the interface name and the input parameters to process the task to be processed through the target NPU service object to obtain a processing result of the task to be processed; The processing result is fed back to the target terminal.
6. The method according to claim 5, It is characterized in that Also includes: When the target NPU service object is called for the first time, a target identifier is allocated to a client that calls the target NPU service object, and context information is saved based on the target identifier, wherein the context information includes at least one of input data, intermediate results, and final results; When the client repeatedly calls the target NPU service object, the saved context information is obtained based on the target identifier, and the to-be-processed task is processed based on the context information.
7. The method according to claim 1, It is characterized in that The communication method between the terminal devices in the network includes at least one of soft bus, Bluetooth, WiFi and star flash.
8. A device for calling NPU across devices, It is characterized in that include: A conversion module, used to convert an NPU of a terminal device in a network into an NPU service object, wherein the number of terminal devices in the network is at least two, and at least one of the terminal devices in the network has an NPU; A registration module, used to call the registration interface of the distributed hardware management service through the NPU virtual service, and register the NPU service object of each terminal device to the distributed hardware management service; The calling module is used to query the target information of each NPU service object in the network through the distributed hardware management service when the target terminal in the network receives the task to be processed, and determine the target NPU service object for processing the task to be processed based on the target information.
9. An electronic device, It is characterized in that The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for calling NPU across devices according to any one of claims 1 to 7.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for calling an NPU across devices according to any one of claims 1 to 7 when executed.
Citation Information
Patent Citations
A GPU service redirection system and method based on cloud and terminal GPU fusion
CN109582425A
Virtualized computing power providing method and system
CN115167985A
Computing service providing method and device and readable storage medium
CN116708122A
Equipment virtualization method and related equipment
CN116974736A
Computing power resource scheduling method and device, terminal equipment and storage medium
CN118051341A
