A method and device for allocating computing resources

Abstract

The present application provides a method and device for allocating computing resources, which relates to the field of Internet of Things technology and allocates computing resources to applications on edge devices. In this method, the edge device can create an account for the first application based on a first request sent by the first application, and can also allocate a first computing resource on at least one AI chip on the edge device to the first application. Based on this solution, the edge device can create an account for the mounted application and allocate computing resources of the AI chip to the application, so that the computing resources used by the application on the AI chip can be managed, which can reduce the problem of insufficient computing resources on the edge device and the inability to meet the computing needs of some applications due to excessive computing resources occupied by a certain application.

Description

Technical Field

[0001] The present application relates to the field of Internet of Things technology, and in particular to a method and device for allocating computing resources. Background Art

[0002] With the rapid development of the Internet of Things (IoT), the amount of data generated by edge devices has increased dramatically, placing increasing demands on network bandwidth. Edge devices are devices located close to objects or data sources. Edge computing (EC) offers advantages such as low latency, bandwidth savings, and guaranteed data privacy and security. Compared to transmitting data to the cloud for processing, edge devices using edge computing process data more securely, privately, and with faster response times. Furthermore, edge devices reduce communication with the cloud, thereby lowering costs.

[0003] At the same time, with the increasing application of artificial intelligence (AI) technology, AI technology can be applied to edge devices to improve their performance. When applying AI technology to edge devices, AI models can be trained and uploaded to the edge devices. The processor can call the AI model on the edge device to perform inference and calculations on the data.

[0004] Currently, an edge device can host multiple applications and store AI models corresponding to multiple applications. However, since multiple applications are hosted on an edge device, when multiple applications call corresponding AI models for inference and calculation, they may compete for computing resources on the edge device. This may also lead to insufficient computing resources on the edge device to meet the computing needs of some applications, resulting in unreasonable use of computing resources on the edge device. Summary of the Invention

[0005] The present application provides a method and device for allocating computing resources to allocate computing resources to applications on edge devices, thereby reducing the problem of unreasonable use of computing resources on edge devices by applications.

[0006] In a first aspect, an embodiment of the present application provides a method for allocating computing resources. The method can be performed by an edge device provided in an embodiment of the present application, or can also be performed by other devices similar to edge devices. Exemplarily, the edge device can be a network device. In this method, the edge device can receive a first request sent by a first application. The first request is a request for requesting the edge device to create an account for the first application. The edge device can create an account for the first application in response to the first request, and can also allocate a first computing resource on at least one AI chip on the edge device to the first application.

[0007] Based on this solution, edge devices can create accounts for mounted applications and allocate computing resources of AI chips to applications, thereby managing the computing resources used by applications on AI chips. This can reduce the problem of insufficient computing resources on edge devices due to a certain application occupying too many computing resources and being unable to meet the computing needs of some applications.

[0008] In one possible implementation, the value of the first computing resource allocated by the edge device to the first application may be a preset value. For example, the value of the first computing resource allocated by the edge device to the first application may be a fixed value or a fixed ratio, and the fixed value or ratio may be allocated in advance.

[0009] Based on this solution, edge devices can allocate the computing resources of the AI chip to various applications, allowing each application to use the computing resources on the AI chip, reducing the problem of a certain application occupying too many computing resources.

[0010] In another possible implementation, the first request sent by the first application may include the size of the computing resource requested by the first application. Then, the size of the first computing resource allocated by the edge device to the first application may be less than or equal to the size of the requested computing resource.

[0011] Based on this solution, the size of computing resources required by different applications may be different. The edge device can allocate the first computing resource to the application according to the size of the computing resource requested by the application, so as to meet the computing resource requirements of the application as much as possible.

[0012] In one possible implementation, the edge device may allocate a first computing resource to the first application based on priority information. For example, the higher the priority of the first application, the greater the first computing resource allocated to it by the edge device. The priority information may be pre-stored on the edge device or sent to the edge device by the first application.

[0013] Based on this solution, edge devices can allocate more computing resources to high-priority applications and give priority to meeting the computing needs of high-priority applications.

[0014] In one possible implementation, the edge device may limit the amount of computing resources of the AI chip occupied by the first application in response to the computing resources occupied by the first application exceeding the first computing resources. The amount of computing resources after limitation may be less than or equal to the first computing resources. Exemplarily, the edge device limits the amount of computing resources of the AI chip occupied by the first application, which may be achieved by, but not limited to, lowering the inference priority of the first application. The inference priority here may be the priority of the first application relative to other applications in calling the AI model on the edge device for inference calculation. Alternatively, the edge device may also limit the amount of computing resources of the AI chip occupied by the first application by reducing the number of inferences of the first application. The number of inferences here may be the number of inferences within a specified time period.

[0015] Based on this solution, the edge device can monitor the computing resources occupied by the AI chip by the first application. When the amount of computing resources occupied by the first application exceeds the first computing resources, the computing resources occupied by the first application can be limited to avoid the problem of insufficient computing resources of the edge device due to the first application occupying too many computing resources, so as to meet the computing needs of other applications.

[0016] In one possible implementation, the edge device may further perform access authentication on the second application in response to the second application accessing the account created for the first application. If the second application's access authentication succeeds, the edge device may allow the second application to access the account created for the first application; if the second application's access authentication fails, the edge device may prohibit the second application from accessing the account created for the first application.

[0017] Based on this solution, when the second application wants to access the account created for the first application, the edge device can perform access authentication on the second application. If the access authentication fails, the account of the first application cannot be accessed. This can achieve access isolation for multiple applications and improve the security of access between applications.

[0018] In one possible implementation, the edge device may also receive an AI model uploaded by the first application. The AI model may be in a format that is usable by the edge device. If the AI model is not in a format that is usable by the edge device, the edge device may convert the AI model into a format that is usable by the edge device.

[0019] Based on this solution, the edge device can convert the format of the AI model when the format of the AI model uploaded by the first application is unavailable, so that the application can call the AI model on the edge device for reasoning and calculation.

[0020] In one possible implementation, the edge device may receive a first request sent by a first application through a first interface, wherein the first interface is an interface provided by the edge device for invoking by the first application and other applications except the first application.

[0021] Based on this solution, the application mounted on the edge device can send a first request to the edge device through the first interface, and can communicate with the edge device through the first interface.

[0022] In a second aspect, an edge device is provided, which includes various modules for executing the computing resource allocation method in the first aspect or any possible implementation of the first aspect and at least one AI chip for providing computing resources for the edge device.

[0023] In a third aspect, an edge device is provided, comprising a processor, a memory, and at least one AI chip that provides computing resources for the edge device; the memory is used to store one or more computer programs; when the one or more computer programs stored in the memory are executed by the at least one processor, the edge device is able to implement the technical solution of the above-mentioned first aspect and any possible design of the first aspect.

[0024] In a fourth aspect, an edge AI system is provided, comprising an edge device, wherein the edge device is used to execute the method described in the first aspect or any optional method of the first aspect.

[0025] In a fifth aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes any possible method of the first aspect.

[0026] In a sixth aspect, the present application provides a computer program product storing instructions, which, when executed on a computer, enables the computer to execute any possible method of the first aspect described above.

[0027] In the seventh aspect, a chip system is also provided, which includes at least one AI chip, and the at least one AI chip calls the program instructions stored in the memory during runtime to implement any possible method provided in the first aspect above.

[0028] The beneficial effects of the second to seventh aspects can refer to the beneficial effects of the first aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of an edge AI system provided in an embodiment of the present application;

[0030] Figure 2 One of the exemplary flow charts of a method for allocating computing resources provided in an embodiment of the present application;

[0031] Figure 3 One of the schematic diagrams of an edge device provided in an embodiment of the present application;

[0032] Figure 4 One of the exemplary flow charts of a method for allocating computing resources provided in an embodiment of the present application;

[0033] Figure 5 One of the exemplary flow charts of a method for allocating computing resources provided in an embodiment of the present application;

[0034] Figure 6 One of the schematic diagrams of an edge device provided in an embodiment of the present application;

[0035] Figure 7 This is one of the schematic diagrams of an edge device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0037] To facilitate understanding of the embodiments of the present application, first Figure 1 The edge artificial intelligence (AI) system applicable to the embodiment of the present application is described in detail as an example. Figure 1 As shown, the edge AI system 100 may include an edge device 101, a first application 102, and an AI chip 1011. Optionally, the edge AI system may further include a second application 104 and an AI chip 1012.

[0038] In the edge AI system 100, the first application 102 can upload a trained AI model to the edge device 101 and, when reasoning and computation are required, call the AI model in the edge device 101. The AI chips 1011 and 1012 provide computing resources for the edge device 101. In other words, the first application 102 or the second application 104 can call the AI model in the edge device 101 based on the computing resources provided by the AI chips 1011 and 1012 for reasoning and computation.

[0039] Each application, such as the first application 102 or the second application 104, can be installed on different or the same device, which can be a user equipment (UE), a computing device, or a network device. For example, the first application 102 and the second application 104 can be installed on the same UE, or can be installed on different UEs.

[0040] It should be understood that Figure 1 This is a simplified schematic diagram for ease of understanding only. The edge AI system may also include other applications or other AI chips. Figure 1 Not drawn in.

[0041] In addition, the technical solution of the present application can also be applied to a wireless communication system, which may include at least one network device and at least one application. For example, the network device in the wireless communication system may correspond to Figure 1 The edge device 101 shown in FIG.

[0042] Edge devices are devices that provide entry points to physical or service provider core networks. Examples include network devices such as routers, switches, integrated access devices (IADs), multiplexers, and various metropolitan area network (MAN) and wide area network (WAN) access devices.

[0043] Currently, multiple applications are typically installed on edge devices, and each application can call the corresponding AI model on the edge device for reasoning and calculation. Each application can be installed on different or the same device, which can be a user device, a computing device, or a network device. However, when an application calls the AI model on the edge device for reasoning and calculation, it may occupy most of the computing resources of the AI chip, which can easily lead to other applications being unable to use the computing resources of the AI chip and unable to call the AI model for reasoning and calculation. In addition, since multiple applications are installed on the same edge device, the data of a certain application may be arbitrarily accessed by other applications, making it impossible to guarantee the security of the accessed data. Therefore, the current edge devices do not have reasonable control over multiple applications.

[0044] In view of this, an embodiment of the present application provides a method for allocating computing resources, which can be applied to various communication systems, such as: long term evolution (LTE) system, Internet of Things (IoT) system, world-wide interoperability for microwave access (WiMAX) communication system, future fifth generation (5G) system, such as new radio access technology (NR) and future communication systems, such as 6G system.

[0045] This application will present various aspects, embodiments, or features in the context of systems that may include multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc. and / or may not include all of the devices, components, modules, etc. discussed in conjunction with the figures. Furthermore, combinations of these aspects may also be used.

[0046] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person skilled in the art will appreciate that, with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are equally applicable to similar technical problems.

[0047] Figure 2 This is an exemplary flow chart of a method for allocating computing resources provided by an embodiment of the present application, shown from the perspective of device interaction, which may include the following steps:

[0048] Step 201: A first application sends a first request to an edge device.

[0049] The first request here can be used to request the edge device to create an account for the first application. The first request can include identity information of the first application. For example, the first request can include the name of the first application.

[0050] Optionally, the edge device can authenticate the first application based on the identity information. For example, the edge device can maintain a list of identity information for applications that are allowed to create accounts. Upon receiving the first request, the edge device can search the list for the identity information included in the first request. If the identity information included in the first request is in the list, the edge device can consider the authentication successful. If the identity information included in the first request is not in the list, the edge device can consider the authentication failed.

[0051] Optionally, the edge device may not authenticate the first application.

[0052] In an embodiment of the present application, the first application may send the first request to the edge device through the first interface provided by the edge device. Figure 3 As shown, the edge device provides a first interface, which is an interface encapsulated in a unified protocol for application programs mounted on the edge device to call. The first interface can be encapsulated in an open source application programming interface (API) query language, such as GraphQL or RESTful. GraphQL here is a query language for querying graph data (graph), and RESTful is an application of representational state transfer (REST), which is a network service that complies with the REST style. REST is a World Wide Web software architecture style that aims to facilitate different software / programs to transmit information to each other in a network (such as the Internet). The first interface may include one or more of the following functions:

[0053] 1. Create an account: The first application can call the account creation function through the first interface to request the edge device to create an account for the first application.

[0054] 2. Upload model: The first application can call the upload model function through the first interface to upload the trained AI model to the edge device. The edge device can store the AI model in the account created for the first application.

[0055] 3. Model loading: The first application can call the model loading function through the first interface and input parameters, such as environmental parameters, condition parameters, etc., to load the uploaded AI model into a specific environment and conditions.

[0056] 4. Inference execution: The first application can call the inference execution function through the first interface and send an inference execution command to the edge device to run the AI model. The edge device can then run the AI model that has been loaded into a specific environment and conditions according to the inference execution command.

[0057] 5. Result Query: After the AI model inference is completed, the edge device can store the inference results in the account created for the first application. The first application can call the result query function through the first interface to access the account and query the inference results.

[0058] In the embodiment of the present application, since the first interface can be obtained by encapsulating multiple types of interfaces using a unified protocol, the application can communicate with the edge device through the first interface, and the communication method is simple.

[0059] Step 202: The edge device creates an account for the first application in response to the first request, and allocates a first computing resource on at least one AI chip on the edge device to the first application.

[0060] The value of the first computing resource here can be a threshold for the edge device to limit the first application to use the computing resources of the at least one AI chip mentioned above. The threshold can be a reference value and does not limit the use of computing resources by the first application.

[0061] In one embodiment, the edge device may create an account for the first application and allocate a first computing resource to the first application after the first application successfully authenticates the first application. If the edge device fails to authenticate the first application, an error indication may be returned to the first application to inform the first application that the account creation could not be performed.

[0062] After the edge device creates an account for the first application, it may send a first response message to the first application. The first response message may be a response message used to notify the first application that the request to create an account is successful. The first response message may also include the address corresponding to the account requested by the first application. This address may be a uniform resource locator (URL) address, or it may be the name corresponding to the account, etc. The first application can access the account created for it through the address included in the first response message.

[0063] In addition, the first application can also call the function of uploading the model through the first interface to send the trained AI model to the edge device. The AI model here can be a model that the first application needs to use when performing inference calculations. If the AI model uploaded by the first application is in a format that can be used by at least one AI chip on the edge device, then the edge device can store the corresponding AI model uploaded by the first application in the account of the first application. If the AI model uploaded by the first application is in a format that is not available to the AI chip on the edge device, then the edge device can convert the format of the AI model into a format that can be used by the AI chip. Among them, the format of the AI model is determined by the engine that trains the AI model, that is, the format of the AI model is different for different engines that train the AI model.

[0064] For example, AI model 1 is trained using PyTorch, and AI model 2 is trained using Caffe (convolutional architecture for fast feature embedding). Therefore, the format of AI model 1 is different from that of AI model 2. Figure 1 As shown, the available formats of AI chips 1011 and AI chips 1012 can be the formats of AI models trained using PyTorch. Therefore, if the AI model uploaded by the first application 102 through the first interface calling the upload model function is trained using PyTorch, the edge device can store the AI model in the account of the first application 102. If the AI model uploaded by the first application 102 through the first interface calling the upload model function is trained using TensorFlow, the edge device can convert the format of the AI model to the format of an AI model trained using PyTorch and store the converted AI model in the account of the first application 102.

[0065] As mentioned above, PyTorch is an open-source machine learning library in the Python programming language, used for applications such as natural language processing. TensorFlow is a symbolic mathematics system based on dataflow programming.

[0066] In an embodiment of the present application, the edge device may convert the format of the AI model in the following manner. For example, the edge device may pre-store a correspondence between a format conversion identifier and a dynamic code. The format conversion identifier may include the format before conversion or the name of the engine for training the AI model, for example, the format conversion identifier may be format A or engine A. Alternatively, the format conversion identifier may also include the format name before conversion and the target format name after conversion or the name of the engine for training the AI model before conversion and the name of the engine for training the AI model after conversion, for example, the format conversion identifier may be format A to format B or engine A to engine B, etc. After receiving the AI model uploaded by the first application, the edge device may determine the format of the AI model. If the format of the AI model is a format that is not available to the AI chip, the edge device may search for the corresponding dynamic code from the pre-stored correspondence between the format conversion identifier and the dynamic code according to the format name. The edge device may perform various conversions on the AI model based on the dynamic code found.

[0067] It should be understood that the format conversion method provided in the embodiment of the present application is only exemplary, and the embodiment of the present application can also use its format conversion method to convert the format of the AI model. The edge device can store the AI model after format conversion in the account of the first application. The first application can then call the AI model for reasoning and calculation and obtain the reasoning result. The edge device can store the reasoning result in the account of the first application. The first application can also access the account to query and obtain the reasoning result.

[0068] Optionally, when multiple applications need to use the computing resources of the edge device, in order to ensure the security of access between multiple applications, the edge device also performs access control on the application access account.

[0069] In one embodiment, the edge device can authenticate access to the second application in response to the second application accessing the account of the first application. For example, the edge device can pre-maintain a list of application identity information and encryption passwords. The edge device can request the encryption password of the first application from the second application. If the encryption password provided by the second application is correct, the edge device can allow the second application to access the first application. If the encryption password provided by the second application is incorrect, the edge device can prohibit the second application from accessing the first application.

[0070] Alternatively, if a second application wants to access the first application, the second application can send a second request to the edge device. This second request can be used to request access to the first application from the edge device. This second request can include the first application's identity information and an encrypted password. In response to the second application accessing the first application's account, the edge device can authenticate the second application. If authentication fails, the edge device can prohibit the second application from accessing the first application's account.

[0071] Based on this solution, the edge device can store the model, inference results and other data of the first application in the account of the first application. If a second application accesses the first application, the edge device can perform access authentication for the second application. Only when the access authentication is passed can the second application access the account of the first application. This can isolate the data of different applications and realize access isolation between different applications, thereby improving data security.

[0072] When the edge device receives the first request, the edge device may also allocate a first computing resource on at least one AI chip to the first application. The following describes a method for allocating the first computing resource to the first application by the edge device.

[0073] Method 1: The value of the first computing resource allocated by the edge device to the first application is a preset value.

[0074] In one example, the first computing resource allocated by the edge device to the first application can be a fixed value or a fixed ratio. The fixed value or fixed ratio here can be pre-allocated. For example, the edge device can pre-divide the computing resources on the AI chip 1011 and the AI chip 1012 into multiple computing resource blocks. The value of each computing resource block can be the same or different. For example, the edge device can divide the computing resources on the AI chip 1011 into 5 computing resource blocks, and the ratio of each computing resource block can be 20% of the computing resources of the AI chip 1011. The edge device can respond to the first request and randomly allocate the value of a computing resource block to the first application as the value of the first computing resource of the first application.

[0075] In another example, the edge device allocates a first computing resource to a first application based on the number of mounted applications. For example, N applications are mounted on the edge device, where N is an integer greater than 1. In response to a request from the N+1th application, the edge device may calculate a value or a ratio based on the number of applications (N+1) and allocate the calculated value or ratio to the N+1th application as the value of its first computing resource.

[0076] For example, if Figure 1 As shown, an edge device has AI chip 1011 and AI chip 1012. A first application 102 and a second application 104 are installed on the edge device. The edge device can allocate a fixed proportion of 30% of the computing resources of AI chip 1011 to first application 102 as the value of the first computing resource of first application 102. The edge device can allocate a fixed proportion of 40% of the computing resources of AI chip 1011 to second application 104 as the value of the first computing resource of second application 104. In this way, first application 102 can use 30% of the computing resources of AI chip 1011 to run the AI model, and second application 104 can use 40% of the computing resources of AI chip 1011 to run the AI model.

[0077] For example, a third application is added to the edge device. The edge device can then calculate the value or ratio of computing resources allocated to the third application. The edge device calculates 50%. Since 70% of the computing resources of AI chip 1011 have already been allocated to first application 102 and second application 104, the edge device can allocate 70% of the computing resources of AI chip 1012 to the third application as the value of its first computing resources.

[0078] Based on the above solution, edge devices can allocate computing resources to each mounted application, allowing the application to use the allocated computing resources to call AI models for reasoning and calculation.

[0079] Method 2: The edge device allocates first computing resources to the first application according to the computing resources requested by the first application.

[0080] A first application may request computing resources of a specified value from an edge device. The computing resources of the specified value requested by the first application may be sent simultaneously with the first request or separately from the first request. In one example, the first application may include the requested computing resources in the first request and send it to the edge device.

[0081] When the edge device receives the computing resources of the specified value requested by the first application, it can also determine whether the specified value exceeds the first threshold. The first threshold here can be the upper limit value of the computing resources used by the application obtained in advance by the edge device. If the edge device determines that the specified value exceeds the first threshold, the edge device can return an error indication, indicating that the computing resources requested by the first application exceed the first threshold. Or the edge device can also allocate computing resources to the first application according to the first threshold. If the edge device determines that the specified value does not exceed the first threshold, the edge device allocates the first computing resource to the first application according to the specified value. In summary, the first computing resource allocated by the edge device to the first application does not exceed the computing resources of the specified value requested above.

[0082] In one example, the specified value is a proportional value, and the first application sends a first request to the edge device. The first request includes the computing resources requested by the first application, and the computing resources are 25%, that is, 25% of the computing resources of the AI chip. It should be understood that the computing resources of 25% can be 25% of the computing resources of an AI chip, or 25% of the computing resources of all AI chips on the edge device. The edge device can allocate the first computing resource to the first application. Among them, the first computing resource is a computing resource less than or equal to 25%, for example, it can be 20%, 15%, etc. Figure 1 As shown, the first application 102 requests 75% of the computing resources, and the edge device determines that it does not exceed the first threshold. Then, the first computing resources that the edge device can allocate to the first application can be 75% of the computing resources of the AI chip 1012, or can be less than 75% of the computing resources of the AI chip 1012.

[0083] In another example, the specified value may also be a value of a determined computing resource.

[0084] Based on the above scheme, the size of computing resources required by each application may be different, so the edge device can allocate computing resources to different applications based on the requests of different applications.

[0085] Method 3: The edge device allocates a first computing resource to the first application according to a priority.

[0086] The priority information here can be pre-stored on the edge device, or can be sent to the edge device by the first application. The first application can send the priority information simultaneously with the first request, or separately. In one example, the first application can include the priority information in the first request and send it to the edge device.

[0087] In one embodiment, when the edge device assigns a fixed value as the value of the first computing resource to the first application, the fixed value may be assigned to the first application based on the priority information. For example, the edge device may pre-divide the computing resources on the AI chip 1011 and the AI chip 1012 into multiple computing resource blocks. The value of each computing resource block may be the same or different. For example, the edge device may divide the computing resources on the AI chip 1011 into three computing resource blocks, and the proportions of each computing resource block are 20% of the computing resources of the AI chip 1011, 50% of the computing resources of the AI chip 1011, and 30% of the computing resources of the AI chip 1011, respectively. The edge device may, in response to the first request, assign the value of a computing resource block to the first application based on the priority information of the first application. Exemplarily, the higher the priority information, the larger the value of the allocated computing resource block.

[0088] For example, an edge device receives first requests from application A, application B, and application C, respectively. Based on the first requests sent by each application, the edge device can determine that application A has the highest priority, application B has the second highest priority, and application C has the lowest priority. The edge device can then allocate 50% of the computing resources of the AI chip 1011 as the first computing resource value to application A, 30% of the computing resources of the AI chip 1011 as the first computing resource value to application B, and 20% of the computing resources of the AI chip 1011 as the first computing resource value to application C.

[0089] In another embodiment, the edge device may pre-maintain a correspondence table between priority information and the first computing resource. The table may include the range of the first computing resource corresponding to each priority information. It should be understood that the range of the first computing resource is a reference value and does not limit the value of the first computing resource allocated to the application. The edge device allocates the first computing resource to the first application based on the priority information of the first application and the above-mentioned correspondence table. For example, the edge device receives a first request from the first application 102, and the priority information contained in the first request is 1. The edge device can determine that the range of the first computing resource corresponding to the priority information 1 is 50%-80% based on the above-mentioned correspondence table, and allocate 70% of the computing resources of the AI chip 1011 to the first application 102 as the value of the first computing resource.

[0090] The above-mentioned priority information can be a priority identifier, which can be used to indicate the level of priority. The edge device can maintain a correspondence between priority identifiers and priorities. For example, priority identifiers 001, 002, and 003 can be arranged in descending order of priority, with priority identifier 001 indicating the highest priority, priority identifier 002 indicating the second highest priority, and priority identifier 003 indicating the lowest priority.

[0091] Based on the above scheme, edge devices can allocate computing resources to different applications according to the priority of the applications. Edge devices can give priority to applications with higher priorities and allocate larger computing resources to applications with higher priorities.

[0092] In an embodiment of the present application, the edge device may also monitor the computing resources occupied by the first application. If the edge device determines that the amount of computing resources occupied by the first application exceeds the first computing resources allocated by the edge device for the first application, the edge device may limit the amount of computing resources occupied by the first application. It should be understood that the amount of computing resources occupied by the first application after the limit is less than or equal to the first computing resources allocated by the edge device.

[0093] For example, the edge device allocates all computing resources of AI chip 1011 and 25% of the computing resources of AI chip 1012 to first application 102, and allocates 75% of the computing resources of AI chip 1012 to first application 104. The edge device monitors first application 102 and second application 104, and if it determines that the computing resources occupied by second application 104 exceed 75% of the computing resources of AI chip 1012, the edge device may limit the computing resources occupied by second application 104.

[0094] In one example, the edge device can reduce the amount of computing resources occupied by the first application by lowering the inference priority of the first application. The inference priority here can be the priority of the first application relative to other applications in calling the AI model on the edge device for inference calculation. For example, if the inference priority of application A is higher than that of application B, then application A can use the computing resources on the AI chip in priority to application B, and run the AI model for inference and calculation. If the edge device detects that the computing resources occupied by application A exceed the first computing resources allocated by the edge device to application A, the edge device can lower the inference priority of application A. The lowered inference priority of application A is lower than that of application B. Then, application B can use the computing resources on the AI chip in priority to application A.

[0095] In another example, the edge device can reduce the amount of computing resources occupied by the first application by reducing the number of inferences of the first application. The number of inferences here can be the number of inferences within a specified time period. For example, the edge device can reduce the number of inferences of the first application within 1s. For example, the number of inferences per second of application A is 12 times. Application A can use the computing resources on the AI chip on the edge device to run the AI model 12 times per second for inference and calculation. If the edge device finds that the amount of computing resources occupied by application A has exceeded the first computing resource, the edge device can reduce the number of inferences of application A, such as reducing the number of inferences of application A to 7 times. Then, application A can use the computing resources on the AI chip on the edge device to call the AI model 7 times per second for inference and calculation.

[0096] Based on the above solution, the edge device can monitor the computing resources occupied by the first application, and when the amount of computing resources occupied by the first application exceeds the first computing resources allocated to it by the edge device, limit the amount of computing resources occupied by the first application on the AI chip, thereby ensuring that other applications can also use the computing resources of the AI chip.

[0097] Hereinafter, a method for allocating computing resources provided by the present application will be described in detail through specific embodiments.

[0098] Example 1: See Figure 4 , is an exemplary flow chart of a method for allocating computing resources in an embodiment of the present application, which can be executed by a first application. Figure 4 As shown, the following steps may be included:

[0099] Step 401: Send a first request to an edge device.

[0100] The first request here may include the identity information of the first application. For related descriptions, please refer to the above Figure 2 The description in the embodiment of the method will not be repeated here.

[0101] Step 402: Determine whether the account creation is successful. If the account creation is successful, execute step 403; if the account creation fails, return to execute step 401.

[0102] It should be understood that if the first application receives the first response sent by the edge device, it can be considered that the account creation is successful. If the first application does not receive the first response sent by the edge device, or the first application receives an error indication sent by the edge device, it can be considered that the account creation fails. For detailed descriptions, please refer to the above Figure 2 The description in the embodiment of the method will not be repeated here to avoid repetition.

[0103] Step 403: Upload the trained AI model to the edge device.

[0104] The format of the AI model uploaded by the first application may be a format that is available to the AI chip on the edge device. If the format of the AI model is not available to the AI chip on the edge device, the edge device may perform model conversion on the AI model. For related descriptions, please refer to the above Figure 2 The description in the embodiment of the method will not be repeated here.

[0105] Step 404: Call the AI model for reasoning and calculation.

[0106] The first application can call the AI model on the edge device by calling the first interface. The description of the first interface can be found in the above Figure 2 The description of the embodiment of the method will not be repeated here.

[0107] Step 405: Call the first interface to query the inference result.

[0108] Example 2: See Figure 5 , is an exemplary flow chart of the method for allocating computing resources provided in an embodiment of the present application, which can be executed by an edge device. Figure 5 As shown, the following steps may be included:

[0109] Step 501: Receive computing resources requested by a first application.

[0110] The description of the requested computing resources can be found in the above Figure 2 The description of the method embodiment shown in steps 201 and 202 will not be repeated here.

[0111] Step 502: Determine whether the requested computing resources exceed a first threshold, if so, proceed to step 503; if not, proceed to step 504.

[0112] The first threshold here can be predetermined based on empirical values, and this application does not make any specific limitations. Figure 2 The description of the embodiment of the method will not be repeated here.

[0113] Step 503: Return an error indication.

[0114] Step 504: Allocate a first computing resource to the first application.

[0115] Among them, the method for the edge device to allocate the first computing resource to the first application can refer to the relevant descriptions of the above methods 1 to 3, which will not be repeated here.

[0116] Step 505: Detect the computing resources occupied by the first application.

[0117] The edge device can monitor the first application calling the AI model for reasoning and calculation, and detect the computing resources occupied by the first application.

[0118] Step 506: Determine whether the computing resources occupied by the first application exceed the first computing resources. If so, execute step 507; if not, return to execute step 505.

[0119] Step 507: Limit the amount of computing resources occupied by the first application.

[0120] It should be understood that the amount of computing resources after the restriction does not exceed the first computing resource. Figure 2 The description of the embodiment of the method will not be repeated here.

[0121] Same as above idea, Figure 6 As shown, the embodiment of the present application further provides an edge device 600, which can implement the functions of the edge device in the above method embodiment, and thus can also achieve the beneficial effects of the above method embodiment. The edge device 600 includes a communication unit 601, a processing unit 602 and at least one AI chip 603.

[0122] In one example, the communication unit 601 and the processing unit 602 may be disposed on at least one AI chip 603 , or may be disposed on other chips other than the AI chip.

[0123] Among them, the communication unit 601 is used to receive a first request sent by a first application. The processing unit 602 is used to create an account for the first application in response to the first request, and allocate a first computing resource on at least one artificial intelligence (AI) chip on the edge device to the first application. Among them, the relevant description of the first request and the first computing resource can be found in the description of the above method embodiment and will not be repeated here. At least one AI chip 603 is used to provide computing resources for the edge device 600.

[0124] For the specific execution process of the communication unit 601 and the processing unit 602, please refer to the description in the above method embodiment. The division of modules / units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, the functional modules in the various embodiments of the present application can be integrated into one processor, or they can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0125] In another example, the edge device may be a chip system. In the embodiment of the present application, the chip system may be composed of a chip, or may include a chip and other discrete devices. Exemplarily, the edge device includes a processor and an interface, which may be an input / output interface. The processor performs the functions of the above-mentioned processing unit 602, and the interface performs the functions of the above-mentioned communication unit 601. The edge device may also include a memory for storing a program that can be run on the processor, and when the processor executes the program, the methods of the above-mentioned embodiments are implemented.

[0126] Same as above idea, Figure 7 As shown, an embodiment of the present application also provides an edge device 700. The edge device 700 includes: a communication interface 701, and at least one AI chip 704. Optionally, the edge device also includes at least one processor 702 and at least one memory 703. The communication interface 701 is used to communicate with other devices through a transmission medium, so that the device in the edge device 700 can communicate with other devices. The memory 703 is used to store computer programs. The processor 702 or at least one AI chip 704 calls the computer program stored in the memory 703, and sends and receives data through the communication interface 701 to implement the method in the above embodiment. At least one AI chip 704 is used to provide computing resources for the edge device 700.

[0127] Exemplarily, the memory 703 is used to store a computer program; the processor 702 calls the computer program stored in the memory 703 to execute the method executed by the edge device in the above embodiment.

[0128] In the embodiment of the present application, the communication interface 701 can be a transceiver, circuit, bus, module or other type of communication interface. The processor 702 can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor to be executed, or the hardware and software modules in the processor can be combined and executed. The memory 703 can be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or a volatile memory (volatile memory), such as a random access memory (RAM). The memory is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in the embodiment of the present application can also be a circuit or any other device that can implement a storage function. The memory 703 is coupled to the processor 702. The coupling in the embodiment of the present application is an interval coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules. As another implementation, the memory 703 can also be located outside the device 700. The processor 702 can operate in conjunction with the memory 703. The processor 702 can execute program instructions stored in the memory 703. At least one of the at least one memory 703 can also be included in the processor 702. The embodiment of the present application does not limit the connection medium between the above-mentioned communication interface 701, the processor 702 and the memory 703. For example, in the embodiment of the present application, Figure 7 The memory 703, the processor 702 and the communication interface 701 may be connected via a bus, which may be divided into an address bus, a data bus, a control bus and the like.

[0129] Understandably, the above Figure 6 The edge device 600 in the illustrated embodiment may be Figure 7 The edge device 700 shown is implemented as follows. Specifically, the processing unit 602 can be implemented by a processor 702 , and the communication unit 601 can be implemented by a communication interface 701 .

[0130] The methods provided in the embodiments of the present application can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0131] The above embodiments are merely a detailed introduction to the technical solutions of the present application. However, the descriptions of the above embodiments are intended only to facilitate understanding of the methods of the embodiments of the present invention and should not be construed as limiting the embodiments of the present invention. Any changes or substitutions that can be readily conceived by those skilled in the art should fall within the scope of protection of the embodiments of the present invention.

Claims

1. A method for allocating computing resources, characterized in that: include: The edge device receives a first request sent by a first application, where the first request is used to request the edge device to create an account for the first application; In response to the first request, the edge device creates an account for the first application and allocates a first computing resource on at least one artificial intelligence (AI) chip on the edge device to the first application; monitoring the computing resources occupied by the first application, and in response to the computing resources occupied by the first application exceeding the first computing resources, limiting, by the edge device, the amount of computing resources occupied by the first application on the at least one AI chip; wherein the limited amount of computing resources is less than or equal to the first computing resources; Herein, limiting the amount of computing resources occupied by the first application program at least one AI chip includes lowering the inference priority of the first application program or reducing the number of inference times of the first application program.

2. The method according to claim 1, characterized in that The value of the first computing resource is a preset value.

3. The method according to claim 1, characterized in that The first request includes computing resources requested by the first application, the first computing resources being less than or equal to the requested computing resources.

4. The method according to claim 1 or 2, characterized in that Allocating a first computing resource on at least one AI chip on the edge device to the first application includes: The edge device allocates a first computing resource on the at least one AI chip to the first application according to the priority information.

5. The method according to any one of claims 1 to 3, characterized in that: Also includes: In response to a second application accessing the account, the edge device performs access authentication on the second application; In response to the access authentication failure, the edge device prohibits the second application from accessing the account.

6. The method according to any one of claims 1 to 3, characterized in that: Also includes: The edge device receives the AI model uploaded by the first application; The edge device converts the format of the received AI model into a format usable by the edge device.

7. The method according to any one of claims 1 to 3, characterized in that: The edge device receives a first request sent by a first application, including: The edge device receives the first request sent by the first application through a first interface; the first interface is an interface provided by the edge device for calling by the first application and other applications except the first application.

8. The method according to any one of claims 1 to 3, characterized in that: The edge device is a network device.

9. An edge device, characterized in that: include: a communication unit, configured to receive a first request sent by a first application, where the first request is used to request the edge device to create an account for the first application; a processing unit, configured to create an account for the first application in response to the first request, and allocate a first computing resource on at least one artificial intelligence (AI) chip on the edge device to the first application; The processing unit is further configured to monitor the computing resources occupied by the first application, and in response to the computing resources occupied by the first application exceeding the first computing resources, limit the amount of computing resources occupied by the first application on the at least one AI chip; wherein the limited amount of computing resources is less than or equal to the first computing resources; Herein, limiting the amount of computing resources occupied by the first application program at least one AI chip includes lowering the inference priority of the first application program or reducing the number of inference times of the first application program.

10. The edge device according to claim 9, wherein: The value of the first computing resource is a preset value.

11. The edge device according to claim 9, wherein: The first request includes computing resources requested by the first application, the first computing resources being less than or equal to the requested computing resources.

12. The edge device according to claim 9 or 10, characterized in that: When allocating the first computing resource on the at least one AI chip on the edge device to the first application, the processing unit is specifically configured to: Allocate a first computing resource on the at least one AI chip to the first application according to the priority information.

13. The edge device according to any one of claims 9 to 11, characterized in that: The processing unit is further configured to: In response to a second application accessing the account, performing access authentication on the second application; In response to the access authentication failing, the second application is prohibited from accessing the account.

14. The edge device according to any one of claims 9 to 11, characterized in that: The communication unit is further configured to receive the AI model uploaded by the first application; The processing unit is further used to convert the format of the received AI model into a format usable by the edge device.

15. The edge device according to any one of claims 9 to 11, characterized in that: When receiving the first request sent by the first application, the communication unit is specifically configured to: Receive the first request sent by the first application through a first interface; the first interface is an interface provided by the edge device for calling by the first application and other applications except the first application.

16. An edge device, characterized in that: The device includes a processor, a memory, and at least one AI chip; The memory is used to store computer programs or instructions; The processor is configured to execute the computer program or instructions in the memory, so that the method according to any one of claims 1 to 8 is performed.

17. An edge artificial intelligence (AI) system, characterized in that: The device comprises an edge device, wherein the edge device is the device according to any one of claims 9 to 15, or the edge device is the device according to claim 16.

Images