Model deployment method and device, medium and product

Abstract

The invention discloses a model deployment method and device, a medium and a product, and the method comprises the steps: obtaining first information which indicates the model related information of a plurality of models contained in a first information base; acquiring second information, wherein the second information indicates chip related information of a plurality of chips contained in a second information base; based on the first information and the second information, managing the plurality of models and the plurality of chips, and determining a first model capable of adapting to the target chip; and deploying the first model on the target chip.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of model deployment technology, and in particular to a method, device, storage medium and computer program product for model deployment. Background Art

[0002] At present, with the continuous expansion of complex computing scenarios such as artificial intelligence (AI), cloud computing, and big data, the types of algorithm models (such as deep learning models, neural network models, etc.) continue to increase, and the usage scenarios have also expanded from terminal scenarios with independent computing power to edge and cloud scenarios integrated with multiple computing power resources. As a result, in terms of model deployment, model deployment on general-purpose computing chips based on the traditional von Neumann architecture can no longer meet the computing needs in more complex computing scenarios.

[0003] In the related technology, a solution is adopted in which a single model is customized and deployed on a fixed chip, which cannot adapt to the flexible deployment of various models in different application scenarios. Summary of the invention

[0004] The present application provides a method, device, medium and product for model deployment.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] A method for model deployment, the method comprising:

[0007] Acquire first information, where the first information indicates model-related information of a plurality of models included in a first information library;

[0008] Acquire second information, where the second information indicates chip-related information of a plurality of chips contained in a second information library;

[0009] Based on the first information and the second information, the multiple models and the multiple chips are managed to determine a first model that can adapt to the target chip;

[0010] The first model is deployed on the target chip.

[0011] In the above solution, the multiple chips include the target chip, the multiple models include a second model, and the managing the multiple models and the multiple chips based on the first information and the second information to determine the first model that can adapt to the target chip includes:

[0012] receiving a first request, where the first request is a deployment request for the second model on the target chip, and the first request carries a serial number of the second model;

[0013] Based on the serial number of the second model, obtaining model related information of the second model;

[0014] Based on the model-related information of the second model and the chip-related information of the target chip, performing adaptation analysis on the second model and the target chip to obtain an adaptation result;

[0015] If the adaptation result indicates that the model-related information of the second model and the chip-related information of the target chip do not meet the adaptation condition, the second model is optimized to obtain the first model that can adapt to the target chip.

[0016] In the above solution, the model-related information includes one or more of the following: parameter scale, weight accuracy, operator type;

[0017] The chip related information includes one or more of the following: the size of the free array, the supported operator precision, and the supported operator type.

[0018] In the above solution, performing adaptation analysis on the second model and the target chip based on the model-related information of the second model and the chip-related information of the target chip to obtain the adaptation result includes:

[0019] Obtaining the adaptation result based on the parameter scale of the second model and the size of the free array of the target chip;

[0020] Obtaining the adaptation result based on the weight accuracy of the second model and the operator accuracy supported by the target chip;

[0021] The adaptation result is obtained based on the operator type of the second model and the operator type supported by the target chip.

[0022] In the above scheme, if the adaptation result indicates that the model-related information of the second model and the chip-related information of the target chip do not meet the adaptation condition, optimizing the second model to obtain the first model that can adapt to the target chip includes:

[0023] If the parameter scale of the second model is larger than the size of the free array of the target chip, the model weights of the second model are compressed to obtain the first model.

[0024] In the above scheme, the method further comprises:

[0025] If the weight accuracy of the second model is inconsistent with the operator accuracy supported by the target chip, the weight accuracy of the second model is adjusted to the operator accuracy supported by the target chip to obtain the first model.

[0026] In the above scheme, the method further comprises:

[0027] If the operator type of the second model is different from the operator type supported by the target chip, the operator type in the second model that is not supported by the target chip is converted into an operator type that can be processed by the target chip to obtain the first model.

[0028] In the above solution, deploying the first model on the target chip includes:

[0029] Loading the control flow file of the first model;

[0030] Parsing the control flow file to obtain a model weight consistent with the operator precision supported by the target chip;

[0031] The model weights are stored in a free array of the target chip, and the control flow file is stored in a storage unit of the target chip.

[0032] In the above solution, the multiple models do not include the second model, and after the first model is deployed on the target chip, the method further includes:

[0033] Acquire model-related information of the first model;

[0034] Adding model-related information of the first model to the first information base;

[0035] Acquire target chip related information of the target chip on which the first model is deployed;

[0036] The original chip-related information of the target chip in the second information base is replaced with the target chip-related information.

[0037] In the above solution, the multiple models do not include the second model, and obtaining model-related information of the second model includes:

[0038] The second model is parsed to obtain model-related information of the second model.

[0039] In the above solution, the plurality of chips do not include the target chip, and the obtaining of the second information includes:

[0040] When a communication connection is established with the target chip, chip related information of the target chip is read.

[0041] The embodiment of the present application also provides a model deployment device, comprising: a processor and a memory for storing a computer program that can be run on the processor; wherein:

[0042] The processor is used to execute the steps of the aforementioned model deployment method when running the computer program.

[0043] An embodiment of the present application also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the aforementioned model deployment method are implemented.

[0044] An embodiment of the present application also provides a computer program product, including a computer program, which can be executed by a processor of an electronic device to complete the steps of the aforementioned model deployment method.

[0045] The embodiment of the present application provides a method, device, medium and product for model deployment, wherein the method includes: obtaining first information, the first information indicating model-related information of multiple models contained in a first information library; obtaining second information, the second information indicating chip-related information of multiple chips contained in a second information library; managing multiple models and multiple chips based on the first information and the second information, and determining a first model that can adapt to a target chip; deploying the first model on the target chip. That is, the present application manages multiple models and multiple chips by establishing an information library, so that when there is a deployment requirement for any model facing a target chip, the first model adapted to the target chip can be determined online and quickly, and flexible deployment of any model on a storage-computing integrated chip is realized in complex application scenarios of multiple models and multiple chips, solving the problem in the related technology that a single model is customized and deployed on a fixed chip, and cannot adapt to the flexible deployment of various models in different application scenarios, thereby improving the reuse rate of the chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flowchart of a method for model deployment according to an embodiment of the present application;

[0047] Figure 2 This is a schematic diagram of the structure of an application system for a storage-computing integrated chip according to an embodiment of the present application;

[0048] Figure 3 A flowchart of an online model deployment method for a storage-computing integrated chip according to an embodiment of the present application;

[0049] Figure 4 A schematic diagram of a flow chart of a model deployment method for a storage-computing integrated chip according to an embodiment of the present application;

[0050] Figure 5 A schematic diagram of the structure of a device deployed in a model according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0052] At present, model deployment management is usually based on general-purpose computing chips (such as graphics processing unit (GPU) and central processing unit (CPU)) of the traditional von Neumann architecture, which relies on the coordinated use of traditional operating systems and compiler software. However, with the continuous expansion of complex computing scenarios such as artificial intelligence (AI), cloud computing, and big data, not only the types of algorithm models (such as deep learning models, neural network models, etc.) continue to increase, but also the use scenarios have expanded from the end side with independent computing power to edge and cloud scenarios integrated with multiple computing power resources. The traditional computing architecture can no longer meet the computing needs of more complex computing scenarios, especially AI application scenarios.

[0053] As a new computing technology with comprehensive innovation from devices to architecture, storage-computing integration can realize parallel multiplication and addition calculations directly on the storage array, reducing the time overhead caused by data movement during the calculation process, greatly improving computing energy efficiency, and is widely applicable to computing scenarios with intensive multiplication and addition operations. With the continuous development and in-depth research of storage-computing integration technology, as well as the continuous enrichment of application scenarios such as artificial intelligence, storage-computing integration chips have gradually moved from the theoretical research stage to the industry application stage. Compared with traditional computing chips designed based on the von Neumann architecture, storage-computing integration chips have better performance in terms of computing power and energy efficiency, and their computing architecture is more adaptable to various complex computing scenarios such as AI. Therefore, how to automatically adapt and deploy various current algorithm models on storage-computing integration chips is of great significance to accelerating the replacement of users on computing chips and expanding the application scope of storage-computing integration chips.

[0054] When deploying AI models, traditional general-purpose computing chips do not consider the compatibility of models with chips, and due to their single computing architecture, they have problems with low computing power and energy efficiency. Due to the differences in devices and architecture between integrated storage and computing chips and traditional general-purpose computing chips, when using integrated storage and computing chips for AI model deployment, the model deployment and management methods based on traditional general-purpose computing chips are no longer applicable to current integrated storage and computing chips.

[0055] In the related technologies, in AI application scenarios, a customized solution is usually adopted to develop a customized single AI model according to user needs, and the AI ​​model is customized and deployed to a fixed storage-computing integrated chip before the chip product leaves the factory. No new model will be deployed on the storage-computing integrated chip during the entire product life cycle. Therefore, it is impossible to achieve flexible deployment of AI models and efficient reuse of storage-computing integrated chips. In other words, the model deployment and model maintenance / model management for storage-computing integrated chips in the related technologies are independent and separated, making the storage-computing integrated chip unable to adapt to various AI models in different application scenarios. Specifically, there are the following problems:

[0056] 1) In the related technologies, a single model is developed for specific application scenarios, and customized development and deployment are performed on fixed chips, which results in high labor costs and poor flexibility in updating and maintenance;

[0057] 2) The solutions adopted by related technologies make it impossible to guarantee the accuracy of model adaptation and the reuse of storage and computing integrated chips for complex scenarios of multi-chip and multi-application model integration, especially when there are complex situations of model replacement and / or computing chip update and upgrade;

[0058] 3) For application scenarios that currently use traditional general-purpose computing chips, it is impossible to automatically realize the migration and deployment of the AI ​​model used in the scenario to the storage and computing integrated chip. The use of related technical solutions will result in additional model design, training, adaptation and deployment work, which increases manpower and time costs.

[0059] Based on this, this application provides a method for model deployment, referring to Figure 1 As shown, the method comprises the following steps:

[0060] Step 101: Obtain first information.

[0061] The first information indicates model-related information of multiple models contained in the first information library. The first information library can also be called a model information library.

[0062] It can be understood that the execution subject of this embodiment is a terminal device, which has functions such as data processing, data communication and program running, and can implement the steps in the method of model deployment of this application. The terminal device can also be called a user device, and the user device can include but is not limited to a computer device, a tablet computer, a portable computer device, etc. Among them, the computer device can also be called a host device.

[0063] It should be noted that the model includes but is not limited to algorithmic models such as AI models and machine learning models. Among them, the model structure and model parameters of AI models designed and trained for different application scenarios are different. Therefore, the model-related information between multiple models in the first information library is different.

[0064] It should be noted that the multiple models in the first information repository include models that have been deployed and models that have not been deployed but are pre-stored in the first information repository.

[0065] Step 102: Obtain second information.

[0066] The second information indicates chip-related information of a plurality of chips contained in the second information library. The second information library may also be referred to as a chip information library.

[0067] The chip in the embodiment of the present application is a storage-computing integrated chip, which can be a storage-computing chip. It can be understood that since there are different implementation paths for the storage-computing integrated chip, the parameter information of the storage-computing integrated chip designed under different implementation paths is different. Therefore, the chip-related information between the multiple chips in the second information library is different.

[0068] It should be noted that the multiple chips in the second information library include chips that have been deployed and chips that have not been deployed but are pre-stored in the second information library.

[0069] Step 103: Based on the first information and the second information, multiple models and multiple chips are managed to determine a first model that can adapt to the target chip.

[0070] It should be noted that the multiple chips are multiple chips to be deployed, and the target chip is a chip to be deployed.

[0071] It should be noted that the multiple models are multiple models to be deployed, the first model is a deployable model, and the deployable model is a model adapted to the target chip. The model structure and model parameters of the deployable model and one of the multiple models to be deployed may be the same or different. When the model structure and model parameters of the deployable model are the same as those of one of the multiple models to be deployed, the one model can be directly used as the first deployable model; when the model structure and model parameters of the deployable model are different from those of one of the multiple models to be deployed, the one model can be optimized and the optimized model can be used as the first deployable model.

[0072] It should be noted that managing multiple models and multiple chips includes managing model-related information of multiple models in the first information library, and managing chip-related information of multiple storage-computing integrated chips in the second information library. The management method includes storing, analyzing, and processing information.

[0073] It can be understood that managing multiple models includes managing the model to be deployed from the multiple models when the multiple models include the model to be deployed; when the multiple models do not include the model to be deployed, adding the model to be deployed and directly managing the model-related information of the model to be deployed, including adding the model-related information of the model to be deployed in the first information database.

[0074] It can be understood that managing multiple chips includes managing the chip to be deployed from the multiple chips when the multiple chips include the chip to be deployed; when the multiple chips do not include the chip to be deployed, adding the chip to be deployed, and directly managing the chip-related information of the chip to be deployed, including adding the chip-related information of the chip to be deployed in the second information library.

[0075] In actual application, by managing multiple models and multiple chips, when a user has a deployment demand for a model for a target chip, the chip-related information of the target chip is analyzed online, and the model to be deployed is managed according to the chip-related information of the target chip to determine the model to be deployed that is compatible with the target chip, and the model to be deployed is used as the first deployable model.

[0076] Step 104: deploy the first model on the target chip.

[0077] It should be noted that one of the multiple chips can be automatically selected as the target chip according to the actual application scenario, or a selected chip can be automatically replaced with another chip according to actual needs, and the other chip can be used as the target chip. It can be understood that when the actual application scenarios are different, the models designed for training are different. At this time, different chips can be automatically selected as target chips in different scenarios according to different application scenarios, so as to deploy different models for different chips in different scenarios. In other words, the same chip can support the deployment of the same model, and can also support the separate deployment of different models; similarly, the same model can be deployed on one chip or on different chips. It is specifically set according to actual needs, and the embodiments of the present application do not specifically limit this.

[0078] In actual application, the models to be deployed are managed according to the deployment requirements of the user for the target chip, and after determining the deployable first model adapted to the target chip, the first model is deployed on the target chip. The first model is adapted to the target chip.

[0079] A method for model deployment provided by an embodiment of the present application includes: obtaining first information, the first information indicating model-related information of multiple models contained in a first information library; obtaining second information, the second information indicating chip-related information of multiple chips contained in a second information library; managing multiple models and multiple chips based on the first information and the second information, and determining a first model that can adapt to a target chip; and deploying the first model on the target chip. That is, the present application manages multiple models and multiple chips by establishing an information library, so that when there is a deployment requirement for any model facing a target chip, the first model adapted to the target chip can be determined online and quickly, and flexible deployment of any model on a storage-computing integrated chip is achieved in complex application scenarios of multiple models and multiple chips, solving the problem in the related technology that a single model is customized and deployed on a fixed chip, and cannot adapt to the flexible deployment of various models in different application scenarios, thereby improving the reuse rate of the chip.

[0080] In one embodiment, the plurality of chips include a target chip, the plurality of models include a second model, and in step 103, the plurality of models and the plurality of chips are managed based on the first information and the second information to determine the first model that can adapt to the target chip, including:

[0081] receiving a first request, where the first request is a deployment request for the second model on the target chip, and the first request carries a serial number of the second model;

[0082] Based on the serial number of the second model, obtaining model related information of the second model;

[0083] Based on the model-related information of the second model and the chip-related information of the target chip, performing adaptation analysis on the second model and the target chip to obtain an adaptation result;

[0084] If the adaptation result indicates that the model-related information of the second model and the chip-related information of the target chip do not meet the adaptation condition, the second model is optimized to obtain a first model that can adapt to the target chip.

[0085] It should be noted that the second model is a model to be deployed among multiple models, and the target chip is a chip to be deployed among multiple chips. In the embodiment of the present application, the second model to be deployed is optimized, and the optimized model is used as the deployable first model to deploy the first model on the target chip.

[0086] In the embodiment of the present application, since the second model is included in the multiple models and the target chip is included in the multiple chips, in actual application, when a user's deployment request for the second model on the target chip is received, the second model is directly found in the multiple models according to the number of the second model carried in the request, and model-related information corresponding to the second model is obtained.

[0087] In one embodiment, the first request may also carry the name of the second model. In actual application, the second model can be found in multiple models according to the name of the second model carried in the request, and model-related information corresponding to the second model can be obtained.

[0088] In one embodiment, the first request may also carry the number and / or name of the target chip. In actual application, when a deployment request from a user for a target chip is received, since the target chip is included in multiple chips, the target chip is directly found among the multiple chips according to the number and / or name of the target chip carried in the request, and chip-related information corresponding to the target chip is obtained.

[0089] In actual application, after obtaining the model-related information of the second model and the chip-related information of the target chip, the adaptation of the second model and the target chip is analyzed and judged from different dimensions to obtain the adaptation results in different dimensions. If the adaptation result indicates that the model-related information of the second model and the chip-related information of the target chip do not meet the adaptation conditions, it means that the second model and the target chip do not meet the adaptation conditions in at least one dimension. Based on the adaptation analysis result in at least one dimension, the second model is optimized in the at least one dimension, and the optimized model is used as the deployable first model. Obviously, the first model obtained after the adaptation analysis and optimization process meets the adaptation conditions with the target chip, and the first model can be adapted and deployed on the target chip.

[0090] It should be noted that the first request is a request issued by a user, which can be a request issued by a user to a terminal device, a request issued by a user through a terminal device, or a request initiated by a user through an application in the terminal device. It is set according to actual needs and is not specifically limited in this embodiment of the present application.

[0091] In the embodiment of the present application, for the deployment request of the second model on the target chip, the compatibility between the second model and the target chip is automatically analyzed and judged, and for the incompatible situation, the second model is optimized so that the optimized second model (i.e., the first model) can adapt to the target chip. In this way, even for the deployment request of any model to be deployed on the target chip, the compatibility between the model to be deployed and the target chip can be analyzed online, and the model to be deployed can be automatically optimized to obtain a deployable model, thereby ensuring the accuracy of model adaptation; in addition, due to the use of an online and automated processing method, the model can be optimized and adjusted based on the existing model and chip, without the need for additional model redesign, training, adaptation, and chip redeployment, thereby reducing the cost of manual processing and realizing automated and efficient deployment of the model on the chip.

[0092] Since there are many different implementation paths for storage-computing integrated chips, there are significant differences in the array structure, array scale, computing accuracy supported by the chip, and supported operator types between different storage-computing integrated chips. At present, storage-computing integrated chips generally have the following limitations: limited array scale, usually support low-precision calculations, and a limited number of supported operator types. In addition, the models designed and trained in different application scenarios are also different in terms of model structure, model parameters, operator types, parameter scales, etc., and usually have large parameter scales, complex operator types, and model parameters of floating point type, integer type, etc. Therefore, the embodiments of the present application analyze the adaptability of model deployment from different dimensions, which is used to deploy and manage models of various models on different storage-computing integrated chips in different application scenarios.

[0093] Based on this, in one embodiment, the model-related information obtained includes one or more of the following: parameter scale, weight accuracy, operator type;

[0094] The acquired chip-related information includes one or more of the following: the size of the free array, the supported operator precision, and the supported operator type.

[0095] It should be understood that the parameter scale of a model indicates the number of all parameters in the model. Among them, parameters include but are not limited to weights, biases, variables, etc. The size of the parameter scale characterizes the number of model parameters and the complexity of the model. It can be understood that the more the number of model parameters, the larger the parameter scale, and the more complex the model.

[0096] It should be understood that the weight precision of the model indicates the data type and its precision used by the model during the training process. When the model parameters are floating point, the weight precision of the model affects the storage and representation of the floating point parameters, affecting the computational efficiency and accuracy of the model. It can be understood that in different application scenarios, appropriate weight precision is used to balance computational efficiency and model precision. Exemplarily, the weight precision of the model may include but is not limited to half precision represented by 16-bit floating point numbers, double precision represented by 64-bit floating point numbers, precision represented by 8-bit integers, etc.

[0097] It should be understood that the operators of the model are used to construct and operate the algorithm model so that the algorithm model can process and recognize complex patterns and structures. The operator type of the model indicates different types of operation logic or operation rules. Exemplarily, the operator types of the neural network model may include convolution operator (Conv), fully connected operator (FC), activation operation (Relu), connection operation (concat), pooling operator (pooling), etc.

[0098] It should be understood that the size of the idle array of the chip can be obtained through the array information of the storage-computing integrated chip, which represents the amount of remaining array that can be used by the storage-computing integrated chip. Exemplarily, the size of the idle array of the chip can be obtained based on the array size, array occupancy, array occupancy, etc.

[0099] It should be understood that the operator precision supported by the chip indicates the numerical precision that the chip can support when performing calculation operations. For example, the numerical precision may include but is not limited to half precision represented by 16-bit floating point numbers, double precision represented by 64-bit floating point numbers, precision represented by 8-bit integers, etc.

[0100] It should be understood that the operator types supported by the chip indicate the operator types of the model that the chip can support when performing computing operations. It can be understood that for an operator type in the model, when the chip does not support the operator type, the model does not support computing operations based on the operator type on the storage-computing integrated chip. Exemplarily, the operator types supported by the chip may include convolution operators (Conv), fully connected operators (FC), activation operations (Relu), connection operations (concat), pooling operators (pooling), etc.

[0101] In actual application, when there is a need for model deployment, obtain the model-related information of the model to be deployed, including parameter scale, weight accuracy, and operator type, and obtain the chip-related information of the chip to be deployed, including the size of the idle array, the operator accuracy supported by the chip, and the operator type supported by the chip. This is used to perform adaptation analysis on the model to be deployed and the chip to be deployed from the three dimensions of chip array scale, supported operator accuracy, and supported operator type, so as to obtain the adaptation results in three different dimensions respectively. In this way, the model to be deployed can be automatically optimized according to the adaptation results in different dimensions, so as to realize the flexible deployment of various models on different storage and computing integrated chips.

[0102] In one embodiment, the step of performing adaptation analysis on the second model and the target chip based on the model-related information of the second model and the chip-related information of the target chip to obtain the adaptation result includes:

[0103] Obtaining an adaptation result based on the parameter scale of the second model and the size of the free array of the target chip;

[0104] Based on the weight accuracy of the second model and the operator accuracy supported by the target chip, an adaptation result is obtained;

[0105] Based on the operator type of the second model and the operator type supported by the target chip, an adaptation result is obtained.

[0106] It should be noted that since the integrated storage and computing chip stores the conductivity value converted from the weight value into the device in the storage and computing array, in the subsequent reasoning calculation process, the input data is converted into voltage to realize multiplication and addition calculation in the storage and computing array, and the storage and computing integrated chip array is usually small, and an excessive array occupancy rate easily leads to a decrease in the chip qualification rate. Therefore, the embodiment of the present application performs adaptation analysis based on the parameter scale of the second model and the size of the idle array of the target chip.

[0107] In actual application, based on the parameter scale of the second model and the size of the idle array of the target chip, it is determined from the dimension of the chip array scale whether the parameter scale of the second model and the size of the idle array of the target chip meet the adaptation condition. In this way, it is ensured that there are enough idle storage and computing arrays (idle arrays) in the storage and computing integrated array for storing the parameters of the second model, so that the model parameters of the second model can be stored on the target chip.

[0108] It should be noted that due to the limitations of the current storage and computing integrated chip technology, it usually supports low-precision integer storage. When the algorithm model in a complex application scenario is a floating-point parameter, the chip cannot support the normal storage and use of the algorithm model. Therefore, in order to ensure the normal storage and use of the model on the chip, as well as the calculation accuracy during the use of the model, the embodiment of the present application performs adaptation analysis based on the weight accuracy of the second model and the operator accuracy supported by the target chip.

[0109] In actual application, based on the parameter scale of the second model and the size of the free array of the target chip, the weight accuracy of the second model and the operator accuracy supported by the target chip are judged from the dimension of the operator accuracy supported by the chip to determine whether they meet the adaptation conditions. In this way, by performing adaptation analysis on the weight accuracy of the model and the operator accuracy supported by the storage-computing integrated chip, it is ensured that the weight parameters of the model can be correctly stored on the storage-computing integrated chip when the model is deployed.

[0110] It should be noted that the peripheral circuits of the storage-computing integrated chip can support the execution of different types of operation logic or operation rules including matrix multiplication and addition operations. For different storage-computing integrated chips, due to the significant differences in the peripheral circuits designed, the types of operators supported by different storage-computing integrated chips are different. Different algorithm models have different model structures and contain different types of operators. When the operator type contained in the model is not supported by the chip, the algorithm model cannot run this type of operator normally on the chip. Therefore, the embodiment of the present application performs adaptation analysis based on the operator type of the second model and the operator type supported by the target chip.

[0111] In actual application, based on the operator type of the second model and the operator type supported by the target chip, it is determined from the dimension of the operator type supported by the chip whether the operator type of the second model and the operator type supported by the target chip meet the adaptation conditions. In this way, by performing adaptation analysis on the operator type of the model and the operator type supported by the storage-computing integrated chip, it is ensured that after the model is deployed, the deployed model can run normally and perform computing operations on the storage-computing integrated chip.

[0112] The embodiment of the present application performs adaptation analysis on the model to be deployed and the chip to be deployed from multiple dimensions. Compared with the method of performing adaptation analysis from one dimension in the related art, the embodiment of the present application performs adaptation analysis from multiple dimensions, so that before the model is deployed, the model can be optimized from multiple dimensions, thereby improving the accuracy of the model deployment on the chip.

[0113] In one embodiment, if the adaptation result indicates that the model-related information of the second model and the chip-related information of the target chip do not meet the adaptation condition, the second model is optimized to obtain a first model that can adapt to the target chip, including:

[0114] If the parameter scale of the second model is larger than the size of the free array of the target chip, the model weights of the second model are compressed to obtain the first model.

[0115] In actual application, when the adaptation result is that the parameter scale of the second model and the size of the free array of the target chip do not meet the adaptation conditions, that is, the parameter scale of the second model is larger than the size of the free array of the target chip, at this time, the second model to be deployed and the target chip cannot complete direct adaptation. Therefore, the embodiment of the present application compresses the model weights of the second model so that the weight scale of the model weights of the compressed second model is smaller than the size of the remaining free array of the target chip.

[0116] In actual application, the model parameter compression algorithm can be trained, and the parameters of the second model can be optimized based on the trained model parameter compression algorithm. The model can be trained with the parameter scale of the second model and the size of the free array of the target chip as input, and the target compression ratio as output, so as to obtain the model parameter compression algorithm. The target compression ratio indicates the degree of compression of the model weights.

[0117] In the embodiment of the present application, when it is determined that the parameter scale of the second model is larger than the size of the idle array of the target chip, the model weight scale of the second model is reduced. In this way, starting from the dimension of the chip array scale, the second model is optimized so that the parameter scale of the optimized second model is smaller than the size of the idle array of the target chip, thereby obtaining an algorithm model that can adapt to the array scale of the target chip.

[0118] In one embodiment, the method further includes:

[0119] If the weight accuracy of the second model is inconsistent with the operator accuracy supported by the target chip, the weight accuracy of the second model is adjusted to the operator accuracy supported by the target chip to obtain the first model.

[0120] In actual application, when the adaptation result is that the weight accuracy of the second model and the operator accuracy supported by the target chip do not meet the adaptation conditions, that is, the weight accuracy of the second model is inconsistent with the operator accuracy supported by the target chip, at this time, the second model to be deployed and the target chip cannot complete direct adaptation. Therefore, the embodiment of the present application adjusts the weight accuracy of the second model so that the weight accuracy of the adjusted second model is consistent with the operator accuracy supported by the target chip.

[0121] In actual application, a suitable model quantization algorithm is selected, and the weight accuracy of the second model is optimized based on the selected model quantization algorithm.

[0122] Exemplarily, a suitable model quantization algorithm may be selected to convert the weight accuracy of the model from high precision (such as 64-bit floating point numbers) to lower precision (such as 8-bit integers).

[0123] In the embodiment of the present application, when it is determined that the weight accuracy of the second model is inconsistent with the operator accuracy supported by the target chip, the operator accuracy supported by the target chip is used as the target accuracy, and a suitable weight quantization algorithm is selected to adjust the weight accuracy of the second model to the operator accuracy supported by the target chip. In this way, starting from the dimension of the operator accuracy supported by the chip, the optimization processing of the second model is realized, so that the weight accuracy of the optimized second model is consistent with the operator accuracy supported by the target chip, thereby obtaining an algorithm model of the operator accuracy supported by the target chip.

[0124] In one embodiment, the method further includes:

[0125] If the operator type of the second model is different from the operator type supported by the target chip, the operator type in the second model that is not supported by the target chip is converted into an operator type that can be processed by the target chip to obtain the first model.

[0126] In actual application, when the adaptation result is that the operator type of the second model and the operator type supported by the target chip do not meet the adaptation conditions, that is, the operator type of the second model is different from the operator type supported by the target chip, then it indicates that the different operator type in the second model cannot be supported by the target chip, and the second model to be deployed and the target chip cannot complete direct adaptation. Therefore, the embodiment of the present application converts the operator type of the second model so that the operator type of the converted second model is the same as the operator type supported by the target chip. It should be noted that all operator types of the converted second model can be supported by the target chip.

[0127] In actual application, the operator optimization algorithm can be trained, and the operator type of the second model can be optimized based on the trained operator optimization algorithm, including: for the operator type in the second model that is not supported by the target chip, the operator corresponding to the operator type is obtained, and the operator is optimized using the operator optimization algorithm. The operator optimization algorithm may include but is not limited to operator conversion, operator fusion, operator splitting, etc.

[0128] In the embodiment of the present application, when it is determined that the operator type of the second model is different from the computing power type supported by the target chip, in order to ensure that the second model can perform normal computing operations on the target chip after the model is deployed, an operator optimization algorithm is used to convert the operator type in the second model that is not supported by the target chip, and convert it into an operator type that can be processed by the target chip. In this way, starting from the dimension of the operator type supported by the chip, the optimization processing of the second model is realized, so that the operator type of the optimized second model is the same as the operator type supported by the target chip, thereby obtaining an algorithm model that can adapt to the array scale of the target chip.

[0129] The embodiment of the present application performs adaptation analysis on the model to be deployed and the chip to be deployed from multiple dimensions, and based on the determined adaptation results, optimizes the model to be deployed from multiple dimensions, thereby obtaining an optimized first model, which is a model that can be deployed on the target chip. In this way, before the model is deployed, the model is optimized from multiple dimensions based on the adaptation results of multiple dimensions to improve the accuracy of the model deployment on the chip.

[0130] In one embodiment, deploying the first model on the target chip in step 104 includes:

[0131] Load the control flow file of the first model;

[0132] Parse the control flow file to obtain the model weights that are consistent with the operator precision supported by the target chip;

[0133] The model weights are stored in the free array of the target chip, and the control flow file is stored in the storage unit of the target chip.

[0134] It should be noted that the control flow file contains the description of the execution path and execution order logic during the program execution of the model. It can be understood that after loading the control flow file, the system runs the first model according to the control flow file. Parsing the control flow file means parsing the model parameters and model structure designed during the execution of the control flow file.

[0135] It should be noted that storing the control flow file in the storage unit of the target chip means storing the model information such as the model structure and model parameters of the first model in the storage unit of the target chip according to the control flow file. The storage unit of the chip includes registers, memories, drive circuits, etc.

[0136] In actual application, after determining the deployable first model, obtain the control flow file of the first model, load and run the first model according to the control flow file, parse the model parameters and model structure designed during the execution of the control flow file, and obtain the weight data corresponding to the model weight consistent with the operator precision supported by the target chip. Based on the weight data and the free array of the target chip, assign an array of appropriate size to the weight data, and then convert these weight data into corresponding resistance values ​​and store them in the array, and complete the register configuration according to the control flow file, so as to complete the deployment of the model on the target storage chip. In this way, the first model deployed on the target chip is a model adapted to the target chip, ensuring that the first model can smoothly perform model reasoning, and ensuring the accuracy and feasibility of model reasoning.

[0137] In actual application, since all operator types in the first model can be supported by the target chip and there are no operators that the target chip cannot handle, after the model deployment is completed, the first model deployed on the target chip can be used to smoothly perform model reasoning, further ensuring the accuracy and feasibility of model reasoning.

[0138] In one embodiment, the plurality of models do not include the second model, and after the first model is deployed on the target chip in step 104, the method further includes:

[0139] Obtain model-related information of the first model;

[0140] adding model-related information of the first model to the first information library;

[0141] Acquire target chip related information of the target chip where the first model is deployed;

[0142] The original chip-related information of the target chip in the second information database is replaced with the target chip-related information.

[0143] Among them, the model-related information of the first model includes the parameter scale, weight accuracy, operator type, number of the deployed chip, and the control flow file of the first model.

[0144] The target chip related information includes the size of the free array of the target chip after the first model is deployed, the supported operator precision, the supported operator type, and the number of the deployed first model.

[0145] It should be noted that the target chip related information may also include the size of the array, the array occupancy rate, the array occupancy status, etc., which are set according to actual needs, and the embodiments of the present application do not make specific limitations on this.

[0146] In actual application, when the multiple models in the first information library do not include the second model, the second model is optimized to obtain the first model as a newly added model. After the first model is deployed on the target chip, the model-related information of the first model is obtained, and the parameter scale, weight accuracy, operator type, number of the deployed chip, etc. in the model-related information of the first model are added one by one to the first information library, thereby completing the addition of information in the first information library, which is convenient for the subsequent management and maintenance of the first information library.

[0147] In one embodiment, when multiple models in the first information library include the second model, the second model is optimized to obtain the first model as an adapted model. After the first model is deployed on the target chip, the model-related information of the first model is obtained, and the model-related information at the original location of the second model is replaced one by one with the parameter scale, weight accuracy, operator type, number of the deployed chip, etc. in the model-related information of the first model, thereby completing the update of the information in the first information library and facilitating the subsequent management and maintenance of the first information library.

[0148] In actual application, when multiple chips in the second information library include a target chip, after the first model is deployed to the target chip, the target chip related information of the target chip on which the first model is deployed is obtained, and the original chip related information of the target chip is replaced with the target chip related information after the first model is deployed, including the size of the free array, the supported operator precision, the supported operator type, the number of the deployed first model, etc., thereby completing the update of the information in the second information library and facilitating the subsequent management and maintenance of the second information library.

[0149] In one embodiment, when multiple chips in the second information library do not include the target chip, after the first model is deployed to the target chip, the target chip related information of the target chip on which the first model is deployed is obtained, and the target chip related information is added to the second information library, thereby completing the addition of information in the second information library, which facilitates the subsequent management and maintenance of the second information library.

[0150] In one embodiment, when multiple models in the first information library include a second model, if the second model has not been optimized, that is, the second model has not been adapted, there is no need to update the parameter scale, weight accuracy, and operator type of the second model in the first information library, and only the numbering information of the deployed target chip of the second model needs to be updated. Furthermore, since the deployed target chip is array-allocated in this model deployment, the array information of the target chip in the second information library is updated. Among them, the array information may include the size of the array, the array occupancy rate, the array occupancy status, the size of the idle array, and the number of the deployed second model. In this way, the information of the first information library and the second information library is updated, which facilitates the subsequent management and maintenance of the first information library and the second information library.

[0151] In one embodiment, the plurality of models do not include the second model, and the step of obtaining model-related information of the second model includes:

[0152] The second model is parsed to obtain model-related information of the second model.

[0153] It should be noted that if the second model is not included in the multiple models, after receiving the deployment request for the second model on the target chip, i.e., the first request, the corresponding model-related information cannot be found in the multiple models in the second information library according to the number of the second model, then the second model can be obtained by other means, and the second model can be parsed to obtain the corresponding model-related information.

[0154] In actual application, the model structure and model parameters of the second model can be parsed by inputting the second model into a compiler to obtain model-related information of the second model, and a corresponding control flow file can be generated based on the model-related information.

[0155] It should be noted that, if the first request for the second model on the target chip does not carry the number of the second model, the second model is assigned a unique number, and a corresponding control flow file is generated based on model-related information of the second model.

[0156] In one embodiment, the plurality of chips do not include the target chip, and the step 102 of acquiring the second information includes:

[0157] When a communication connection is established with the target chip, chip related information of the target chip is read.

[0158] It should be noted that if the multiple chips do not include the target chip, after receiving the deployment request for the second model on the target chip, that is, the first request, according to the number of the target chip carried in the first request, the corresponding chip-related information cannot be found in the multiple chips in the first information library, then it indicates that the target chip has not been connected to the terminal device, and the chip-related information of the target chip can be obtained through other means.

[0159] In actual application, the target chip can be connected to the terminal device to establish a communication connection with the target chip, and the basic chip information and array usage stored and recorded on the target chip can be read to obtain chip-related information of the target chip, thereby obtaining the second information.

[0160] It should be noted that, when the first request does not carry the number of the target chip, a unique number is assigned to the target chip, and chip-related information of the target chip is obtained by establishing a communication connection with the target chip, thereby obtaining the second information.

[0161] The present application is described in further detail below in conjunction with application examples.

[0162] In a practical scenario, refer to Figure 2 As shown, an application system 200 for a storage-computing integrated chip 2021 is proposed, including a host device 201, a computing device 202 and a bus 203; wherein,

[0163] The host device 201 is responsible for model deployment and management. The host device 201 is configured with an artificial intelligence algorithm application 2011, a runtime management system 2012, and a model adaptation algorithm 2013, wherein the artificial intelligence algorithm application 2011 is used to initiate a model deployment request according to user needs, and the runtime management system 2012 is used to perform model and chip compatibility analysis, model deployment and management; the model adaptation algorithm 2013 is used to optimize the model so that the model information corresponding to the optimized model (corresponding to the aforementioned model-related information) and the chip information of the chip (corresponding to the aforementioned chip-related information) meet the adaptation conditions.

[0164] The computing device 202 is used to manage various storage and computing integrated chips 2021, including computing devices 202 in the form of single-chip or multi-chip integrated boards; the storage and computing integrated chip 2021 includes a storage and computing array 2022 and a digital circuit 2023, wherein the storage and computing array 2022 is used to store information, and the digital circuit 2023 is used to process and transmit electrical signals; the storage and computing integrated chip 2021 stores and records basic chip information and model information of deployed models. The computing device 202 integrated by multiple storage and computing integrated chips 2021 supports the dynamic deployment and iteration of multiple AI models.

[0165] The bus 203 is used to implement the communication connection between the computing device 202 and the host device 201 .

[0166] Exemplarily, the bus 203 may include but is not limited to a universal serial bus (USB), a high-speed serial computer expansion bus (peripheral component interconnect express, PCIE), a serial port, etc.

[0167] In actual application, the runtime management system 2012 on the host device 201 collects and stores basic information of the storage-computing integrated chip 2021 on the computing device 202 (such as array usage, supported operators, etc.), model information deployed on the storage-computing integrated chip 2021 (such as the deployed model name, model number (Identification, ID), deployment location, etc.), and establishes a model and chip information library; for the deployment request for the AI ​​model initiated by the user through the artificial intelligence algorithm application, the runtime system obtains the AI ​​model, and performs on the model structure and model parameters of the AI Online analysis is performed to analyze and determine whether the AI ​​model is compatible with the storage and computing integrated chip 2021. For AI models that cannot be adapted, the reasons for the inability to adapt are obtained based on the analysis. According to the reasons for the inability to adapt, the corresponding adaptation algorithm (such as model compression, weight quantization, operator optimization and other adaptation algorithms) is automatically called to perform model adaptation transformation (model optimization) on the AI ​​model to obtain an optimized AI model compatible with the storage and computing integrated chip 2021, and the optimized AI model is deployed on the storage and computing integrated chip 2021, thereby realizing online analysis of the model for the storage and computing integrated chip 2021 and automatic adaptation and deployment of the AI ​​model.

[0168] The application system for storage-computing integrated chips provided in the embodiments of the present application is based on the communication interaction between the host device and the computing device. The host device obtains the basic chip information of multiple storage-computing integrated chips in the computing device and the model information of the deployed models through the communication interaction, and performs online analysis and management on the multiple storage-computing integrated chips in the computing device and the models to be deployed on the storage-computing integrated chips, so that the aforementioned model adaptation deployment and management are automatically implemented according to the deployment requirements of the AI ​​model, which can effectively ensure the automatic adaptation deployment and flexible application of AI models in complex AI application scenarios in multiple storage-computing integrated chips, thereby expanding the application scope of the storage-computing integrated chip.

[0169] In an actual scenario where a user re-adapts, deploys and manages an online old model, refer to Figure 3 As shown, a model deployment method for storage-computing integrated chip is proposed, which can be applied to the aforementioned application system for storage-computing integrated chip, and the specific description is as follows:

[0170] Step 301: According to the model deployment requirements, query the information base for relevant information of the model to be deployed and the target chip to be deployed.

[0171] It should be noted that the model in the embodiment of the present application is an AI model.

[0172] It should be noted that the information library is a model deployment information library, including a model information sub-library (corresponding to the aforementioned first information library) and a chip information sub-library (corresponding to the aforementioned second information library), which can be called a model and chip information library. The model information sub-library is shown in Table 1, and the chip information sub-library is shown in Table 2.

[0173]

[0174] Table 1

[0175]

[0176] Table 2

[0177] In actual application, based on the model to be deployed provided by the user, it is determined that the model to be deployed is an old online model, indicating that the model to be deployed has been deployed in the storage and computing integrated chip in this application system. Therefore, the model deployment information library maintained by the runtime system in the application system is directly called to obtain relevant information of the model to be deployed, as shown in Table 1.

[0178] In actual application, based on the target chip to be deployed provided by the user, it is determined that the target storage chip to be deployed is an online old chip, indicating that the target storage chip to be deployed has been connected to this application system or has been deployed in this application system. Therefore, the model deployment information library maintained by the runtime system in the application system is directly called to obtain the relevant information of the target chip to be deployed, as shown in Table 2.

[0179] Exemplarily, referring to Tables 1 and 2, when the model to be deployed is model A, the model ID, parameter scale, deployment chip, control flow file, included operator types, and weight precision of model A are obtained from the model deployment information library; further, based on the number of the deployment chip of model A being C002, it is determined that model A has been deployed on storage and computing chip B, and the chip ID, array size, array occupancy, array occupancy, supported operator types, and operator precision of storage and computing chip B are obtained from the model deployment information library; when the chip to be deployed is chip A, the chip ID, array size, array occupancy, array occupancy, supported operator types, and operator precision of storage and computing chip A are obtained from the model deployment information library.

[0180] Step 302: Analyze and judge the compatibility of the model and the chip.

[0181] After obtaining relevant information about the AI ​​model to be deployed and the storage and computing chip to be deployed, analyze and judge the adaptability of the model and the storage and computing chip.

[0182] For example, in the case where model A has been deployed on storage and computing chip B, if the deployment requirement is to deploy model A to storage and computing chip B, directly go to step 304 .

[0183] For example, in the case where model A has not been deployed on storage and computing chip A, if the deployment requirement is to deploy model A to storage and computing chip A, the adaptation of model A and storage and computing chip A is analyzed to determine whether the size of the free array of storage and computing chip A is larger than the parameter scale of model A; further, determine whether the weight accuracy of model A is consistent with the operator accuracy supported by storage and computing chip A; further, determine whether storage and computing chip A supports all types of operators / operations contained in model A.

[0184] Step 303: Call the adaptation algorithm to process the model to be deployed into a deployable model that is compatible with the target chip.

[0185] Through adaptability analysis and judgment, for the AI ​​models to be deployed that cannot be adapted to the target chip, the adaptation algorithm supported by the system is called, and the models to be deployed are optimized based on the results of adaptability analysis and judgment to achieve adaptive conversion of the models on the target chip.

[0186] Step 304: deploy the deployable model on the target chip.

[0187] For the deployable AI model obtained, the system loads the control flow file of the deployable model, parses the deployable model weights, downloads the model weights to the storage array of the storage chip, and configures the registers according to the control flow, thereby completing the deployment of the deployable model on the target storage chip.

[0188] Step 305: Update the information library based on the deployable model and the deployed target chip.

[0189] After completing the download of the model weights on the target chip, the relevant information in the model deployment information library maintained in the runtime system is automatically updated, including the model-related information of the deployable AI model and the chip-related information of the deployed target chip, so as to manage and maintain the information library, reduce the labor cost of information maintenance, and improve the information reuse rate. In this way, through the model deployment information library, efficient management of the properties and application status of the AI ​​model and storage and computing chips in the system can be achieved, and multi-chip and multi-model runtime management systems can be realized through the host device, achieving efficient chip and model fitness comparison, and accelerating the speed of adaptation and deployment of online models on target chips.

[0190] The online model deployment method for storage-computing integrated chips proposed in the embodiment of the present application automatically evaluates the compatibility between the online old model and the storage-computing chip, and in the case of incompatibility, calls the model optimization algorithm to adapt and transform the model, thereby managing the model and the storage-computing chip online. Since the online management method can adapt and transform any model that is not compatible with the target chip, it is not restricted by the relevant information of the model and the chip, and realizes the flexible deployment of any AI model on the target chip in complex scenarios of multiple models and multiple chips. In addition, since there is no need to retrain the model, the deployed and used chips can be flexibly deployed again, which improves the reuse rate of the chip and reduces the update and maintenance cost of the chip.

[0191] In a practical scenario, refer to Figure 4 As shown, another model deployment method for storage-computing integrated chip is proposed, including two cases: the user provides a new model, and provides an online old model for re-adaptation deployment and management. It can be applied to the aforementioned application system for storage-computing integrated chip, and the specific description is as follows:

[0192] Step 401: Receive a model deployment request for a target chip.

[0193] According to the model deployment request, the AI ​​model to be deployed and the target chip to be deployed are obtained. The target chip is a storage and computing integrated chip.

[0194] Step 402 , determine whether the model to be deployed is a deployed model, if yes, proceed to step 403 , otherwise proceed to step 404 .

[0195] According to the name and ID of the model to be deployed, determine whether the model has been deployed in this system. For the model to be deployed, if it has been deployed on the storage and computing chip of this system, the model-related information of the old model can be directly obtained online from the information library maintained by this system, and then go to step 403; if it has not been deployed on the storage and computing chip of this system, the model-related information of the new model needs to be parsed and obtained, and then go to step 404.

[0196] Among them, if the request does not provide a model ID or the corresponding model information is not found in the information library maintained in the system according to the provided model name and / or model ID, it is judged that the model has not been deployed in this system; if the model is found in the information library according to the model ID provided in the request, it is judged that the model has been deployed in this system.

[0197] Step 403: Obtain model related information from the information database.

[0198] Referring to Table 1 above, the model and deployment target chip information are directly queried in the information database.

[0199] Step 404: parse the model to be deployed using a compiler to obtain model-related information.

[0200] The compiler parses the model structure and model parameters of the new model, obtains model-related information of the model information to be deployed, and assigns a unique model ID within the system to the model.

[0201] Step 405 , determine whether the chip to be deployed is a deployed chip, if yes, proceed to step 406 , otherwise proceed to step 407 .

[0202] According to the name and ID of the target chip to be deployed, it is determined whether the target chip has been connected to the system. For the chip to be deployed, if it has been connected or deployed in the system, the chip-related information can be directly obtained from the information library maintained by the system, and then the process goes to step 406; if it has not been connected or deployed in the system, the new chip needs to be connected to the system and the chip-related information is re-read, and then the process goes to step 407.

[0203] Among them, if the request does not provide the target chip ID or the corresponding chip information is not found in the information database maintained by the system according to the provided chip name and / or chip ID, it indicates that the chip is not connected to this system; if the chip is found in the information database according to the chip name and / or chip ID provided in the request, it indicates that the chip has been connected to this system or has been deployed on this system.

[0204] Step 406: Obtain chip related information from the information database.

[0205] Referring to Table 2 above, the chip-related information of the target chip to be deployed is directly queried in the information database.

[0206] Step 407: access the chip and read chip related information.

[0207] Connect the target chip to this system, read the basic chip information and array usage recorded on the storage chip, and assign the chip a unique chip ID within this system.

[0208] Step 408 , determine whether the size of the free array of the target chip is larger than the parameter scale of the model to be deployed, if not, proceed to step 409 ; if yes, proceed to step 410 .

[0209] Step 409: Call a compression algorithm to reduce the model weight of the model to be deployed, and return to step 408.

[0210] Among them, according to the information of the remaining free array size of the target chip, the compression target of the model weight is determined, and the compression algorithm is called to reduce the model weight of the model to be deployed, and the operation is repeated until the compressed model weight scale is smaller than the free array size of the target chip.

[0211] Step 410: determine whether the weight accuracy of the model to be deployed is consistent with the operator accuracy supported by the target chip. If not, proceed to step 411; if yes, proceed to step 412.

[0212] Step 411: call the weight quantization algorithm to adjust the model weight accuracy, and return to 410.

[0213] Among them, for high-precision floating-point model weights, the goal is to adapt to the operator precision supported by the target chip, select a suitable weight quantization algorithm to adjust the model weight precision, and repeat the operation until the weight corresponding to the model precision is adjusted to the precision supported by the storage and computing chip.

[0214] Step 412: determine whether the target chip supports all types of operators included in the model to be deployed. If not, proceed to step 413; if yes, proceed to step 414.

[0215] Step 413: Call the operator optimization algorithm to convert the operator type in the model to be deployed, and return to step 412.

[0216] For operator types not supported by the target chip, the unsupported operator types in the model to be deployed are optimized, including operator conversion, operator fusion, operator splitting and other optimization algorithms. The operation is repeated until all operator types in the model to be deployed are converted into operators that can be processed by the target chip.

[0217] Step 414: Get a deployable model.

[0218] After completing the adaptability analysis and judgment, as well as the adaptation and transformation of the model, a model that can be deployed to the target storage and computing chip is obtained, and model-related information of the model is obtained, including the operator type, weight accuracy, parameter scale, control flow file, etc. of the model.

[0219] Step 415: deploy the deployable model on the target chip.

[0220] Step 416: Update the model information sub-library in the information library based on the model-related information of the deployable model.

[0221] Step 417: based on the chip-related information of the deployed target chip, update the chip information sub-library in the information library.

[0222] In actual application, for the AI ​​models and storage and computing chips that originally exist in the system, when the adaptation algorithm is called to perform model transformation, it is necessary to update the model's parameter scale, weight accuracy, operator type and other model-related information according to the situation after the model transformation; if no adaptation transformation is performed, it is only necessary to update the array occupancy rate, array occupancy status and other chip-related information of the deployed target chip, and continue to maintain it.

[0223] In actual application, for newly added AI models and storage and computing chips in the system, it is necessary to add the information of the AI ​​model and storage and computing chip to the system's model information sub-library and chip information sub-library respectively, and continuously maintain them.

[0224] The embodiments of the present application realize online updating and efficient management of AI models and storage-computing integrated chips by constructing an information database, which can adapt to the scenarios of model and chip update and upgrade and improve the reuse of chips and models. In addition, through the update, maintenance and efficient management of the information database, during the model deployment process, relevant information of the model to be deployed and the chip to be deployed can be quickly and accurately obtained based on the information database, thereby improving the efficiency of model and chip adaptation processing and model deployment, reducing the difficulty of model adaptation deployment, simplifying the deployment process, and realizing reliable deployment of the model on the storage-computing chip.

[0225] Step 418: Complete the model deployment for the target chip.

[0226] The embodiments of the present application can perform online adaptation analysis on the information base maintained by the runtime system for any deployment requirements of any AI model on any storage and computing chip, and for incompatible situations, can flexibly and automatically call various algorithms from different dimensions to adapt and transform the model, and automatically deploy the model and update the information base based on the transformed model without human intervention, thereby realizing automatic adaptation deployment and information management of the model; in addition, since the existing model is optimized and adjusted, compared with traditional model retraining and re-migration and deployment on the storage and computing chip, the present application does not require additional model design, adaptation deployment and other work, effectively reducing the difficulty and process of model deployment, improving deployment efficiency, and further realizing flexible deployment of AI models on storage and computing chips.

[0227] In order to implement the method for model deployment in the embodiment of the present application, the embodiment of the present application also provides a device for model deployment, referring to Figure 5 As shown, the device 500 deployed in the model includes: a processor 501, a memory 502, and a communication bus 503; wherein,

[0228] Processor 501, configured to execute the method provided by one or more of the above technical solutions when running a computer program;

[0229] A memory 502 for storing computer programs that can be run on the processor 501;

[0230] The communication bus 503 is used to implement the communication connection between the processor 501 and the memory 502 .

[0231] It should be noted that the specific processing process of the processor 501 can be understood by referring to the above method, which will not be repeated here.

[0232] Of course, in actual application, the various components in the device 500 deployed by the model are coupled together through the communication bus 503. It can be understood that the communication bus 503 is used to realize the connection and communication between these components. In addition to the data bus, the communication bus 503 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 5 Various buses are labeled as communication buses 503.

[0233] The memory 502 in the embodiment of the present application is used to store various types of data to support the operation of the model deployment device 500. Examples of such data include: any computer program used to operate on the model deployment device 500.

[0234] The method disclosed in the above embodiment of the present application can be applied to the processor 501, or implemented by the processor 501. The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 501. The processor 501 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 501 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 502, and the processor 501 reads the information in the memory 502 and completes the steps of the above method in combination with its hardware.

[0235] In an exemplary embodiment, the model deployment device 500 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0236] It can be understood that the memory (memory 502) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0237] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, a memory 502 storing a computer program, and the computer program in the memory 502 can be executed by a processor 501 of a model-deployed device 500 to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0238] In an exemplary embodiment, the embodiment of the present application also provides a computer program product, including a computer program, which can be executed by the processor 501 of the model deployment device 500 to complete the steps of the above method of the aforementioned model deployment device.

[0239] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0240] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0241] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A method for model deployment, characterized in that: The method comprises: Acquire first information, where the first information indicates model-related information of a plurality of models included in a first information library; Acquire second information, where the second information indicates chip-related information of a plurality of chips contained in a second information library; Based on the first information and the second information, the multiple models and the multiple chips are managed to determine a first model that can adapt to the target chip; The first model is deployed on the target chip.

2. The method according to claim 1, characterized in that The multiple chips include the target chip, the multiple models include a second model, and the managing the multiple models and the multiple chips based on the first information and the second information to determine the first model that can adapt to the target chip includes: receiving a first request, where the first request is a deployment request for the second model on the target chip, and the first request carries a serial number of the second model; Based on the serial number of the second model, obtaining model related information of the second model; Based on the model-related information of the second model and the chip-related information of the target chip, performing adaptation analysis on the second model and the target chip to obtain an adaptation result; If the adaptation result indicates that the model-related information of the second model and the chip-related information of the target chip do not meet the adaptation condition, the second model is optimized to obtain the first model that can adapt to the target chip.

3. The method according to claim 2, characterized in that The model-related information includes one or more of the following: parameter scale, weight accuracy, operator type; The chip related information includes one or more of the following: the size of the free array, the supported operator precision, and the supported operator type.

4. The method according to claim 3, characterized in that The performing adaptation analysis on the second model and the target chip based on the model related information of the second model and the chip related information of the target chip to obtain the adaptation result includes: Obtaining the adaptation result based on the parameter scale of the second model and the size of the free array of the target chip; Obtaining the adaptation result based on the weight accuracy of the second model and the operator accuracy supported by the target chip; The adaptation result is obtained based on the operator type of the second model and the operator type supported by the target chip.

5. The method according to claim 4, characterized in that If the adaptation result indicates that the model-related information of the second model and the chip-related information of the target chip do not meet the adaptation condition, optimizing the second model to obtain the first model that can adapt to the target chip includes: If the parameter scale of the second model is larger than the size of the free array of the target chip, the model weights of the second model are compressed to obtain the first model.

6. The method according to claim 5, characterized in that The method further comprises: If the weight accuracy of the second model is inconsistent with the operator accuracy supported by the target chip, the weight accuracy of the second model is adjusted to the operator accuracy supported by the target chip to obtain the first model.

7. The method according to claim 6, characterized in that The method further comprises: If the operator type of the second model is different from the operator type supported by the target chip, the operator type in the second model that is not supported by the target chip is converted into an operator type that can be processed by the target chip to obtain the first model.

8. The method according to claim 2, characterized in that The step of deploying the first model on the target chip includes: Loading the control flow file of the first model; Parsing the control flow file to obtain a model weight consistent with the operator precision supported by the target chip; The model weights are stored in a free array of the target chip, and the control flow file is stored in a storage unit of the target chip.

9. The method according to claim 2, characterized in that: The plurality of models do not include the second model, and after deploying the first model on the target chip, the method further includes: Acquire model-related information of the first model; Adding model-related information of the first model to the first information base; Acquire target chip related information of the target chip on which the first model is deployed; The original chip-related information of the target chip in the second information base is replaced with the target chip-related information.

10. The method according to claim 2, characterized in that The multiple models do not include the second model, and obtaining model-related information of the second model includes: The second model is parsed to obtain model-related information of the second model.

11. The method according to claim 1, characterized in that: The plurality of chips do not include the target chip, and obtaining the second information includes: When a communication connection is established with the target chip, chip related information of the target chip is read.

12. A model deployment device, characterized in that: The device comprises: a processor and a memory for storing a computer program that can be run on the processor; wherein, The processor is used to execute the steps of the method according to any one of claims 1 to 11 when running the computer program.

13. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer program product comprising a computer program, characterized in that The computer program implements the steps of the method according to any one of claims 1 to 11 when executed by a processor.

Images