Artificial intelligence model running method and computer device

Abstract

The application relates to an artificial intelligence model running method and a computer device, relates to the technical field of artificial intelligence models, and can improve the running efficiency of artificial intelligence models on different terminal devices. The method comprises the following steps: in the case that a trigger event of calling an artificial intelligence model is detected, determining the current business requirement of a client; obtaining a target model file matched with the business requirement and a model running environment of the client from a plurality of candidate model files; the plurality of candidate model files correspond to different artificial intelligence models respectively, and inference engines relied on by each artificial intelligence model are matched with corresponding model running environments; obtaining engine running resources related to an inference engine corresponding to the target model file; and based on the engine running resources and the target model file, running a target artificial intelligence model corresponding to the target model file on the client.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence model technology, and in particular to an artificial intelligence model operation method, computer device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of computer technology, the reasoning ability of artificial intelligence models has been widely applied in mobile terminal devices.

[0003] In related technologies, developers mainly train artificial intelligence models according to actual needs and apply the same artificial intelligence models to various terminal devices. For example, a trained artificial intelligence model can be pre-packaged in the client and run to perform relevant reasoning during the use of the client.

[0004] However, in practice, it was found that when running the artificial intelligence model on different terminal devices in the above manner, some devices were slow to obtain inference results, and there is still room for improvement in the running efficiency of the artificial intelligence model on terminal devices. Summary of the Invention

[0005] Therefore, it is necessary to provide an artificial intelligence model running method, computer equipment, computer-readable storage medium, and computer program product that can improve the running efficiency of artificial intelligence models on different terminal devices in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a method for operating an artificial intelligence model, including:

[0007] Upon detecting a trigger event that invokes the artificial intelligence model, determine the client's current business requirements;

[0008] Target model files that match the business requirements and the client's model runtime environment are obtained from multiple candidate model files; the multiple candidate model files correspond to different artificial intelligence models, and the inference engine on which each artificial intelligence model depends is matched with a corresponding model runtime environment;

[0009] Obtain the engine runtime resources related to the inference engine corresponding to the target model file;

[0010] Based on the engine's runtime resources and the target model file, the target artificial intelligence model corresponding to the target model file is run on the client.

[0011] In one embodiment, obtaining the target model file that matches the business requirements and the client's model runtime environment from multiple candidate model files includes:

[0012] Determine the model functions that match the stated business requirements;

[0013] If the client stores a list of valid models corresponding to the model function, then the target model identifier in the list of valid models is obtained; the inference engine on which the artificial intelligence model corresponding to the target model identifier depends matches the model running environment of the client.

[0014] The target model file is obtained by querying the model file on the client based on the target model identifier.

[0015] In one embodiment, the target model identifier is the most recently added model identifier;

[0016] After querying the model file on the client based on the target model identifier to obtain the target model file, the process further includes:

[0017] Send a model update request to the server; the model update request is used to trigger the server to determine a recommended artificial intelligence model based on the target model identifier and the model running environment carried in the model update request, and to obtain the model comparison result between the recommended artificial intelligence model and the target artificial intelligence model;

[0018] When the model comparison result indicates that the recommended AI model is superior to the target AI model, the model file corresponding to the recommended AI model returned by the server is stored, and the model identifier corresponding to the recommended AI model is added to the list of valid models.

[0019] In one embodiment, after running the target AI model corresponding to the target model file on the client based on the engine runtime resources and the target model file, the method further includes:

[0020] When retrieving the target model file based on the list of valid models again, the client's historical model file is queried based on the historical model identifier in the list of valid models; the historical model identifier is a model identifier whose addition time is later than the addition time of the target model identifier.

[0021] Delete the historical model file and the historical model identifier.

[0022] In one embodiment, the model update request is further used to trigger the server to obtain the validity information of the target artificial intelligence model;

[0023] The method further includes:

[0024] If the validity information indicates that the target AI model has failed, the target model identifier is added to the list of failed models, and / or the target model file stored on the client is deleted.

[0025] In one embodiment, obtaining the target model file that matches the business requirements and the client's model runtime environment from multiple candidate model files includes:

[0026] Determine the model functions that match the stated business requirements;

[0027] When the client does not store the model file for the model function, a model loading request carrying the model function and the client's model runtime environment information is sent to the server; the model loading request is used to trigger the server to determine the target artificial intelligence model that matches the model runtime environment from multiple candidate artificial intelligence models under the model function, and obtain the target model file corresponding to the target artificial intelligence model;

[0028] Receive the target model file returned by the server.

[0029] In one embodiment, before obtaining the target model file that matches the business requirements and the client's model runtime environment from multiple candidate model files, the method further includes:

[0030] Obtain the client attribute information of the client and the device attribute information of the terminal device on which the client is deployed;

[0031] Obtain the execution strategies of each candidate inference engine that are supported locally;

[0032] The model runtime environment of the client is determined based on the client attribute information, the device attribute information, and the candidate inference engine running strategy.

[0033] In one embodiment, obtaining the engine runtime resources related to the inference engine corresponding to the target model file includes:

[0034] Determine the target inference engine execution strategy for the target model file locally;

[0035] A resource loading request is generated according to the target inference engine running strategy, and the resource loading request is sent to the server; the resource loading request is used to trigger the server to obtain the target resource file required for the inference engine corresponding to the target model file to run according to the target inference engine running strategy;

[0036] Based on the target resource file returned by the server, the engine running resources are obtained.

[0037] In one embodiment, running the target AI model corresponding to the target model file on the client based on the engine runtime resources and the target model file includes:

[0038] Obtain the engine configuration parameters of the inference engine corresponding to the target model file;

[0039] Based on the engine configuration parameters, the engine runtime resources, and the target model file, an inference processor is created on the client.

[0040] The inference processor is used to run the target artificial intelligence model corresponding to the target model file.

[0041] Secondly, this application also provides another method for operating an artificial intelligence model, including:

[0042] Receive a model loading request from the client; the model loading request is sent by the client upon detecting a trigger event that calls the artificial intelligence model;

[0043] In response to the model loading request, a target model file matching the client's current business requirements and the client's model runtime environment is obtained from multiple candidate model files;

[0044] The target model file is returned to the client to trigger the client to obtain the engine running resources related to the inference engine corresponding to the target model file, and to run the target artificial intelligence model corresponding to the target model file on the client based on the engine running resources and the target model file.

[0045] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0046] Upon detecting a trigger event that invokes the artificial intelligence model, determine the client's current business requirements;

[0047] Target model files that match the business requirements and the client's model runtime environment are obtained from multiple candidate model files; the multiple candidate model files correspond to different artificial intelligence models, and the inference engine on which each artificial intelligence model depends is matched with a corresponding model runtime environment;

[0048] Obtain the engine runtime resources related to the inference engine corresponding to the target model file;

[0049] Based on the engine's runtime resources and the target model file, the target artificial intelligence model corresponding to the target model file is run on the client.

[0050] Fourthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0051] Receive a model loading request from the client; the model loading request is sent by the client upon detecting a trigger event that calls the artificial intelligence model;

[0052] In response to the model loading request, a target model file matching the client's current business requirements and the client's model runtime environment is obtained from multiple candidate model files;

[0053] The target model file is returned to the client to trigger the client to obtain the engine running resources related to the inference engine corresponding to the target model file, and to run the target artificial intelligence model corresponding to the target model file on the client based on the engine running resources and the target model file.

[0054] Fifthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0055] Upon detecting a trigger event that invokes the artificial intelligence model, determine the client's current business requirements;

[0056] Target model files that match the business requirements and the client's model runtime environment are obtained from multiple candidate model files; the multiple candidate model files correspond to different artificial intelligence models, and the inference engine on which each artificial intelligence model depends is matched with a corresponding model runtime environment;

[0057] Obtain the engine runtime resources related to the inference engine corresponding to the target model file;

[0058] Based on the engine's runtime resources and the target model file, the target artificial intelligence model corresponding to the target model file is run on the client.

[0059] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0060] Receive a model loading request from the client; the model loading request is sent by the client upon detecting a trigger event that calls the artificial intelligence model;

[0061] In response to the model loading request, a target model file matching the client's current business requirements and the client's model runtime environment is obtained from multiple candidate model files;

[0062] The target model file is returned to the client to trigger the client to obtain the engine running resources related to the inference engine corresponding to the target model file, and to run the target artificial intelligence model corresponding to the target model file on the client based on the engine running resources and the target model file.

[0063] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0064] Upon detecting a trigger event that invokes the artificial intelligence model, determine the client's current business requirements;

[0065] Target model files that match the business requirements and the client's model runtime environment are obtained from multiple candidate model files; the multiple candidate model files correspond to different artificial intelligence models, and the inference engine on which each artificial intelligence model depends is matched with a corresponding model runtime environment;

[0066] Obtain the engine runtime resources related to the inference engine corresponding to the target model file;

[0067] Based on the engine's runtime resources and the target model file, the target artificial intelligence model corresponding to the target model file is run on the client.

[0068] Eighthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0069] Receive a model loading request from the client; the model loading request is sent by the client upon detecting a trigger event that calls the artificial intelligence model;

[0070] In response to the model loading request, a target model file matching the client's current business requirements and the client's model runtime environment is obtained from multiple candidate model files;

[0071] The target model file is returned to the client to trigger the client to obtain the engine running resources related to the inference engine corresponding to the target model file, and to run the target artificial intelligence model corresponding to the target model file on the client based on the engine running resources and the target model file.

[0072] The aforementioned artificial intelligence model running method, computer device, computer-readable storage medium, and computer program product can determine the current business needs of the client upon detecting a trigger event for invoking the artificial intelligence model. Then, it retrieves a target model file from multiple candidate model files that matches the business needs and the client's model runtime environment. Each candidate model file corresponds to a different artificial intelligence model, and each artificial intelligence model relies on an inference engine with a corresponding model runtime environment. Furthermore, it can obtain the engine runtime resources related to the inference engine corresponding to the target model file, and based on the engine runtime resources and the target model file, run the target artificial intelligence model corresponding to the target model file on the client. In this embodiment, by retrieving a target model file from multiple candidate model files that matches the business needs and the client's model runtime environment, a target artificial intelligence model that meets the business needs and whose inference engine matches the model runtime environment can be dynamically loaded. Subsequently, by obtaining the engine runtime resources related to the inference engine and running the target artificial intelligence model in conjunction with the target model file, the running mode of the artificial intelligence model matches the actual model runtime environment. This satisfies the inference needs of various artificial intelligence models in the differentiated model runtime environments of terminal devices, improving the running efficiency of artificial intelligence models on different terminal devices. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0074] Figure 1 This is a flowchart illustrating an artificial intelligence model operation method in one embodiment;

[0075] Figure 2 This is a schematic diagram of the overall framework for the operation of an artificial intelligence model in one embodiment.

[0076] Figure 3 This is a schematic diagram of a list of valid models and a list of failed models in one embodiment;

[0077] Figure 4 This is a flowchart illustrating one step in obtaining an artificial intelligence model in one embodiment;

[0078] Figure 5 This is a schematic diagram illustrating the conversion of a model identifier in one embodiment;

[0079] Figure 6This is a schematic diagram illustrating how a server matches a target model file based on information provided by a client in one embodiment.

[0080] Figure 7 This is a schematic diagram illustrating one strategy classification in one embodiment;

[0081] Figure 8 This is a schematic diagram of the framework of a client software development kit in one embodiment;

[0082] Figure 9 This is a schematic diagram of the initialization process of a software development kit in one embodiment;

[0083] Figure 10 This is a schematic diagram illustrating the classification of one type of inference engine resource in one embodiment;

[0084] Figure 11 This is a schematic diagram of a process for creating an inference processor in one embodiment;

[0085] Figure 12 This is a schematic diagram of a process for running an artificial intelligence model via an inference processor in one embodiment;

[0086] Figure 13 This is an internal structural diagram of a computer device in one embodiment;

[0087] Figure 14 This is an internal structural diagram of another computer device in one embodiment. Detailed Implementation

[0088] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0089] In one embodiment, such as Figure 1As shown, an artificial intelligence model operation method is provided. This embodiment illustrates the application of this method to a terminal, but it is understood that the method can also be applied to a server, or to a system including both a terminal and a server, and is implemented through the interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc.; portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc.; head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0090] In this embodiment, the method includes the following steps:

[0091] S101, upon detecting a trigger event that invokes the artificial intelligence model, determines the client's current business requirements.

[0092] In practical applications, the client can detect whether a trigger event for invoking an artificial intelligence model exists. In some exemplary embodiments, a trigger event can be determined when a user-triggered business request is received, or when the client is detecting that it is performing a target task. For example, if a user clicks the karaoke rating button while using a karaoke client, a trigger event can be considered detected. Alternatively, a trigger event can be considered detected when it is recognized that the user is in a karaoke state and the karaoke client is collecting the user's voice data. Of course, in other examples, a trigger event can also be determined when a component or module of the client generates information that invokes an artificial intelligence model.

[0093] Upon detecting a triggering event, the client's business requirement for calling the artificial intelligence model can be determined. For example, when a triggering event is associated with a corresponding business request, the client's current business requirement can be determined based on that business request.

[0094] Specifically, an artificial intelligence (AI) model, also known as an AI model, can be understood as a mathematical model built based on artificial intelligence technology. This model can automatically process and analyze input data and output corresponding results. AI models can consist of numerous algorithms and programs, and their accuracy and efficiency are continuously improved through training and learning. Through pre-training, different types of AI models can infer and predict based on different types of input, resulting in different outputs. For example, a karaoke scoring model (also known as a "good-sounding" scoring model) can output a corresponding evaluation score based on the input audio recording of a song. Similarly, an AEC (Acoustic Echo Canceller) noise reduction model can cancel the echo in the original input signal and output the echo-cancelled sound signal.

[0095] To address this, AI models with corresponding functions can be pre-trained to meet various business needs encountered during client-side applications. Taking a karaoke client as an example, it can provide at least one of the following functions: playing original and / or instrumental songs, collecting vocal data generated during user karaoke and generating corresponding vocal audio, scoring user karaoke audio, and providing online live streaming rooms for online karaoke competitions or concerts. For these functions, song recommendation models, noise reduction models, and karaoke scoring models can be trained. Furthermore, in practical applications, multiple functions can be integrated simultaneously into the client to meet different business needs. Based on this, when detecting trigger events for calling AI models, the client's current business needs can be obtained, thereby determining the type of AI model to be called and using that type of AI model to meet the current business needs.

[0096] S102, Obtain the target model file that matches the business requirements and the client's model runtime environment from multiple candidate model files; the multiple candidate model files correspond to different artificial intelligence models, and the inference engine on which each artificial intelligence model depends matches the corresponding model runtime environment.

[0097] Inference engines can be understood as inference framework components used to run artificial intelligence models. Different inference engines, such as the TFLite inference engine and the Mnn inference engine, have different characteristics and are suitable for different use cases.

[0098] In related technologies, when using artificial intelligence models for inference on terminal devices, the main approach is to apply the same AI model to various terminal devices. However, the inventors discovered in practice that different terminal devices have significant performance differences, and the support capabilities and effects of the same inference engine vary across different operating systems, resulting in substantial differences in the performance of the same AI model on different terminal devices.

[0099] Based on this, in this embodiment, multiple AI models can be pre-trained based on different inference engines to meet the same business needs. In other words, AI models dependent on different inference framework components can be trained for the same business scenario or the same business need. For example, for a pleasantness scoring model, pleasantness scoring model a corresponding to inference engine 1 and pleasantness scoring model b corresponding to inference engine 2 can be trained. Since the two models depend on different inference engines, the suitable model running environments for pleasantness scoring model 1 and pleasantness scoring model 2 are different. Then, each pre-trained AI model can be converted into an executable file as the model file for each AI model and stored as a candidate model file. For example, different inference engine frameworks can have corresponding model conversion tools. After training an AI model based on a certain inference engine framework, the AI ​​model can be converted into an executable model file using the model conversion tool corresponding to that inference engine framework.

[0100] In one exemplary embodiment, such as Figure 2 As shown, for various business scenarios such as recording and gaming, and for each business requirement within these scenarios, datasets collected through the business backend service are used. After being labeled by relevant personnel, this large amount of data is sent to the model production module. The model production module then generates multiple AI models for each business requirement, each using a different inference engine framework (or, alternatively, an accelerated proxy model). The model generation process involves model training, quantization, and tuning. Then, one or more model conversion tools can be used to convert each AI model into a corresponding model file, and the models can be managed via a server (such as a model web management platform).

[0101] Since multiple AI models trained based on different inference frameworks and their corresponding candidate model files have been pre-acquired, after determining the client's current business needs, the target model file can be selected from the candidate model files based on the client's model runtime environment. The model runtime environment can represent the hardware and / or software environment used to run the model. The candidate model files can be stored locally on the client or on the server, such as data stored on a model web management platform; alternatively, they can be partially stored locally on the client and partially on the server.

[0102] It is understandable that different clients and the terminal devices running those clients have different operating environments. For example, deploying the same client on different terminal devices, or installing different versions of the same client on the same device, can all affect the operation of the AI ​​model. In this embodiment, AI models that depend on different inference engines are pre-trained, and a target model file that matches the business requirements and the client's operating environment is matched based on multiple candidate model files. This allows the AI ​​model to be run using an inference engine that matches the client's operating environment, optimizing the AI ​​model's operating efficiency on the terminal device while meeting the client's business needs.

[0103] S103, Obtain the engine runtime resources related to the inference engine corresponding to the target model file.

[0104] Since different model files rely on different inference engines, and different inference engines require different engine running resources, after obtaining the target model file, the inference engine corresponding to the target model file can be determined, that is, the inference engine that the AI ​​model corresponding to the target model file relies on, and then the engine running resources related to the inference engine can be dynamically called.

[0105] In some exemplary examples, engine runtime resources may include Native SO (Native SharedObject) library files, which may include underlying C++ library files called by the Android client NDK (Native Developer Kits).

[0106] S104 runs the target AI model corresponding to the target model file on the client, based on engine runtime resources and target model files.

[0107] Since the inference engine can utilize the corresponding engine runtime resources for data processing, and the target model file can store the data processing logic of the artificial intelligence model, in this step, the target artificial intelligence model corresponding to the target model file can be run on the client based on the engine runtime resources and the target model file.

[0108] In the aforementioned AI model operation method, upon detecting a trigger event that calls the AI ​​model, the client's current business requirements can be determined. Then, a target model file matching the business requirements and the client's model runtime environment can be obtained from multiple candidate model files. These candidate model files correspond to different AI models, and each AI model relies on an inference engine with a corresponding model runtime environment. Furthermore, the engine runtime resources related to the inference engine corresponding to the target model file can be obtained. Based on the engine runtime resources and the target model file, the target AI model corresponding to the target model file is run on the client. In this embodiment, by obtaining a target model file matching the business requirements and the client's model runtime environment from multiple candidate model files, a target AI model that meets the business requirements and whose inference engine matches the model runtime environment can be dynamically loaded. Subsequently, by obtaining the engine runtime resources related to the inference engine and running the target AI model in conjunction with the target model file, the operation mode of the AI ​​model matches the actual model runtime environment. This satisfies the inference needs of various AI models in the differentiated model runtime environments of terminal devices, improving the operating efficiency of AI models on different terminal devices.

[0109] In one embodiment, S102, obtaining the target model file that matches the business requirements and the client's model runtime environment from multiple candidate model files may include the following steps:

[0110] S301, Determine the model functionality that matches the business requirements.

[0111] In practical applications, for each trained artificial intelligence model and its corresponding model file, the models can be classified according to the model functions they can provide, thereby dividing multiple artificial intelligence models into artificial intelligence models belonging to different model functions.

[0112] Furthermore, after obtaining the client's current business requirements, the model functions that satisfy those requirements can be used as the model functions for matching those requirements. For example, for the business requirement of obtaining karaoke rating results, the matching model function could be a "good-sounding" score.

[0113] S302, if the client stores a list of valid models corresponding to the model function, then obtain the target model identifier in the list of valid models; the inference engine on which the artificial intelligence model corresponding to the target model identifier depends matches the client's model running environment.

[0114] In practical applications, after determining the model functions that match the business requirements, you can first check whether the client has the relevant model files stored locally.

[0115] Specifically, such as Figure 3 As shown, a valid model list (EffectModelDB) and an invalid model list (InvalidModelDB) can be configured. In one example, the client can configure a model file management module to manage the valid and invalid model lists. The valid model list stores the model identifier (ModelUnique) of all valid models issued by the server for each model function; that is, the unique identifier for each AI model. The identifiers in the valid model list can be sorted by insertion time.

[0116] In this step, after determining the model functionality, the client can query the list of valid models based on the model functionality identifier (ModelMainKey). When the list of valid models is obtained, such as... Figure 4 As shown, you can first check if the list of valid models is empty. If it is, you can determine that it failed to obtain a valid local model. You can then obtain the target model file by sending a model loading request to the server.

[0117] If the list of valid models is not empty, the target model identifier can be obtained from the list. The AI ​​model corresponding to the target model identifier must have an inference engine that matches the client's model runtime environment, and the model file of that AI model must be pre-loaded locally on the client. In some examples, model files previously obtained from the server can be stored locally on the client, and the corresponding model identifier can be added to the list of valid models.

[0118] S303: Based on the target model identifier, perform a model file query on the client side to obtain the target model file.

[0119] After obtaining the target model identifier, you can attempt to perform a file query on the client's local machine using this target file identifier to verify whether the model file mapped to the target model identifier exists. If no target model file is found based on the target model identifier, it can be determined that the local valid model file acquisition failed, and the target model identifier should be removed from the list of valid models. If the corresponding model file is found based on the target model identifier, that model file can be used as the target model file.

[0120] In some exemplary embodiments, the model identifier may include information representing the model functionality (ModelMainKey or mainkey), the model acceleration proxy mode (subkey), the inference engine type (enginetype), and the model version number (version). For example, the model identifier's organization structure can be "mainkey_subkey_enginetype_version". For instance, the model identifier for a sound quality scoring model could be "aiRecordScore_Normal_tfliteEngine_0.tflite". When upper-layer business accesses the system, only the mainkey needs to be considered. The subkey can represent a specific model identifier for the same mainkey under a specified acceleration proxy mode. When the operators supported by the acceleration proxy mode differ, the corresponding model files may also differ. When the model does not have an acceleration proxy mode distinction, the subkey can default to "Normal".

[0121] like Figure 5 As shown, the model identifier (ModelUnique), the model identifier data class (AiModelIdentifier), and the local storage path of the model file (ModelLocalFilePath) are in a one-to-one correspondence and can be converted to each other. Based on this, after obtaining the model identifier, the storage path corresponding to that model identifier can be determined, and the model file can be queried in the storage space corresponding to that storage path. In addition, the AiModelIdentifier can be used to parse the inference engine configuration parameters (AiInterpretCommonEngineConfig) required for model operation, as well as the extended SO libraries (ExtendNativeLibsType) required for inference engine operation.

[0122] In this embodiment, by obtaining the target model identifier from the list of valid models and querying the model file on the client based on the target model identifier, the target model file can be obtained quickly based on the previously loaded and cached model files in local resources, effectively improving the model loading efficiency of the artificial intelligence model corresponding to the business requirements.

[0123] In one embodiment, the target file identifier is the identifier of the most recently added file. After step S303, the method may further include the following steps:

[0124] Send a model update request to the server; the model update request can be used to trigger the server to determine the recommended AI model based on the target model identifier and model runtime environment carried in the model update request, and obtain the model comparison result between the recommended AI model and the target AI model; when the model comparison result indicates that the recommended AI model is better than the target AI model, store the model file corresponding to the recommended AI model returned by the server, and add the model identifier corresponding to the recommended AI model to the list of valid models.

[0125] In practical applications, after obtaining the target model file, it's possible to check if the target AI model has been updated. Specifically, a model update request can be sent to the server, carrying the target model identifier and the model's runtime environment. The server can determine the recommended AI model based on the target model identifier and runtime environment in the update request. For example, the client's model functionality (mainkey) can be read from the target model identifier. In another example, the model update request can also carry the model functionality alone. Thus, the server can determine the AI ​​model that matches the business requirements and the client's runtime environment from multiple candidate AI models. For ease of distinction, this AI model is also called the recommended AI model.

[0126] Then, the recommended AI model can be compared with the target AI model corresponding to the target model identifier to obtain the model comparison result. In some embodiments, such as Figure 2 As shown, during model production, model evaluation tools can be used to obtain the model's performance evaluation results under each evaluation metric. Then, the model evaluation result is determined based on these results. For example, each trained AI model can be evaluated using a model benchmark tool, which assigns a score priority. The priority determines whether further model optimization should proceed. Here, model benchmark refers to the benchmark performance evaluation of the model, including evaluations of metrics such as model execution time and memory usage. Furthermore, when comparing the recommended AI model with the target model, the comparison result can be determined based on the evaluation results of both.

[0127] When the model comparison result indicates that the recommended AI model is inferior to the target AI model, the server may not return a new model file. Conversely, when the model comparison result indicates that the recommended AI model is superior to the target AI model, the server can store the model file corresponding to the recommended AI model and add it to the list of valid models. This allows for timely updates to the local model file of the matched AI model, ensuring that when retrieving AI models from the list of valid models again, the latest AI model can be obtained based on the most recently added model identifier. Furthermore, by triggering the model update only after determining the target model file, the normal operation of the current target AI model can be avoided.

[0128] In one embodiment, after running the target AI model corresponding to the target model file on the client based on engine runtime resources and the target model file, the method further includes:

[0129] When retrieving the target model file from the list of valid models again, query the client's historical model file based on the historical model identifier in the list of valid models; then delete the historical model file and the historical model identifier.

[0130] Among them, the historical model identifier is the model identifier added to the list of valid models later than the target model identifier. The model file corresponding to the historical model identifier is the historical model file. The historical model file can be understood as the model file that was stored before the model update in the same model function, and its loading time or update time is earlier than the target model file.

[0131] In practice, after a model update, historical model files can be deleted to prevent irrelevant files from consuming limited local storage resources. However, in some cases, when the latest target model file is loaded onto the client, the client is running an AI model based on a previously acquired historical model file. For example, while the client is running the corresponding AI model based on the latest locally acquired model file 1, a model update may be triggered, and model file 2 may be retrieved from the server. This means the latest local model file will replace model file 1 with model file 2. If model file 1 is deleted directly at this time, it will affect the normal operation of the corresponding AI model and interfere with the client's normal business processing.

[0132] To address this, when retrieving the target model file from the list of valid models again, the historical model identifier in the list can be determined, the client's historical model file can be queried based on the historical model identifier, and then the historical model file can be deleted, along with the historical model identifier being removed from the list of valid models. This avoids impacting the current AI model's operation and promptly deletes redundant historical model files on the client's local machine, preventing the consumption of limited storage resources.

[0133] In one embodiment, the model update request can also trigger the server to obtain validity information of the target AI model, which can indicate whether the target AI model has failed. After obtaining the validity information, the method may further include the following steps:

[0134] If the validity information indicates that the target AI model has failed, add the target model identifier to the list of failed models, and / or delete the target model file stored on the client.

[0135] In practice, the server may remove the AI ​​model in the background, rendering it invalid. Consequently, the model file corresponding to that AI model on the client will no longer be used. In this embodiment, if the client already has the target model file stored, it can also trigger the server to provide the validity information of the corresponding target AI model through a model update request. When the server receives the update trigger request, it can query the validity information of the corresponding target AI model based on the target model identifier carried in the request and then feed it back to the client.

[0136] If the validity information indicates that the target AI model is valid, the target model file and its corresponding identifier can continue to be stored locally. If the validity information indicates that the target AI model is invalid, the client can add the target model identifier to the invalid model list, recording that the corresponding target AI model has become invalid. This invalid model list can be used to store the model identifiers of globally notified invalid models in the background. Figure 3 As shown, for multiple AI models that have been declared invalid by the server, their respective model identifiers can be added to the invalid model list. When querying the local machine again for a model file that matches the current business requirements, such as... Figure 4 As shown, you can first search the list of failed models using MainKey. If a model is found, delete the corresponding model file and then remove the corresponding model identifier from the local list of failed models.

[0137] In other embodiments, if the validity information indicates that the target artificial intelligence model has expired, the client can also delete the corresponding target model file locally, so that the server can be triggered to provide the model file corresponding to the valid artificial intelligence model again when the target artificial intelligence model is used next time.

[0138] In this embodiment, the validity information provided by the server is obtained through the model update request. On the one hand, the backend server can dynamically remove the artificial intelligence model, and on the other hand, the client can know the validity status of the model in a timely manner. By adding the target model identifier to the list of invalid models or deleting the target model file stored on the client, the invalid models that have been removed from the client's local storage can be removed in a timely manner, avoiding errors in the business requirement response process due to the use of models that have been removed from the backend.

[0139] In one embodiment, S102, obtaining the target model file that matches the business requirements and the client's model runtime environment from multiple candidate model files, may include the following steps:

[0140] Determine the model function that matches the business requirements; when the client does not store the model file of the model function, send a model loading request to the server carrying the model function and the client's model runtime environment information; the model loading request is used to trigger the server to determine the target artificial intelligence model that matches the model runtime environment from multiple candidate artificial intelligence models under the model function, and obtain the target model file corresponding to the target artificial intelligence model; receive the target model file returned by the server.

[0141] In practice, after determining the client's business requirements, the model functionality matching those requirements can be identified. Then, the client can be queried to see if a relevant model file is stored locally. If a relevant model file is found, the target model file can be obtained according to the methods described in one or more of the aforementioned embodiments.

[0142] If the client does not store the model file containing the model functionality, the client can send a model loading request to the server, carrying the model functionality and the client's model runtime environment information. After receiving the request, the server can determine the target artificial intelligence model that matches the model runtime environment from multiple candidate artificial intelligence models with the model functionality, obtain the target model file corresponding to the target artificial intelligence model, and then return the target model file to the client.

[0143] In this embodiment, by sending a model loading request to the server and receiving the target model file returned by the server, the server's computing resources can be used to flexibly and quickly obtain a target model file that matches the client's business needs and the model's operating environment. This helps clients on different terminal devices to efficiently meet their business needs by running different artificial intelligence models in a diversified manner.

[0144] In some embodiments, such as Figure 4 As shown, when obtaining a valid model file, the business side only needs to pass the MainKey identifier. The client's internal SDK (Software Development Kit) will then find the latest local model. If no local model is available or during model update checks, a model download task, AiModelDownloadTask, can be triggered. When requesting the backend, this task will carry the model identifier of the latest local target AI model, as well as information about the model's runtime environment, including machine environment information and the local supported inference engine strategy. The data returned by the backend server can include two parts: whether the client's latest target AI model is valid, and whether there is a new available model, NewModelParam. If a new available model exists, after successfully downloading the NewModelParam file, it will check whether the HitUseStrategy (the local inference engine strategy) already supports the model's operation. If not, it will initiate the download of the required engine runtime resources (such as ExtendNativeLibsType, an extension library file, or a type of target resource file).

[0145] In one embodiment, before obtaining the target model file that matches the business requirements and the client's model runtime environment from multiple candidate model files, the client's model runtime environment information can be obtained through the following steps:

[0146] Obtain client attribute information and device attribute information of the terminal device on which the client is deployed; obtain the running strategies of each candidate inference engine that can be supported locally; determine the model running environment of the client based on the client attribute information, device attribute information, and candidate inference engine running strategies.

[0147] In practical applications, different client versions may affect the running efficiency of artificial intelligence models. This step allows us to obtain client attribute information, which can reflect client performance or characterize the client software's operating environment. In some examples, such as... Figure 6As shown, the client attribute information includes at least one of the following: client version, client instruction set architecture (AppBits, such as 32-bit or 64-bit), and the underlying C++ library for client AI model inference.

[0148] On the other hand, the device performance of the terminal device where the client is deployed also affects the model's operation. This embodiment can also obtain the device attribute information of the terminal device where the client is deployed. In some examples, such as... Figure 6 As shown, device attribute information can reflect the terminal device's machine environment, including at least one of the following: device platform type (such as iOS or Android), device model, developer, system version, chip model or chip type, and evaluation score that characterizes the device's performance (such as device performance score).

[0149] Furthermore, inference engines can run on different hardware on terminal devices. Therefore, the same inference engine can have different inference engine running strategies, also known as inference strategies. For example, inference engine 1 can run on the terminal device's CPU (Central Processing Unit) or GPU (Graphics Processing Unit), while inference engine 2 only runs on the CPU. In some embodiments, when there are multiple inference running strategies for the inference engine, corresponding artificial intelligence models can be trained and corresponding model files can be generated for different inference running strategies. At the same time, different terminal devices with clients deployed have different resources or configurations, and therefore, the inference engine running strategies that the terminal devices can actually support also differ. For ease of distinction, the inference engine running strategy actually used by the terminal is called the candidate inference engine running strategy (also known as the native support strategy). The candidate inference engine strategy can be understood as an inherent attribute of the device, mainly determined by the version of the relevant configuration in the client and the machine environment of the terminal device. It is determined during global initialization (such as global initialization of the software development kit) after the client starts and remains unchanged during client use.

[0150] In this embodiment, the execution strategies of each candidate inference engine locally supported by the terminal device can also be obtained. Then, the model execution environment of the client can be determined based on the client attribute information, device attribute information, and candidate inference engine execution strategies. Figure 6As shown, the acquired client attribute information, device attribute information, and candidate inference engine running strategies can be sent to the Web model configuration platform (i.e., the server). The Web model configuration platform can then determine the target AI model from multiple AI models with different functions based on the information provided by the client and the configuration information of each model under the corresponding model function. For example, the platform can pre-train corresponding AI models for sound quality scoring and AEC noise reduction. Taking the sound quality scoring model as an example, models under inference engine framework 1 and inference engine framework 2 can be trained separately. Inference engine framework 1 can be further subdivided into CPU-based and GPU-based scenarios. In some examples, if multiple AI models match the business requirements and model running environment, the target AI model can be selected based on the model's usage priority in the model set (such as the priority determined by model evaluation results).

[0151] In this embodiment, by determining the client's model operating environment based on client attribute information, device attribute information, and candidate inference engine operating strategy, the accuracy of model operating environment identification can be improved, making it easier to more accurately and meticulously determine the target artificial intelligence model that matches the specific environment of the terminal device.

[0152] In one embodiment, S103, obtaining the engine runtime resources related to the inference engine corresponding to the target model file, may include the following steps:

[0153] Determine the target inference engine's runtime strategy for the target model file locally; generate a resource loading request based on the target inference engine's runtime strategy and send the resource loading request to the server; the resource loading request is used to trigger the server to obtain the target resource files required for the inference engine corresponding to the target model file to run according to the target inference engine's runtime strategy; obtain the engine runtime resources based on the target resource files returned by the server.

[0154] After obtaining the target model file, the target inference engine running strategy of the target model file on the client's local machine can be determined. The target inference engine running strategy is the inference engine running strategy that the inference engine corresponding to the target model file actually implements locally, also known as the local usable inference engine strategy (HitUseStrategy).

[0155] like Figure 7 As shown, the inference engine's operating strategy can be divided into locally supported inference strategies and locally usable inference strategies. The inference strategies usable in this instance are a subset of the locally supported inference strategies. Figure 7In the example shown, locally supported inference strategies include running the TFLite inference engine or the Mnn inference engine on the CPU or GPU, and running the Onnx inference engine on the CPU; locally available inference strategies are dynamically expanded on demand during runtime, and relevant resources are loaded when a specific model requires a corresponding strategy.

[0156] Then, a resource loading request can be generated based on the determined target inference engine's runtime strategy and sent to the server. In response to the received resource loading request, the server can obtain the target resource file, which is the resource file required by the inference engine corresponding to the target model file to run according to the target inference engine's runtime strategy.

[0157] In some embodiments, the artificial intelligence model operation method provided in this application can be encapsulated as a client-side software development kit (SDK). The SDK framework (AiInterpretSdk) can be as follows: Figure 8 As shown.

[0158] In this example, the C++ library is the underlying C++ library used during the client-side AI model inference process, providing the lowest-level inference engine capabilities. Internally, it can encapsulate various engines and available accelerated proxy logic, providing a unified inference capability interface to the outside. The client-side Java layer inference framework, AiInterpretSdk, provides common upper-level inference encapsulation interfaces (such as audio, images, and video). Business access parties can obtain model inference results by calling the SDK's simple interfaces. The SDK consists of several core modules, including the AiInterpretCommonEngine module, ModelFileDownloadManager module, ModelOperateStrategyManager module, NativeLibsDynamicLoad module, InterpretProcessor module, and TechniqueReporter module.

[0159] The AiInterpretCommonEngine interfaces with the underlying unified engine module via JNI (Java Native Interface) and encapsulates model data processing. The ModelFileDownloadManger module manages model file downloads, updates, and caching. The ModelOperateStrategyManger module determines the target AI model available for specific business scenarios or needs based on the local model runtime environment and backend model configuration. In other words, in some embodiments, the client can determine the target AI model from multiple candidate AI models based on the model runtime environment and business requirements. The NativeLibsDynamicLoad module enables dynamic loading and management of library files. Since various frameworks support multiple inference capabilities and involve numerous SO library files, this module allows for dynamic loading before use. The InterpretProcessor provides an interface for business layer calls. The TechniqueReporter module reports model-related loading success rates, inference time, etc.

[0160] Since multiple inference engines involve different engine runtime resources, they are generally not all loaded when terminal device storage resources are limited. In some examples, a default inference engine runtime strategy that needs to be supported can be set according to client requirements, and the relevant engine runtime resources can be included by default when generating the SDK underlying library. For example... Figure 9 As shown, after the app starts, global SDK initialization can be performed. During the initialization of the model execution strategy module, the locally supported candidate inference engine execution strategies can be determined based on the underlying C++ library version and machine environment. Simultaneously, the initialization process can load core engine execution resources and the engine execution resources required by the default supported inference engine execution strategies. The core engine execution resources can include the common core library CommonCoreLibrary, which contains the most core common basic logic code. This can be included in the client's compressed packaging or dynamically downloaded and pre-loaded during SDK global initialization. For other specific inference engine inference capability extension libraries, such as... Figure 10 As shown, since different terminal devices and business scenarios require different capabilities, they can be dynamically loaded on demand to maximize operating efficiency.

[0161] Since some engine runtime resources have been pre-loaded locally, the client can generate resource loading requests for other missing engine runtime resources, thereby obtaining the other missing resource files from the server and feeding them back to the client as target resource files.

[0162] In this embodiment, by generating a resource loading request according to the target inference engine's running strategy and sending the resource loading request to the server, the relevant resources required by the inference engine to run according to the target inference engine's running strategy can be dynamically loaded. This allows for flexible use of different inference engines and engine running strategies to run artificial intelligence models on different terminal devices to meet the client's business needs, greatly increasing the adaptability of the artificial intelligence model's running mode to the client's model running environment.

[0163] In one embodiment, step S104 may include the following steps:

[0164] Obtain the engine configuration parameters of the inference engine corresponding to the target model file; create an inference processor on the client based on the engine configuration parameters, engine runtime resources, and target model file; and run the target artificial intelligence model corresponding to the target model file using the inference processor.

[0165] In this step, after obtaining the target model file, the engine configuration parameters (AiInterpretCommonEngineConfig) of the inference engine corresponding to the target model file can be determined first. These engine configuration parameters characterize the inference engine configuration required for model operation. Then, combining these engine configuration parameters, pre-acquired engine runtime resources, and the target model file, an inference processor (InterpretProcessor) can be created on the client. For example, the inference processor can include at least one of an audio inference processor, an image inference processor, and a video inference processor. The inference processor can encapsulate a common workflow used by a business processing model. Taking an audio processing model as an example, the inference processor can be configured with streaming input source data (PCM), and the model runs in a single-input, single-output manner, enabling rapid integration and use. When the inference processor is created, the model can be retrieved from the cache, and then the download and update of the model can be checked synchronously or asynchronously, thereby avoiding blocking the current model call and exhibiting the characteristic of delayed model file processing.

[0166] like Figure 11As shown, after obtaining the target artificial intelligence model (EffectModel), the library files (ExtendNativeLib) required for the model to run can be loaded, the underlying modules of the C++ library can be initialized, and HitUseStrategy can be dynamically updated as needed. Then, it is determined whether HitUseStrategy meets the running requirements of the target artificial intelligence model. In some cases, failure to download or loading of the library files will cause the inference processor creation to fail.

[0167] After creating an inference processor, the target artificial intelligence model can be run using the inference processor. For example, such as Figure 12 As shown, the InterpretProcessor contains the file content of the target model file (EffectModelData), the process parameters used by the business processing model (InterpretProcessConfig, such as audio data format, DataInterceptor data processing plugin, etc.), and the engine configuration parameters (AiInterpretCommonEngineConfig). The InterpretProcessor can transmit the engine configuration parameters to the underlying C++ library for engine initialization, then load the model, parse out the relevant model data, and subsequently initialize the data processing buffer module and the DataInterceptor module. Then, during business-related inference, the source data, after being processed by the DataInterceptor and standardized, can be input into the model to trigger the inference task.

[0168] In this embodiment, by creating an inference processor on the client based on engine configuration parameters, engine running resources, and target model files, it is possible to dynamically run the artificial intelligence model in different ways on the same terminal device, thereby improving the flexibility of different artificial intelligence model running methods and the adaptability to the client model running environment, and effectively improving model running efficiency.

[0169] In one embodiment, a method for running an artificial intelligence model is provided. This embodiment illustrates the application of this method to a server. This embodiment includes the following steps:

[0170] S401, Receive model loading request from client; the model loading request is sent by client when it detects a trigger event that calls the artificial intelligence model.

[0171] In a specific implementation, after detecting a trigger event that calls an AI model, the client can send a model loading request to the server to obtain the corresponding AI model file. In some embodiments, the client can send a model loading request to the server every time it obtains an AI model, or it can send a model loading request only when it is determined that the client's local machine lacks the corresponding model file.

[0172] S402, in response to a model loading request, retrieves a target model file from multiple candidate model files that matches the client's current business requirements and the client's model runtime environment.

[0173] In some embodiments, upon receiving a model loading request, the server can match the target model file from multiple candidate model files based on the client's current business needs and the model's runtime environment. In other embodiments, the client can also match its own business needs and the configuration information corresponding to multiple artificial intelligence models to determine the target artificial intelligence model, and add the model identifier of the target artificial intelligence model to the model loading request, thereby specifying the server to obtain the target model file corresponding to the model identifier from multiple candidate model files.

[0174] S403, return the target model file to the client to trigger the client to obtain the engine running resources related to the inference engine corresponding to the target model file, and run the target artificial intelligence model corresponding to the target model file on the client based on the engine running resources and the target model file.

[0175] After obtaining the target model file, the server can send the target model file to the client. Then, the client can obtain the relevant engine runtime resources and run the corresponding target artificial intelligence model on the client based on the engine runtime resources and the target model file.

[0176] In the above-described AI model operation method, a model loading request can be received from a client. This request is sent by the client upon detecting a trigger event that calls an AI model. In response to the loading request, a target model file matching the client's current business requirements and model operation environment is obtained from multiple candidate model files. The target model file is then returned to the client, triggering the client to acquire engine operation resources related to the inference engine corresponding to the target model file. Based on the engine operation resources and the target model file, the target AI model corresponding to the target model file is run on the client. In this embodiment, by acquiring a target model file matching the business requirements and the client's model operation environment from multiple candidate model files, a target AI model that meets the business requirements and whose inference engine matches the model operation environment can be dynamically loaded. Subsequently, by acquiring the engine operation resources related to the inference engine and running the target AI model in conjunction with the target model file, the operation mode of the AI ​​model matches the actual model operation environment. This satisfies the inference requirements of various AI models in the differentiated model operation environments of terminal devices, improving the operating efficiency of AI models on different terminal devices.

[0177] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0178] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores model data and inference engine data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an artificial intelligence model execution method.

[0179] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 14 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an artificial intelligence model operation method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0180] Those skilled in the art will understand that Figure 13 and Figure 14The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0181] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0182] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0183] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0184] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0185] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0186] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0187] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for running an artificial intelligence model, characterized in that, The method includes: Upon detecting a trigger event that invokes the artificial intelligence model, determine the client's current business requirements; Obtain the client attribute information of the client and the device attribute information of the terminal device on which the client is deployed; The client obtains multiple candidate inference engine operation strategies that are locally supported; the multiple candidate inference engine operation strategies include the operation strategies of the same inference engine on different hardware of the same terminal device, and the operation strategies of different inference engines on the same terminal device. The model runtime environment of the client is determined based on the client attribute information, the device attribute information, and the candidate inference engine running strategy; Target model files that match the business requirements and the client's model runtime environment are obtained from multiple candidate model files; the multiple candidate model files correspond to different artificial intelligence models, and the inference engine on which each artificial intelligence model depends is matched with a corresponding model runtime environment; Obtain the engine runtime resources related to the inference engine corresponding to the target model file; Based on the engine's runtime resources and the target model file, the target artificial intelligence model corresponding to the target model file is run on the client.

2. The method according to claim 1, characterized in that, The step of obtaining a target model file that matches the business requirements and the client's model runtime environment from multiple candidate model files includes: Determine the model functions that match the stated business requirements; If the client stores a list of valid models corresponding to the model function, then the target model identifier in the list of valid models is obtained; the inference engine on which the artificial intelligence model corresponding to the target model identifier depends matches the model running environment of the client. The target model file is obtained by querying the model file on the client based on the target model identifier.

3. The method according to claim 2, characterized in that, The target model identifier is the most recently added model identifier; After querying the model file on the client based on the target model identifier to obtain the target model file, the process further includes: Send a model update request to the server; the model update request is used to trigger the server to determine a recommended artificial intelligence model based on the target model identifier and the model running environment carried in the model update request, and to obtain the model comparison result between the recommended artificial intelligence model and the target artificial intelligence model; When the model comparison result indicates that the recommended AI model is superior to the target AI model, the model file corresponding to the recommended AI model returned by the server is stored, and the model identifier corresponding to the recommended AI model is added to the list of valid models.

4. The method according to claim 3, characterized in that, After running the target AI model corresponding to the target model file on the client based on the engine's runtime resources and the target model file, the process further includes: When retrieving the target model file based on the list of valid models again, the client's historical model file is queried based on the historical model identifier in the list of valid models; the historical model identifier is a model identifier whose addition time is later than the addition time of the target model identifier. Delete the historical model file and the historical model identifier.

5. The method according to claim 3, characterized in that, The model update request is also used to trigger the server to obtain the validity information of the target artificial intelligence model; The method further includes: If the validity information indicates that the target AI model has failed, the target model identifier is added to the list of failed models, and / or the target model file stored on the client is deleted.

6. The method according to claim 1, characterized in that, The step of obtaining a target model file that matches the business requirements and the client's model runtime environment from multiple candidate model files includes: Determine the model functions that match the stated business requirements; When the client does not store the model file for the model function, a model loading request carrying the model function and the client's model runtime environment information is sent to the server; the model loading request is used to trigger the server to determine the target artificial intelligence model that matches the model runtime environment from multiple candidate artificial intelligence models under the model function, and obtain the target model file corresponding to the target artificial intelligence model; Receive the target model file returned by the server.

7. The method according to claim 1, characterized in that, The step of obtaining the engine runtime resources related to the inference engine corresponding to the target model file includes: Determine the target inference engine execution strategy for the target model file locally; A resource loading request is generated according to the target inference engine running strategy, and the resource loading request is sent to the server; the resource loading request is used to trigger the server to obtain the target resource file required for the inference engine corresponding to the target model file to run according to the target inference engine running strategy; Based on the target resource file returned by the server, the engine running resources are obtained.

8. The method according to any one of claims 1 to 7, characterized in that, The step of running the target artificial intelligence model corresponding to the target model file on the client based on the engine's runtime resources and the target model file includes: Obtain the engine configuration parameters of the inference engine corresponding to the target model file; Based on the engine configuration parameters, the engine runtime resources, and the target model file, an inference processor is created on the client. The inference processor is used to run the target artificial intelligence model corresponding to the target model file.

9. A method for running an artificial intelligence model, characterized in that, The method includes: Receive a model loading request from the client; the model loading request is sent by the client upon detecting a trigger event that calls the artificial intelligence model; Obtain the client attribute information of the client and the device attribute information of the terminal device on which the client is deployed; The client obtains multiple candidate inference engine operation strategies that are locally supported; the multiple candidate inference engine operation strategies include the operation strategies of the same inference engine on different hardware of the same terminal device, and the operation strategies of different inference engines on the same terminal device. The model runtime environment of the client is determined based on the client attribute information, the device attribute information, and the candidate inference engine running strategy; In response to the model loading request, a target model file matching the client's current business requirements and the client's model runtime environment is obtained from multiple candidate model files; The target model file is returned to the client to trigger the client to obtain the engine running resources related to the inference engine corresponding to the target model file, and to run the target artificial intelligence model corresponding to the target model file on the client based on the engine running resources and the target model file.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the artificial intelligence model running method according to any one of claims 1 to 8 or the artificial intelligence model running method according to claim 9.

Images