A model reasoning method, device and equipment based on artificial intelligence

Abstract

The present invention discloses a model reasoning method, device and equipment based on artificial intelligence, which includes: writing a reasoning file including a model loading method and a model reasoning method; uploading the reasoning file and the model to be reasoned to a reasoning platform built based on Python; using the model loading method to load the model to be reasoned through the reasoning platform, and using the model reasoning method to perform parsing processing according to the reasoning request sent by the client, so that the model to be reasoned performs reasoning operations and obtains model reasoning results. This application can simplify the user's operating steps and simplify the complex configuration process, and realize the deployment and reasoning services of general and lightweight models that are easier to use and more efficient.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based model reasoning method, device and equipment. Background Art

[0002] With the rapid development of artificial intelligence technology, the efficiency and flexibility of inference services, as the key link between model training and practical applications, have become important indicators for measuring the performance of AI systems. In the current market, NVIDIA's Triton Inference Server, TensorFlow Serving, and KServe and other inference frameworks have occupied an important position in the industry with their unique advantages, promoting the efficient deployment and application of AI models.

[0003] However, despite the significant progress these frameworks have made in improving inference performance and simplifying deployment processes, they still face a series of challenges and limitations, especially in terms of ease of use, compatibility, and deployment complexity. Take NVIDIA's Triton Inference Server as an example. Although its powerful features support multi-framework model deployment and performance optimization, its complex configuration and high learning threshold limit the rapid start-up of novice users. Similarly, although TensorFlow Serving is designed for TensorFlow models and provides efficient service capabilities, it fails to effectively support popular frameworks such as PyTorch, limiting its scope of application. KServe, as an open source project based on Kubernetes, aims to simplify the deployment and management of machine learning models through cloud-native technologies. Although it provides high flexibility and scalability, the complexity of Kubernetes and related technologies significantly increases the learning cost of users, and the collaborative configuration and debugging process of multiple components is cumbersome, which brings considerable challenges to the initial deployment. In addition, the architecture that relies on multiple external components also increases the complexity and maintenance difficulty of the system.

[0004] In view of the above problems, the current field of reasoning frameworks urgently needs a solution that is compatible with multiple machine learning frameworks and can significantly reduce the user learning curve and deployment difficulty while ensuring efficient reasoning performance. Summary of the invention

[0005] In view of this, the purpose of the present invention is to propose a model reasoning method, device and equipment based on artificial intelligence, aiming to solve how to overcome the limitations of the existing reasoning framework and realize a general, lightweight, easier to use and efficient reasoning service.

[0006] To achieve the above object, the present invention provides a model reasoning method based on artificial intelligence, the method comprising:

[0007] Write inference files including model loading methods and model inference methods;

[0008] Upload the inference file and the model to be inferred to the inference platform built based on Python;

[0009] The model to be inferred is loaded by the inference platform using the model loading method, and the model inference method is used to perform parsing processing according to the inference request sent by the client, so that the model to be inferred performs the inference operation and obtains the model inference result.

[0010] Preferably, the method of loading the model to be inferred by the inference platform using the model loading method, and performing parsing processing according to the inference request sent by the client using the model inference method, includes:

[0011] An inference service launcher is created in the inference platform, the inference service launcher is used to launch the inference process according to the assembled launch command, and the inference process is used to execute the tasks in the model loading method and the model inference method.

[0012] Preferably, the inference process includes a parameter parsing engine; based on the parameter parsing engine, the argparse module in Python is used to parse the command line parameters of the startup command to obtain the path of the inference file, the file path of the model to be inferred, the prediction endpoint URI, the running port, and the supported HTTP methods.

[0013] Preferably, the inference process includes an inference file dynamic loading engine; based on the inference file dynamic loading engine, the importlib module and the sys module in Python are used to dynamically load the inference file of the corresponding path, and the inference file is dynamically loaded into the memory, so that the inference process executes the model loading method and the tasks in the model inference method.

[0014] Preferably, the inference process includes a model file loading engine, and the inference file is executed based on the model file loading engine to call the model loading method to load the model to be inferred into the memory or video memory.

[0015] Preferably, the inference process includes a web service engine, a web service is started based on the web service engine, the model inference method is executed according to the request parameters in the inference request through the web service, and the request parameters are sent to the input parameters of the model inference method.

[0016] To achieve the above object, the present invention also provides a model reasoning device based on artificial intelligence, the device comprising:

[0017] A writing unit, used for writing an inference file including a model loading method and a model inference method;

[0018] A transmission unit, used for uploading the inference file and the model to be inferred to an inference platform built based on Python;

[0019] The inference unit is used to load the model to be inferred by using the model loading method through the inference platform, and use the model inference method to perform parsing processing according to the inference request sent by the client, so that the model to be inferred performs the inference operation and obtains the model inference result.

[0020] In order to achieve the above objectives, the present invention also proposes an artificial intelligence-based model reasoning device, including a processor, a memory, and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of an artificial intelligence-based model reasoning method as described in the above embodiment.

[0021] In order to achieve the above objectives, the present invention also proposes a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of an artificial intelligence-based model reasoning method as described in the above embodiment.

[0022] In order to achieve the above objectives, the present invention also proposes a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of an artificial intelligence-based model reasoning method as described in the above embodiment.

[0023] Beneficial effects:

[0024] The above solution writes an inference file containing model loading and inference methods and uploads it to the inference platform built on Python. Users do not need to have an in-depth understanding of the complex inference framework configuration. By simplifying the user's operation steps and reducing configuration requirements, the model can be deployed efficiently. This process greatly simplifies the complex configuration process of the traditional inference framework, lowers the usage threshold, allows users to get started quickly, and can meet the growing demand for artificial intelligence applications.

[0025] The inference service launcher parses the startup command to start the inference process, realizing the automated processing of inference tasks. At the same time, the inference file can be dynamically loaded into the memory for execution using Python's dynamic loading capabilities (such as the importlib module), and the inference logic can be updated without restarting the service, which improves the flexibility and response speed of the inference service.

[0026] This method supports models of multiple machine learning frameworks, is not limited to a specific training framework, and is applicable to a variety of machine learning and deep learning models. By writing the corresponding inference files, the model can be loaded and inferred, improving compatibility and flexibility. In addition, since the inference logic is decoupled from the inference platform, users can easily expand or modify the inference logic to adapt to different application scenarios and needs.

[0027] The above parameter parsing engine can accurately control the startup parameters of the inference service, such as the operating port, supported HTTP methods, etc., to optimize resource allocation and improve service performance. At the same time, loading the model into memory or video memory reduces disk I / O operations and further improves the inference speed.

[0028] The web service engine provides a unified interface to receive reasoning requests and executes the model reasoning method based on the request parameters, so that users can obtain reasoning results through simple HTTP requests, which improves the user experience. In addition, since the reasoning logic is separated from the web service, the coupling degree of the system is reduced, and the maintainability and scalability of the system are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 A flowchart of a model reasoning method based on artificial intelligence is provided for one embodiment of the present invention.

[0031] Figure 2 A schematic diagram of an overall process provided by an embodiment of the present invention.

[0032] Figure 3 A schematic diagram of the structure of an artificial intelligence-based model reasoning device provided for one embodiment of the present invention.

[0033] The realization of the purpose, functional features and advantages of the invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0035] The content of the present invention is described in detail below with reference to embodiments.

[0036] Reference Figure 1 The figure is a flow chart of a model reasoning method based on artificial intelligence provided by one embodiment of the present invention.

[0037] In this embodiment, the method includes:

[0038] S11, write an inference file including a model loading method and a model inference method.

[0039] In this embodiment, the user can write an inference file according to their needs and model type, such as inference.py. The inference file includes a model loading method: load_model(model_path) and a model inference method: predict(data). Among them, the load_model() method is responsible for loading the model and loading the model into the memory or video memory. In this method, the user can implement the logic of model loading according to the specific model loading requirements; the predict(data) method is responsible for processing the inference request, receiving the input data data, and using the loaded model to perform the inference operation. The data can be text, or it can be a picture, text file, etc. After predict receives the parameters passed by the user, the business code is written here to pass the data to the model (such as TensorFlow module, PyTorch model, ONNX model, etc.). The user only needs to write the business code here, without writing the web layer code and without paying attention to the service deployment details.

[0040] S12, uploading the inference file and the model to be inferred to an inference platform built based on Python.

[0041] In this embodiment, a Python-based reasoning platform is constructed, which can receive reasoning files uploaded by users and perform dynamic loading and parsing calls. Users only need to write reasoning files according to requirements and upload them to the reasoning platform. After receiving the reasoning files, the reasoning platform performs subsequent dynamic loading and parsing calls.

[0042] S13, loading the model to be inferred by the inference platform using the model loading method, and performing parsing processing according to the inference request sent by the client using the model inference method, so that the model to be inferred performs the inference operation and obtains the model inference result.

[0043] Furthermore, the inference platform uses the model loading method to load the model to be inferred, and uses the model inference method to perform parsing processing according to the inference request sent by the client, including:

[0044] An inference service launcher is created in the inference platform, the inference service launcher is used to launch the inference process according to the assembled launch command, and the inference process is used to execute the tasks in the model loading method and the model inference method.

[0045] Reference Figure 2 As shown. In this embodiment, an inference service launcher is designed on the inference platform. The inference service launcher can scan the available ports of the current system (inference platform) and assemble a startup command to start an inference process. For example, the user can specify the path of the inference file, the file path of the model, the prediction endpoint URI, the running port and the supported HTTP method through command line parameters. That is, the startup command:

[0046] python \

[0047] main.py \

[0048] --predict_file= / inference.py \

[0049] --mode_file= / model / \

[0050] --port=8080 \

[0051] --uri= / predict \

[0052] --methods=POST

[0053] Among them, --file represents the path of the inference file, --model_file represents the directory where the model file is located, --port represents the service port, --uri represents the prediction endpoint URI, and --methods represents the service request type. In particular, the model is usually loaded when the inference process is started, rather than loading the model every time the client sends an inference request.

[0054] Furthermore, the inference process includes a parameter parsing engine; based on the parameter parsing engine, the argparse module in Python is used to parse the command line parameters of the startup command to obtain the path of the inference file, the file path of the model to be inferred, the prediction endpoint URI, the running port, and the supported HTTP methods.

[0055] In this embodiment, the inference process includes a parameter parsing engine, an inference file dynamic loading engine, a model file loading engine, and a web service engine. Furthermore, there are multiple inference processes, and multiple inference processes can exist simultaneously by using different inference files and models. Among them, by designing a parameter parsing engine, the argparse module in Python is used to parse the command line parameters of the startup command to obtain the path of the inference file, the file path of the model, the prediction endpoint URI, the running port, and the supported HTTP methods.

[0056] Furthermore, the inference process includes an inference file dynamic loading engine; based on the inference file dynamic loading engine, the importlib module and the sys module in Python are used to dynamically load the inference file of the corresponding path, and the inference file is dynamically loaded into the memory, so that the inference process executes the model loading method and the tasks in the model inference method.

[0057] In this embodiment, by designing an inference file dynamic loading engine, the importlib module and the sys module in Python are used to dynamically load the inference file of the specified path, and the inference file is dynamically loaded into the memory (the inference file (code) is loaded into the memory), so that the inference process can execute the code in the inference file. Specifically, through the given module path (the path of the inference file, that is, the file path of inference.py), the importlib library is used to dynamically load and execute the inference file module. This dynamic module loading method enables the program to load a new inference module without restarting, thereby improving development efficiency and flexibility. First, the function creates a module specification (spec), and then creates an inference module object based on the module specification; next, the module object is added to the sys.modules dictionary for subsequent reference; finally, the module specification loader executes the module code to complete the loading process. This step returns the loaded inference module object inference.

[0058] Furthermore, the inference process includes a model file loading engine, and the inference file is executed based on the model file loading engine to call the model loading method to load the model to be inferred into the memory or video memory.

[0059] In this embodiment, a model file loading engine is designed, which can execute the inference file, call the load_model method, and load the model into the memory or video memory. Specifically, based on the above steps, the inference file has been loaded into the memory, and the inference process can execute the code in the inference file and return the inference module object inference. According to the inference specification, according to the load_model method in the inference file, combined with the model file model_file_path parsed by the above-mentioned command line parameter parsing step, inference.load_model(model_file_path) can be directly executed to load the model file into the memory or video memory.

[0060] Furthermore, the inference process includes a web service engine, based on which a web service is started, the model inference method is executed according to request parameters in the inference request through the web service, and the request parameters are sent to input parameters of the model inference method.

[0061] In this embodiment, a web service engine is designed to scan the ports available in the current system, use the port to open a new reasoning process, start the web service, and bind the prediction endpoint URI (for example, / predict) to the predict method in the reasoning file. The system obtains the path of the reasoning file, the prediction endpoint URI, the running port, and the supported HTTP method by parsing the command line parameters. The predict method is dynamically bound and used for reasoning, thereby realizing the binding of the URI and the predict method. When the client sends the reasoning request, the server can receive the reasoning request, receive the request parameters sent by the client, and execute the predict method in the reasoning file, and pass the parameters to the input parameters of the predict method.

[0062] Based on the model service reasoning capability, by sending reasoning requests through tools such as Postman, the system will return the model reasoning results, thereby realizing rapid verification of reasoning; or the business system sends reasoning requests to realize rapid deployment of model services without relying on the Kubernetes environment. This application makes model deployment more convenient and efficient by simplifying the user's operating steps and reducing configuration requirements, which can lower the threshold for model deployment and realize rapid launch of model services. It significantly improves the deployment efficiency and ease of use of machine learning models, and is suitable for various application scenarios that require rapid iteration and verification of models.

[0063] Reference Figure 3 Shown is a schematic diagram of the structure of an artificial intelligence-based model reasoning device provided by an embodiment of the present invention.

[0064] In this embodiment, the device 20 includes:

[0065] A writing unit 21, used for writing an inference file including a model loading method and a model inference method;

[0066] A transmission unit 22, used for uploading the inference file and the model to be inferred to an inference platform built based on Python;

[0067] The reasoning unit 23 is used to load the model to be inferred by using the model loading method through the reasoning platform, and use the model reasoning method to perform parsing processing according to the reasoning request sent by the client, so that the model to be inferred performs reasoning operations and obtains model reasoning results.

[0068] Each unit module of the device 20 can respectively execute the corresponding steps in the above method embodiment, so each unit module will not be described in detail here. Please refer to the description of the above corresponding steps for details.

[0069] The embodiment of the present invention further provides a model reasoning device based on artificial intelligence, the device comprising the model reasoning device based on artificial intelligence as described above, wherein the model reasoning device based on artificial intelligence can adopt Figure 3 The structure of the embodiment can be executed accordingly. Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects. For details, please refer to the relevant records in the above embodiments, which will not be repeated here.

[0070] The device includes: a mobile phone, a digital camera, a tablet computer, or other devices with a camera function, or a device with an image processing function, or a device with an image display function. The device may include components such as a memory, a processor, an input unit, a display unit, and a power supply.

[0071] Among them, the memory can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as an image playback function, etc.), etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory may also include a memory controller to provide the processor and the input unit with access to the memory.

[0072] The input unit can be used to receive input digital or character or image information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit of this embodiment includes not only a camera, but also a touch-sensitive surface (such as a touch display) and other input devices.

[0073] The display unit can be used to display information input by the user or information provided to the user and various graphical user interfaces of the device, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit may include a display panel, and optionally, the display panel may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface may cover the display panel, and when the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor to determine the type of touch event, and then the processor provides a corresponding visual output on the display panel according to the type of touch event.

[0074] The embodiment of the present invention further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory in the above embodiment; or a computer-readable storage medium that exists independently and is not installed in a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement Figure 1 The model reasoning method based on artificial intelligence is shown. The computer readable storage medium can be a read-only memory, a disk or an optical disk, etc.

[0075] The embodiment of the present invention also provides a computer program product, including a computer program / instruction, which is loaded and executed by a processor to implement Figure 1 An artificial intelligence-based model reasoning method is shown.

[0076] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, equipment embodiment and storage medium embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0077] Furthermore, in this document, the terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device that includes the element.

[0078] The above description shows and describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the invention, through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not depart from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A model reasoning method based on artificial intelligence, characterized in that: The method comprises: Write inference files including model loading methods and model inference methods; Upload the inference file and the model to be inferred to the inference platform built based on Python; The inference platform uses the model loading method to load the model to be inferred, and uses the model inference method to perform parsing processing according to the inference request sent by the client, so that the model to be inferred performs the inference operation and obtains the model inference result; further comprising: By creating an inference service launcher in the inference platform, using the inference service launcher to start the inference process according to the assembled start command, and using the inference process to execute the tasks in the model loading method and the model inference method; wherein, There are multiple inference processes; the inference process includes a parameter parsing engine, an inference file dynamic loading engine, a model file loading engine and a web service engine; based on the inference file dynamic loading engine, the importlib module and the sys module in Python are used to dynamically load the inference file of the corresponding path, and the inference file is dynamically loaded into the memory, so that the inference process executes the tasks in the model loading method and the model inference method; Based on the parameter parsing engine, the argparse module in Python is used to parse the command line parameters of the startup command to obtain the path of the inference file, the file path of the model to be inferred, the prediction endpoint URI, the running port, and the supported HTTP methods; Execute the inference file based on the model file loading engine to call the model loading method to load the model to be inferred into the memory or video memory; The web service is started based on the web service engine, the model inference method is executed according to the request parameters in the inference request through the web service, and the request parameters are sent to the input parameters of the model inference method.

2. A model reasoning device based on artificial intelligence, characterized in that: The device comprises: A writing unit, used for writing an inference file including a model loading method and a model inference method; A transmission unit, used for uploading the inference file and the model to be inferred to an inference platform built based on Python; An inference unit, used to load the model to be inferred by using the model loading method through the inference platform, and perform parsing processing according to the inference request sent by the client by using the model inference method, so that the model to be inferred performs the inference operation and obtains the model inference result; further comprising: By creating an inference service launcher in the inference platform, using the inference service launcher to start the inference process according to the assembled start command, and using the inference process to execute the tasks in the model loading method and the model inference method; wherein, There are multiple inference processes; the inference process includes a parameter parsing engine, an inference file dynamic loading engine, a model file loading engine and a web service engine; based on the inference file dynamic loading engine, the importlib module and the sys module in Python are used to dynamically load the inference file of the corresponding path, and the inference file is dynamically loaded into the memory, so that the inference process executes the tasks in the model loading method and the model inference method; Based on the parameter parsing engine, the argparse module in Python is used to parse the command line parameters of the startup command to obtain the path of the inference file, the file path of the model to be inferred, the prediction endpoint URI, the running port, and the supported HTTP methods; Execute the inference file based on the model file loading engine to call the model loading method to load the model to be inferred into the memory or video memory; The web service is started based on the web service engine, the model inference method is executed according to the request parameters in the inference request through the web service, and the request parameters are sent to the input parameters of the model inference method.

3. A model reasoning device based on artificial intelligence, characterized in that: It includes a processor, a memory and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of the model reasoning method based on artificial intelligence as claimed in claim 1.

4. A computer program product, comprising a computer program / instruction, wherein the computer program / instruction is executed by a processor to implement the steps of an artificial intelligence-based model reasoning method as described in claim 1.

Images