Inference calculation method and related equipment

Abstract

The invention discloses a reasoning calculation method and related equipment, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining first to-be-reasoned data in response to a first reasoning calculation instruction, and carrying out the reasoning calculation of the first to-be-reasoned data based on a locally deployed preset reasoning calculation model, and obtaining a first reasoning calculation result, the preset reasoning calculation model is constructed based on a basic reasoning model framework and a first reasoning fragment, and the first reasoning fragment is determined based on a first reasoning type corresponding to the first reasoning calculation instruction. According to different types of reasoning calculation instructions, in the process of reasoning by using different models, only the reasoning fragments of the models need to be changed, the whole model does not need to be changed, and the model deployment efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a reasoning calculation method and related equipment. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, the application of AI large models has become a key driving force in many industries. In actual scenarios, different large models need to be called for different types of reasoning tasks. However, large models often involve multiple dependent libraries and environment configurations. In the process of changing the model, a large amount of data needs to be loaded for deploying the model, resulting in low efficiency of deploying the model.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a reasoning calculation method and related equipment, aiming to solve the technical problem of how to improve the efficiency of deploying the model in the process of changing the model.

[0005] To achieve the above purpose, the present application provides a reasoning calculation method, which comprises:

[0006] In response to a first reasoning calculation instruction, first reasoning data to be calculated is obtained;

[0007] A preset reasoning calculation model based on local deployment is used to perform reasoning calculation on the first reasoning data to be calculated, and a first reasoning calculation result is obtained, wherein the preset reasoning calculation model is constructed based on a basic reasoning model framework and a first reasoning segment, and the first reasoning segment is determined based on a first reasoning type corresponding to the first reasoning calculation instruction.

[0008] In an embodiment, before the step of using the preset reasoning calculation model based on local deployment, the method further comprises:

[0009] A first reasoning type corresponding to the first reasoning calculation instruction is determined, and a first reasoning segment in a current locally deployed reasoning calculation model is determined.

[0010] A second reasoning segment corresponding to the first reasoning type in a preset reasoning segment database is determined.

[0011] It is determined whether the first reasoning segment and the second reasoning segment are the same.

[0012] If they are the same, the current locally deployed reasoning calculation model is set as the preset reasoning calculation model.

[0013] If not, the first inference segment is replaced by the second inference segment to obtain a preset inference computing model.

[0014] In an embodiment, after the step of obtaining the preset inference computing model, the method further comprises:

[0015] determining an inference environment corresponding to the first inference computing instruction;

[0016] judging whether a format of the preset inference computing model corresponds to the inference environment based on the inference environment;

[0017] If not, performing a format conversion operation on the preset inference computing model based on the inference environment and a preset ONNX model exchange format, and deploying the preset inference computing model after the format conversion.

[0018] In an embodiment, before the step of determining the second inference segment corresponding to the first inference type in the preset inference segment database, the method further comprises:

[0019] determining a plurality of groups of preset inference types;

[0020] constructing an inference segment corresponding to each group of preset inference types based on the plurality of groups of preset inference types to obtain a plurality of groups of inference segments;

[0021] constructing a preset inference segment database based on the plurality of groups of inference segments.

[0022] In an embodiment, the step of replacing the first inference segment with the second inference segment to obtain a preset inference computing model further comprises:

[0023] resetting an inference segment loading function corresponding to a currently localized deployed inference computing model based on the second inference segment;

[0024] loading the second inference segment from the preset inference segment database based on the reset inference segment loading function;

[0025] replacing the first inference segment with the second inference segment to obtain a preset inference computing model.

[0026] In an embodiment, the step of performing inference calculation on the first inference data based on the preset inference computing model deployed locally to obtain a first inference calculation result further comprises:

[0027] determining a second inference result in a historical inference database, second inference data corresponding to the second inference result, and a second inference type corresponding to the second inference result;

[0028] a first similarity value between the second inference data and the first inference data is calculated, and a second similarity value between the second inference type and the first inference type is calculated;

[0029] It is judged whether the first similarity value is greater than or equal to a preset inference calculation instruction similarity threshold value, to obtain a first judgment result, and whether the second similarity value is greater than or equal to a preset inference type similarity threshold value, to obtain a second judgment result;

[0030] If the first judgment result and the second judgment result are both yes, the second inference result is set as the first inference result.

[0031] If there is no in the first judgment result and the second judgment result, a preset inference calculation model based on local deployment is used to perform inference calculation on the first inference data, to obtain a first inference calculation result.

[0032] In an embodiment, before the step of using the preset inference calculation model based on local deployment, the method further comprises:

[0033] A current inference calculation model corresponding to the first inference type and sample data are obtained, and an inference calculation result corresponding to the sample data is a third inference calculation result.

[0034] The sample data is subjected to inference calculation using the current inference calculation model, to obtain a fourth inference calculation result.

[0035] It is judged whether the third inference calculation result and the fourth inference calculation result are consistent.

[0036] If not, the parameters of the current inference calculation model are adjusted, and the step of using the current inference calculation model to perform inference calculation on the sample data to obtain the fourth inference calculation result is returned based on the current inference calculation model after the adjustment, until the third inference calculation result and the fourth inference calculation result are consistent, to obtain the preset inference calculation model.

[0037] In addition, to achieve the above object, the application further provides an inference calculation device, which comprises:

[0038] An acquisition module is configured to acquire first inference data in response to a first inference calculation instruction.

[0039] An inference calculation module is configured to perform inference calculation on the first inference data based on a preset inference calculation model based on local deployment, to obtain a first inference calculation result, wherein the preset inference calculation model is constructed based on a basic inference model framework and a first inference segment, and the first inference segment is determined based on a first inference type corresponding to the first inference calculation instruction.

[0040] In an embodiment, the inference computing device further comprises a determining module, the determining module comprising:

[0041] a first determining unit configured to determine a first inference type corresponding to the first inference computing instruction, and determine a first inference segment in a currently localized deployed inference computing model;

[0042] a second determining unit configured to determine a second inference segment corresponding to the first inference type in a preset inference segment database;

[0043] a first judging unit configured to judge whether the first inference segment and the second inference segment are the same;

[0044] a first setting unit configured to, if the first inference segment and the second inference segment are the same, set the currently localized deployed inference computing model as a preset inference computing model;

[0045] a first replacing unit configured to, if the first inference segment and the second inference segment are not the same, replace the first inference segment with the second inference segment to obtain the preset inference computing model.

[0046] In an embodiment, the inference computing device further comprises a format conversion module, the format conversion module comprising:

[0047] a third determining unit configured to determine an inference environment corresponding to the first inference computing instruction;

[0048] a second judging unit configured to judge, based on the inference environment, whether a format of a preset inference computing model corresponds to the inference environment;

[0049] a format conversion unit configured to, if not, perform a format conversion operation on the preset inference computing model based on the inference environment and a preset ONNX model exchange format, and deploy the preset inference computing model after format conversion.

[0050] In an embodiment, the inference computing device further comprises a constructing module, the constructing module comprising:

[0051] a fourth determining unit configured to determine a plurality of groups of preset inference types;

[0052] a first constructing unit configured to construct, based on the plurality of groups of preset inference types, an inference segment corresponding to each group of preset inference types, to obtain a plurality of groups of inference segments;

[0053] a second constructing unit configured to construct, based on the plurality of groups of inference segments, a preset inference segment database.

[0054] In an embodiment, the determining module further comprises:

[0055] A resetting unit is configured to reset, based on the second inference fragment, an inference fragment loading function corresponding to the currently localized deployed inference calculation model;

[0056] A loading unit is configured to load, based on the reset inference fragment loading function, the second inference fragment from a preset inference fragment database;

[0057] A second replacing unit is configured to replace the first inference fragment with the second inference fragment to obtain a preset inference calculation model.

[0058] In an embodiment, the inference calculation module further comprises:

[0059] A fifth determining unit is configured to determine a second inference result in a historical inference database, second to-be-inferred data corresponding to the second inference result, and a second inference type corresponding to the second inference result;

[0060] A calculation unit is configured to calculate a first similarity value between the second to-be-inferred data and the first to-be-inferred data, and calculate a second similarity value between the second inference type and the first inference type;

[0061] A third determining unit is configured to determine whether the first similarity value is greater than or equal to a preset inference calculation instruction similarity threshold to obtain a first determination result, and determine whether the second similarity value is greater than or equal to a preset inference type similarity threshold to obtain a second determination result;

[0062] A second setting unit is configured to set the second inference result as the first inference result if the first determination result and the second determination result are both yes.

[0063] A first inference unit is configured to perform inference calculation on the first to-be-inferred data based on a preset inference calculation model deployed locally to obtain a first inference calculation result if there is no in the first determination result and the second determination result.

[0064] In an embodiment, the inference calculation apparatus further comprises a model training module, and the model training module comprises:

[0065] An acquisition unit is configured to acquire a current inference calculation model corresponding to a first inference type and sample data, and an inference calculation result corresponding to the sample data is a third inference calculation result;

[0066] A second inference unit is configured to perform inference calculation on the sample data using the current inference calculation model to obtain a fourth inference calculation result;

[0067] A third determining unit is configured to determine whether the third inference calculation result and the fourth inference calculation result are consistent.

[0068] a model training unit configured to, if the third inference calculation result and the fourth inference calculation result are inconsistent, adjust parameters of the current inference calculation model, return to a step of performing inference calculation on the sample data using the current inference calculation model based on the current inference calculation model after the parameters are adjusted to obtain a fourth inference calculation result, and repeat the step until the third inference calculation result and the fourth inference calculation result are consistent, thereby obtaining a preset inference calculation model.

[0069] In addition, to achieve the above object, the present application further provides an inference calculation device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the inference calculation method as described above.

[0070] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the inference calculation method as described above.

[0071] In addition, to achieve the above object, the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the inference calculation method as described above.

[0072] The one or more technical solutions provided by the present application have at least the following technical effects:

[0073] The inference calculation method and related device provided by the present application relate to the field of artificial intelligence technology. In related technologies, in actual scenarios, different large models need to be called for different types of inference tasks. However, the large models often involve multiple dependent libraries and environment configurations. In the process of changing the model, a large amount of data needs to be loaded for deploying the model, which leads to low efficiency of deploying the model. In the present application, first, in response to a first inference calculation instruction, first inference data is obtained. Then, based on a preset inference calculation model deployed locally, inference calculation is performed on the first inference data to obtain a first inference calculation result. The preset inference calculation model is constructed based on a basic inference model framework and a first inference segment. The first inference segment is determined based on a first inference type corresponding to the first inference calculation instruction. In the present application, according to different types of inference calculation instructions, only the inference segment of the model needs to be changed during the inference process using different models, and the entire model does not need to be changed, thereby improving the efficiency of deploying the model. BRIEF DESCRIPTION OF DRAWINGS

[0074] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0075] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings required by the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.

[0076] Figure 1 The flowchart provided by the reasoning calculation method embodiment one of the present application;

[0077] Figure 2 The flowchart provided by the reasoning calculation method embodiment two of the present application;

[0078] Figure 3 The flowchart provided by the reasoning calculation method embodiment three of the present application;

[0079] Figure 4 The module structure diagram of the reasoning calculation device of the present application embodiment;

[0080] Figure 5 The device structure diagram of the hardware running environment involved in the reasoning calculation method in the present application embodiment.

[0081] The purpose of the present application, the function characteristics and the advantages will be further explained in combination with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0082] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0083] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings in the specification and specific embodiments.

[0084] The main solution of the present application embodiment is:

[0085] In the present embodiment, in order to facilitate description, the following is described with reasoning calculation device as the execution subject.

[0086] Because of the prior art: in actual scene, for different categories of reasoning tasks, different large models need to be called, however, large models often involve multiple dependent libraries and environment configurations, in the process of changing the model, the deployed model needs to load a large amount of data, resulting in low efficiency of deploying the model.

[0087] The application provides a solution, which comprises the following steps: first, obtaining first inference data in response to a first inference calculation instruction; and then, performing inference calculation on the first inference data based on a preset inference calculation model deployed locally to obtain a first inference calculation result, wherein the preset inference calculation model is constructed based on a basic inference model framework and a first inference segment, and the first inference segment is determined based on a first inference type corresponding to the first inference calculation instruction. According to different types of inference calculation instructions, only the inference segment of the model needs to be changed in the process of using different models for inference, and the entire model does not need to be changed, thereby improving the efficiency of deploying the model.

[0088] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone or the like, or an electronic device or an inference calculation device capable of realizing the above functions. The embodiment and the following embodiments will be described below by taking an inference calculation device as an example.

[0089] Based on this, the embodiment of the application provides an inference calculation method, which is described below with reference to Figure 1 , Figure 1 FIG. 1 is a flowchart of a first embodiment of the inference calculation method of the application.

[0090] In the embodiment, the inference calculation method comprises steps S10-S20:

[0091] Step S10: obtaining first inference data in response to a first inference calculation instruction.

[0092] The first inference calculation instruction is an instruction signal, which is usually issued in a certain system (such as a computer system, an artificial intelligence model, an automatic control system or the like). Its function is to tell the system to start performing a certain inference calculation task.

[0093] The first inference data is data that needs to be subjected to inference calculation. Here, “first” may mean that this is the first batch of data that needs to be processed in the inference task, or is a number used to distinguish other data.

[0094] In the embodiment, when the system receives an explicit instruction (i.e., the “first inference calculation instruction”) indicating that the inference calculation task needs to be started, the system will automatically obtain the data (i.e., the “first inference data”) related to the task so as to subsequently perform inference calculation.

[0095] In the application, the specific application scenario can be:

[0096] AI model inference: In machine learning or deep learning, a model needs input data to perform inference (e.g., an image recognition model needs input image data). When the system receives an inference instruction, it goes to obtain the corresponding image data.

[0097] In this application, the specific application scenario can also be:

[0098] Automated control system: In industrial automation, when the system receives a control instruction (such as "start detecting products on the production line"), it will obtain sensor data on the production line for subsequent analysis and control.

[0099] Step S20, based on the preset inference calculation model of the local deployment, the first inference calculation is performed on the first inference data, and the first inference calculation result is obtained, wherein the preset inference calculation model is constructed based on the basic inference model framework and the first inference segment, and the first inference segment is determined based on the first inference type corresponding to the first inference calculation instruction.

[0100] Local deployment refers to deploying models and related computing resources on local servers or devices, rather than relying on cloud or other remote resources. This approach can improve data processing speed and security while reducing dependence on the network.

[0101] The preset inference calculation model is a model that has been pre-constructed specifically for inference calculation tasks. It may be trained and optimized to handle specific types of data and tasks.

[0102] Inference calculation refers to the process of using a model to analyze and process input data to draw conclusions or results. This can be the prediction of a machine learning model, the deduction of a logical inference system, etc.

[0103] The basic inference model framework is a general model framework that provides the basic structure and algorithms for inference calculation. It may be a certain deep learning framework (such as TensorFlow, PyTorch, etc.) or a logical inference system's basic architecture.

[0104] The first inference segment is an extension or customization of the basic model framework, designed to adapt to specific inference tasks. It may contain specific algorithms, parameters, or logic to handle specific types of data or tasks.

[0105] In this application, the preset inference calculation model is formed by combining the basic model framework with the first inference segment. This combination allows the model to have both generality and specificity.

[0106] The first inference type refers to the specific type of inference task, such as image recognition, language translation, logical reasoning, etc. Different inference types require different processing methods and model structures.

[0107] In this embodiment, the system constructs a preset inference computing model based on the basic inference model framework and specific inference fragments. The inference fragments are selected according to the type of the inference task (determined by the inference instruction). The system obtains the first inference data (which may have been triggered by a previous instruction). Then, the system uses the preset inference computing model to perform inference calculation on the first inference data, and obtains the first inference calculation result after the inference calculation is completed.

[0108] It can be understood that in this way, the system can flexibly adjust the model structure according to different task requirements, while improving efficiency and security through local deployment.

[0109] Specifically, the preset inference computing model based on local deployment, in the step of performing inference calculation on the first inference data to obtain the first inference calculation result, further includes steps S21-S25:

[0110] Step S21, determining the second inference result in the historical inference database, the second inference data corresponding to the second inference result, and the second inference type corresponding to the second inference result;

[0111] The historical inference database is a database that stores past inference calculation results and their related information. It includes: inference results (conclusions or outputs obtained by previous inference calculation), inference data (input data used for previous inference calculation), inference types (task types that previous inference calculation is aimed at, such as image recognition, language translation, logical reasoning, etc.), and other metadata (may also include information such as inference timestamp, model version used, inference accuracy, etc.).

[0112] The second inference result is a specific inference result found in the historical inference database. Here, "second" is to distinguish from other inference results, which may be because it is the target of the current query or analysis.

[0113] The second inference data is the input data used to generate the "second inference result". In other words, it is the data processed by the inference model when obtaining the "second inference result".

[0114] The second inference type is the inference task type related to the "second inference result". For example, if the inference result is the result of image recognition, the inference type may be "image recognition"; if it is the result of language translation, the inference type may be "language translation".

[0115] Step S22, a first similarity value between the second to-be-reasoned data and the first to-be-reasoned data is calculated, and a second similarity value between the second reasoning type and the first reasoning type is calculated.

[0116] The first similarity value is a numerical value representing the similarity between the second to-be-reasoned data and the first to-be-reasoned data.

[0117] The second similarity value is also a numerical value representing the similarity between the second reasoning type and the first reasoning type.

[0118] Step S23, it is judged whether the first similarity value is greater than or equal to a preset reasoning calculation instruction similarity threshold value, to obtain a first judgment result, and it is judged whether the second similarity value is greater than or equal to a preset reasoning type similarity threshold value, to obtain a second judgment result.

[0119] The preset reasoning calculation instruction similarity threshold value is a preset threshold value for judging whether the similarity of two data sets is high enough. This threshold value can be set according to specific application scenarios and requirements.

[0120] The preset reasoning type similarity threshold value is a preset threshold value for judging whether the similarity of two reasoning types is high enough. This threshold value can also be set according to specific application scenarios and requirements.

[0121] Step S24, if the first judgment result and the second judgment result are both yes, the second reasoning result is set as the first reasoning result.

[0122] Step S25, if there is no in the first judgment result and the second judgment result, a first to-be-reasoned data is calculated based on a locally deployed preset reasoning calculation model, to obtain a first reasoning calculation result.

[0123] The present application proposes an inference calculation method and related equipment, which relate to the field of artificial intelligence technology. Compared with the related technologies, in actual scenarios, different large models need to be called for different categories of reasoning tasks. However, large models often involve multiple dependent libraries and environment configurations. In the process of changing the model, the deployment model needs to load a large amount of data, resulting in low efficiency of the deployment model. In the present application, first, in response to a first inference calculation instruction, the first data to be inferred is obtained, and then, based on the locally deployed preset inference calculation model, the first data to be inferred is inferred and calculated to obtain a first inference calculation result, wherein the preset inference calculation model is constructed based on the basic inference model framework and the first inference fragment, and the first inference fragment is determined based on the first inference type corresponding to the first inference calculation instruction. According to different types of inference calculation instructions, the present application only needs to change the inference fragment of the model in the process of using different models for reasoning, without changing the entire model, thereby improving the efficiency of the deployed model.

[0124] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 2 Before step S20, the inference calculation method further includes steps S01 to S05:

[0125] Step S01: determining a first reasoning type corresponding to the first reasoning computing instruction, and determining a first reasoning segment in a currently locally deployed reasoning computing model;

[0126] Step S02: determining a second reasoning segment corresponding to the first reasoning type in a preset reasoning segment database;

[0127] Specifically, before the step of determining the second reasoning segment corresponding to the first reasoning type in the preset reasoning segment database, the method further includes steps A01 to A03:

[0128] Step A01, determining multiple groups of preset inference types;

[0129] Step A02: Based on multiple groups of preset reasoning types, construct reasoning snippets corresponding to each group of preset reasoning types to obtain multiple groups of reasoning snippets;

[0130] Step A03: constructing a preset reasoning segment database based on the multiple groups of reasoning segments.

[0131] Predefined reasoning types are predefined reasoning task types or categories of reasoning logic. For example, they can be: image recognition (used to identify objects in images), natural language processing (used for text analysis or language translation), and speech recognition (used to convert speech signals into text).

[0132] Multiple preset inference types mean that there are multiple different inference types, each corresponding to a specific inference task or logic.

[0133] An inference fragment is a module or part of an inference model that is used to complete a specific inference task. For example, in a deep learning model, an inference fragment can be a neural network layer or a set of layers.

[0134] In this embodiment, according to each preset inference type, the corresponding inference fragment is designed and implemented.

[0135] For example:

[0136] For image recognition, the inference fragment can be certain layers of a convolutional neural network (CNN). For natural language processing, the inference fragment can be part of a Transformer architecture.

[0137] The preset inference fragment database is a collection of all inference fragments, which facilitates subsequent calling and updating.

[0138] Specifically, the step of replacing the first inference fragment with the second inference fragment to obtain a preset inference computing model further includes steps A04-A06:

[0139] Step A04, based on the second inference fragment, reset the inference fragment loading function corresponding to the currently localized deployed inference computing model;

[0140] Step A05, based on the reset push fragment loading function, load the second inference fragment from the preset inference fragment database;

[0141] Step A06, replace the first inference fragment with the second inference fragment to obtain a preset inference computing model.

[0142] The inference fragment loading function is a program or function that loads the inference fragment from storage into the inference computing model.

[0143] In this embodiment, according to the characteristics (such as format, size, etc.) of the second inference fragment, the parameters or logic of the loading function are adjusted to ensure that the second inference fragment can be correctly loaded, and then the second inference fragment is found and loaded from the preset inference fragment database using the reset loading function. The loading function will correctly load the second inference fragment into memory according to its characteristics (such as storage location, format, etc.) for subsequent use.

[0144] Further, the first inference fragment is removed from the model, and the second inference fragment is inserted into the corresponding position in the model. After the replacement is completed, the inference computing model is updated to a new model containing the second inference fragment, thereby having new inference capabilities or optimized performance.

[0145] Step S03, determining whether the first inference fragment and the second inference fragment are the same;

[0146] Step S04, if the same, setting the current localized deployed inference computing model as the preset inference computing model;

[0147] Step S05, if not the same, replacing the first inference fragment with the second inference fragment to obtain the preset inference computing model.

[0148] In this embodiment, it is determined whether the first inference fragment and the second inference fragment are completely consistent through a certain comparison mechanism (such as code comparison, logical structure comparison, function verification, etc.). The comparison content includes: code implementation (checking whether the codes of the two are completely the same), logical function (verifying whether the two perform the same function), input and output (comparing whether the output results of the two to the same input are consistent).

[0149] It can be understood that, because the first inference fragment and the second inference fragment are the same, it means that the current model is the latest or meets the requirements.

[0150] Specifically, after the step of obtaining the preset inference computing model, steps S06-S08 are further included:

[0151] Step S06, determining the inference environment corresponding to the first inference computing instruction;

[0152] Step S07, based on the inference environment, determining whether the format of the preset inference computing model corresponds to the inference environment;

[0153] Step S08, if not, performing a format conversion operation on the preset inference computing model based on the inference environment and the preset ONNX model exchange format, and deploying the preset inference computing model after format conversion.

[0154] The inference environment refers to the running environment for executing inference tasks, including hardware (such as CPU, GPU, FPGA, etc.), operating system, framework (such as TensorFlow, PyTorch, ONNX Runtime, etc.), and related libraries and tools.

[0155] In this embodiment, according to the content or context of the first inference calculation instruction, it is identified that the instruction needs to be executed in which specific inference environment. For example: if the instruction is issued through a specific deep learning framework (such as TensorFlow), the inference environment may be a hardware and software environment that supports TensorFlow. If the instruction needs to be executed on an edge device (such as an embedded system), the inference environment may be a lightweight operating system and limited hardware resources.

[0156] The model format refers to the storage and representation form of the model, for example: TensorFlow's.pb or.tflite format, PyTorch's.pt or.pth format, ONNX format (an open model exchange format).

[0157] It can be understood that if the inference environment is TensorFlow, the model format must be a format supported by TensorFlow. If the inference environment is ONNX Runtime, the model format must be ONNX format.

[0158] ONNX (Open Neural Network Exchange) is an open model exchange format that allows model conversion and sharing between different frameworks. For example, TensorFlow or PyTorch models can be converted to ONNX format and then used in other ONNX-supported environments.

[0159] In this embodiment, in this way, it can be ensured that the model can run efficiently in different environments while maintaining the integrity and accuracy of the model.

[0160] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as the above embodiments one and two can refer to the above introduction, and the following will not be repeated. On this basis, please refer to Figure 3 , before step S20, the inference calculation method further includes steps B01-B04:

[0161] Step B01, obtaining a current inference calculation model corresponding to a first inference type and sample data, the inference calculation result corresponding to the sample data being a third inference calculation result;

[0162] Step B02, using the current inference calculation model to perform inference calculation on the sample data to obtain a fourth inference calculation result;

[0163] Step B03, determining whether the third inference calculation result and the fourth inference calculation result are consistent;

[0164] Step B04, if there is inconsistency, adjust the parameters of the current inference calculation model, and based on the current inference calculation model after adjusting the parameters, return to the step of using the current inference calculation model to perform inference calculation on the sample data to obtain the fourth inference calculation result, until the third inference calculation result is consistent with the fourth inference calculation result, and obtain the preset inference calculation model.

[0165] The current reasoning computing model is a model used to perform the first reasoning type task, which may be a deep learning model, a traditional machine learning model, or another type of reasoning model.

[0166] Sample data is a dataset used to test and validate a model, and usually contains input data and corresponding expected outputs (i.e., labels or results).

[0167] The third inference calculation result is the correct or expected inference result corresponding to the sample data, which is usually pre-defined during the model training process or obtained through other reliable methods.

[0168] The fourth inference calculation result is the actual inference result of the model on the sample data, that is, the prediction or classification result obtained by the model based on the current parameters and logic.

[0169] Furthermore, the actual output of the model (the fourth inference calculation result) is compared with the expected correct result (the third inference calculation result). Check whether the two are completely consistent or within an acceptable error range. If they are consistent, it means that the model performs correctly on the current sample; if they are inconsistent, it means that the model needs to be adjusted. Adjust the model parameters based on the type of model and the reason for the inconsistency. For example: deep learning models (parameters such as neural network weights, biases, and learning rates can be adjusted) and traditional machine learning models (hyperparameters of the model can be adjusted, such as the depth of the decision tree and the kernel function of the support vector machine).

[0170] Furthermore, the model with the adjusted parameters is used to re-infer the sample data to obtain a new fourth inference calculation result. Steps B02 and B03 are repeated until the third inference calculation result is consistent with the fourth inference calculation result.

[0171] When the output of the model is consistent with the expected result, it means that the model has been adjusted to a state that meets the requirements. At this time, the model is called a "preset inference calculation model."

[0172] It is understandable that in this way, the accuracy and reliability of the model on a specific task can be ensured.

[0173] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the reasoning and calculation method of the present application. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.

[0174] The application also provides an inference computing device, which comprises: Figure 4

[0175] An acquisition module 10 is configured to acquire first inference data in response to a first inference computing instruction;

[0176] An inference computing module 20 is configured to perform inference computation on the first inference data based on a preset inference computing model deployed locally to obtain a first inference computing result, wherein the preset inference computing model is constructed based on a basic inference model framework and a first inference segment, and the first inference segment is determined based on a first inference type corresponding to the first inference computing instruction.

[0177] In an embodiment, the inference computing device further comprises a determination module, which comprises:

[0178] A first determination unit is configured to determine the first inference type corresponding to the first inference computing instruction and determine a first inference segment in an inference computing model deployed locally at present;

[0179] A second determination unit is configured to determine a second inference segment corresponding to the first inference type in a preset inference segment database;

[0180] A first judgment unit is configured to judge whether the first inference segment and the second inference segment are the same;

[0181] A first setting unit is configured to set the inference computing model deployed locally at present as the preset inference computing model if they are the same;

[0182] A first replacement unit is configured to replace the first inference segment with the second inference segment to obtain the preset inference computing model if they are not the same.

[0183] In an embodiment, the inference computing device further comprises a format conversion module, which comprises:

[0184] A third determination unit is configured to determine an inference environment corresponding to the first inference computing instruction;

[0185] A second judgment unit is configured to judge whether a format of the preset inference computing model corresponds to the inference environment based on the inference environment;

[0186] A format conversion unit is configured to perform a format conversion operation on the preset inference computing model based on the inference environment and a preset ONNX model exchange format and deploy the preset inference computing model after format conversion if the format of the preset inference computing model does not correspond to the inference environment.

[0187] ​In an embodiment, the inference computing apparatus further comprises a constructing module, the constructing module comprising:

[0188] a fourth determining unit configured to determine a plurality of groups of preset inference types;

[0189] a first constructing unit configured to construct, based on the plurality of groups of preset inference types, an inference segment corresponding to each group of preset inference types, to obtain a plurality of groups of inference segments;

[0190] a second constructing unit configured to construct, based on the plurality of groups of inference segments, a preset inference segment database.

[0191] In an embodiment, the determining module further comprises:

[0192] a resetting unit configured to reset, based on the second inference segment, an inference segment loading function corresponding to the currently localized deployed inference computing model;

[0193] a loading unit configured to load, based on the reset inference segment loading function, the second inference segment from the preset inference segment database;

[0194] a second replacing unit configured to replace the first inference segment with the second inference segment to obtain a preset inference computing model.

[0195] In an embodiment, the inference computing module further comprises:

[0196] a fifth determining unit configured to determine a second inference result in a historical inference database, second to-be-inferred data corresponding to the second inference result, and a second inference type corresponding to the second inference result;

[0197] a computing unit configured to calculate a first similarity value of the second to-be-inferred data and the first to-be-inferred data, and calculate a second similarity value of the second inference type and the first inference type;

[0198] a third judging unit configured to judge whether the first similarity value is greater than or equal to a preset inference computing instruction similarity threshold to obtain a first judgment result, and judge whether the second similarity value is greater than or equal to a preset inference type similarity threshold to obtain a second judgment result;

[0199] a second setting unit configured to set the second inference result as the first inference result if the first judgment result and the second judgment result are both yes;

[0200] a first inference unit configured to perform inference computation on the first to-be-inferred data based on the preset inference computing model deployed locally to obtain a first inference computation result if there is no in the first judgment result and the second judgment result.

[0201] In an embodiment, the inference computing apparatus further comprises a model training module, and the model training module comprises:

[0202] an acquisition unit configured to acquire a current inference computing model corresponding to the first inference type and sample data, wherein the inference computing result corresponding to the sample data is a third inference computing result;

[0203] a second inference unit configured to perform inference computation on the sample data using the current inference computing model to obtain a fourth inference computing result;

[0204] a third judgment unit configured to judge whether the third inference computing result and the fourth inference computing result are consistent;

[0205] a model training unit configured to, if the third inference computing result and the fourth inference computing result are not consistent, adjust the parameters of the current inference computing model, return to the step of performing inference computation on the sample data using the current inference computing model to obtain the fourth inference computing result based on the current inference computing model after the parameters are adjusted, until the third inference computing result and the fourth inference computing result are consistent, and obtain a preset inference computing model.

[0206] The inference computing apparatus provided in the present application adopts the inference computation method in the above embodiments, and can solve the technical problem of inference computation. Compared with the prior art, the inference computing apparatus provided in the present application has the same beneficial effects as the inference computation method provided in the above embodiments, and other technical features in the inference computing apparatus are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0207] The present application provides an inference computing device, which comprises at least one processor and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the inference computation method in the above embodiment one.

[0208] Reference will be made to the following description Figure 5 which shows a structural schematic diagram of an inference computing device suitable for implementing the embodiments of the present application. The inference computing device in the embodiments of the present application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 5The illustrated inference computing device is merely one example and should not be taken as limiting the scope of functionality or use of embodiments of the application.

[0209] As shown in Figure 5 The inference computing device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 1002 or programs loaded into a random access memory (RAM) 1004 from a storage device 1003. Various programs and data required for operation of the inference computing device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the inference computing device to communicate wirelessly or wired with other devices to exchange data. While the inference computing device is shown with various systems, it should be understood that not all of the illustrated systems are required to implement or be present. More or fewer systems can alternatively be implemented or present.

[0210] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0211] The inference computing device provided by the present application adopts the inference computing method in the above-mentioned embodiments, and can solve the technical problems. Compared with the prior art, the inference computing device provided by the present application has the same beneficial effects as the inference computing method provided by the above-mentioned embodiments, and other technical features in the inference computing device are the same as the features disclosed in the above-mentioned embodiments, which will not be repeated here.

[0212] It should be understood that portions of the application disclosed can be implemented in hardware, software, firmware, or combinations thereof. In the description of the embodiments above, specific features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0213] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any modifications or equivalents of the application should be construed as falling within the scope of the application. The scope of the application should be determined by the appended claims.

[0214] The application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the inference computing method in the above-described embodiments.

[0215] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any suitable combination thereof.

[0216] The above-described computer readable storage medium can be contained in the inference computing device; or can exist separately and not be assembled into the inference computing device.

[0217] The above-described computer readable storage medium carries one or more programs, which, when executed by the inference computing device, cause the inference computing device to:

[0218] In response to the first inference calculation instruction, the first to-be-inferred data is acquired;

[0219] Based on the preset inference calculation model of the localized deployment, the first to-be-inferred data is subjected to inference calculation to obtain a first inference calculation result, wherein the preset inference calculation model is constructed based on a basic inference model framework and a first inference segment, and the first inference segment is determined based on a first inference type corresponding to the first inference calculation instruction.

[0220] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0221] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified functions. It should also be noted in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks depicted in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and / or flow diagrams and combinations of blocks in the block diagrams and / or flow diagrams can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0222] The modules involved in the embodiments of the present application can be implemented in a software manner or in a hardware manner. In some cases, the name of the module does not constitute a limitation on the module itself.

[0223] The readable storage medium provided by the application is a computer readable storage medium, which stores computer readable program instructions (i.e. computer programs) for executing the above inference calculation method, and can solve the technical problem of inference calculation. Compared with the prior art, the beneficial effects of the computer readable storage medium provided by the application are the same as those of the inference calculation method provided by the above-mentioned embodiments, and are not described here.

[0224] The application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the inference calculation method as described above.

[0225] The computer program product provided by the application can solve the technical problem of inference calculation. Compared with the prior art, the beneficial effects of the computer program product provided by the application are the same as those of the inference calculation method provided by the above-mentioned embodiments, and are not described here.

[0226] The above-mentioned is only part of the embodiments of the application, and does not limit the protection scope of the application, and any equivalent structural transformation, direct / indirect application in other related technical fields made by using the content of the application specification and drawings within the technical concept of the application are included in the patent protection scope of the application.

Claims

1. A method of inference calculation, characterized in that: The inference calculation method includes: In response to the first inference calculation instruction, obtaining first data to be inferred; Based on a locally deployed preset reasoning calculation model, reasoning calculation is performed on the first data to be inferred to obtain a first reasoning calculation result, wherein the preset reasoning calculation model is constructed based on a basic reasoning model framework and a first reasoning fragment, and the first reasoning fragment is determined based on the first reasoning type corresponding to the first reasoning calculation instruction.

2. The inference calculation method according to claim 1, wherein: Before the step of presetting the inference computing model based on local deployment, the method further includes: Determining a first reasoning type corresponding to the first reasoning computing instruction, and determining a first reasoning segment in the currently locally deployed reasoning computing model; Determine a second reasoning segment corresponding to the first reasoning type in a preset reasoning segment database; Determining whether the first reasoning segment is identical to the second reasoning segment; If they are the same, setting the current locally deployed inference computing model as the default inference computing model; If they are not the same, the first reasoning segment is replaced by the second reasoning segment to obtain a preset reasoning calculation model.

3. The inference calculation method according to claim 2, wherein: After the step of obtaining the preset inference calculation model, the method further includes: Determining an inference environment corresponding to the first inference computation instruction; Based on the reasoning environment, determining whether the format of the preset reasoning calculation model corresponds to the reasoning environment; If not, based on the exchange format between the inference environment and the preset ONNX model, a format conversion operation is performed on the preset inference computing model, and the preset inference computing model after format conversion is deployed.

4. The inference calculation method according to claim 2, wherein: Before the step of determining the second reasoning segment corresponding to the first reasoning type in the preset reasoning segment database, the method further includes: determining multiple sets of preset reasoning types; Based on multiple groups of preset reasoning types, construct reasoning fragments corresponding to each group of preset reasoning types to obtain multiple groups of reasoning fragments; Based on the multiple groups of reasoning fragments, a preset reasoning fragment database is constructed.

5. The inference calculation method according to claim 2, wherein: The step of replacing the first reasoning segment with the second reasoning segment to obtain a preset reasoning calculation model further includes: Based on the second inference fragment, resetting the inference fragment loading function corresponding to the currently locally deployed inference computing model; Based on the reset reasoning segment loading function, load the second reasoning segment from a preset reasoning segment database; The first reasoning segment is replaced by the second reasoning segment to obtain a preset reasoning calculation model.

6. The inference calculation method according to claim 1, wherein: The step of performing inference calculation on the first data to be inferred based on the locally deployed preset inference calculation model to obtain a first inference calculation result further includes: Determining a second reasoning result in a historical reasoning database, second data to be inferred corresponding to the second reasoning result, and a second reasoning type corresponding to the second reasoning result; Calculating a first similarity value between the second data to be inferred and the first data to be inferred, and calculating a second similarity value between the second inference type and the first inference type; Determine whether the first similarity value is greater than or equal to a preset inference calculation instruction similarity threshold to obtain a first determination result, and determine whether the second similarity value is greater than or equal to a preset inference type similarity threshold to obtain a second determination result; If both the first judgment result and the second judgment result are yes, setting the second inference result as the first inference result; If there is a negative relationship between the first judgment result and the second judgment result, then based on the preset reasoning calculation model deployed locally, the first data to be inferred is inferred and a first reasoning calculation result is obtained.

7. The inference calculation method according to claim 1, wherein: Before the step of presetting the inference computing model based on local deployment, the method further includes: Obtaining a current reasoning calculation model and sample data corresponding to the first reasoning type, wherein the reasoning calculation result corresponding to the sample data is a third reasoning calculation result; Performing inference calculation on the sample data using the current inference calculation model to obtain a fourth inference calculation result; Determining whether the third reasoning calculation result is consistent with the fourth reasoning calculation result; If there is inconsistency, adjust the parameters of the current inference calculation model, and based on the current inference calculation model after adjusting the parameters, return to the step of using the current inference calculation model to perform inference calculation on the sample data to obtain the fourth inference calculation result, until the third inference calculation result is consistent with the fourth inference calculation result, and obtain the preset inference calculation model.

8. An inference computing device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the inference calculation method according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the inference calculation method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the inference computing method according to any one of claims 1 to 7 are implemented.

Images