Model reasoning method based on artificial intelligence platform and artificial intelligence platform

Abstract

The invention provides a model reasoning method based on an artificial intelligence platform and the artificial intelligence platform, and the method comprises the steps: obtaining a to-be-reasoned model and model data of the to-be-reasoned model, the model data comprising model source data, model demand data and a model data conversion rule; determining an open format converter of the to-be-reasoned model according to the model source data of the to-be-reasoned model, and converting the to-be-reasoned model into an open format model according to the open format converter corresponding to the to-be-reasoned model; packaging the open format model into an object model corresponding to a model format based on a conversion tool according to the model demand data, a model data conversion rule and the model format supported by a reasoning chip of the artificial intelligence platform; and under the condition that a reasoning request of the object model is received, reasoning the object model to obtain a reasoning result.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a model reasoning method based on an artificial intelligence platform and an artificial intelligence platform. Background Art

[0002] There are many frameworks for model training at present. Models trained by these architectures have their own unique formats. These formats are usually not universal, and the operators supported in them are not exactly the same. There is currently a universal onnx (OpenNeural Network Exchange, open format) model format that can achieve mutual conversion with these formats. However, some specific artificial intelligence (AI) platforms have their own dedicated inference chips. The platform framework needs to be designed to adapt to the dedicated inference chip. Only when the platform uses its own framework can it be combined with the inference chip to provide better performance and efficiency. If onnx is used, performance and efficiency may be reduced. Summary of the invention

[0003] The embodiments of the present application provide a model reasoning method and an artificial intelligence platform based on an artificial intelligence platform.

[0004] According to the first aspect of the present application, a model reasoning method based on an artificial intelligence platform is provided, the method comprising:

[0005] Acquire a model to be inferred and model data of the model to be inferred, wherein the model data includes model source data, model requirement data and model data conversion rules;

[0006] Determining an open format converter of the model to be inferred according to the model source data of the model to be inferred, and converting the model to be inferred into an open format model according to the open format converter corresponding to the model to be inferred;

[0007] According to the model requirement data, the model data conversion rules, and the model format supported by the inference chip of the artificial intelligence platform, based on a conversion tool, the open format model is encapsulated into an object model corresponding to the model format;

[0008] When receiving the inference request of the object model, the object model is inferred to obtain an inference result.

[0009] According to an embodiment of the present application, the model to be inferred includes a plurality of sub-models having logical relationships, and accordingly,

[0010] According to the open format converter corresponding to the model to be inferred, converting the model to be inferred into an open format model includes:

[0011] Each sub-model is converted into a sub-open format model according to the open format converter corresponding to the model to be inferred.

[0012] According to an embodiment of the present application, the model requirement data includes sub-model requirement data of each sub-model, and the model data conversion rule includes sub-model data conversion rules of each sub-model; accordingly,

[0013] According to the model requirement data, the model data conversion rules and the model format supported by the inference chip of the artificial intelligence platform, the open format model is encapsulated into an object model corresponding to the model format, including:

[0014] According to the sub-model requirement data corresponding to each sub-open format model, the sub-model data conversion rules and the model format supported by the inference chip of the artificial intelligence platform, each sub-open format model is encapsulated as a sub-object model corresponding to the model format.

[0015] According to an embodiment of the present application, when receiving an inference request for the object model, inferring the object model includes:

[0016] Reasoning is performed on multiple sub-object models based on the logical relationships between the multiple sub-models.

[0017] According to one embodiment of the present application, the model to be inferred is an optical character recognition model, and the multiple sub-models include a text box detection model, a detection box direction model, and a text recognition model.

[0018] According to one embodiment of the present application, the sub-model requirement data of the text box detection model includes size requirements, and the sub-model data conversion rules of the text box detection model include size conversion rules; the sub-model requirement data of the detection box orientation model includes box size requirements, and the sub-model data conversion rules of the detection box orientation model include box size conversion rules and a multi-box detection mechanism; the sub-model requirement data of the text recognition model includes recognition size requirements, and the sub-model data conversion rules of the text recognition model include recognition size conversion rules and a multi-box recognition mechanism.

[0019] According to an implementation manner of the present application, the inference result includes a text recognition position and a text recognition content; accordingly,

[0020] The step of, upon receiving the inference request of the object model, inferring the object model to obtain an inference result, comprises:

[0021] When receiving an inference request of the object model, obtaining the to-be-inferred graphic and text data corresponding to the inference request;

[0022] According to the size requirements and size conversion rules of the text box detection model, the size of the image and text data to be inferred is converted into a target size;

[0023] Input the size-converted image and text data to be inferred into the sub-object model corresponding to the text box detection model to obtain a text box detection result;

[0024] Converting the frame size of the text frame shown in the text frame detection result into a first target frame size according to the frame size requirement and frame size conversion rule of the detection frame direction model;

[0025] Inputting the text frame detection result after the frame size conversion into the sub-object model corresponding to the detection frame direction model, so that the sub-object model corresponding to the detection frame direction model performs frame direction detection on the text frame detection result after the frame size conversion according to the multi-frame detection mechanism to obtain a frame direction detection result;

[0026] According to the recognition size requirement and recognition size conversion rule of the text recognition model, the sizes of the multiple detection frames shown in the frame direction detection result are converted into the second target frame size;

[0027] The frame direction detection result after the frame size conversion is input into the sub-object model corresponding to the text recognition model, so that the sub-object model corresponding to the text recognition model performs text recognition on the frame direction detection result according to the multi-frame recognition mechanism to obtain the text recognition position and text recognition content.

[0028] According to an implementation of the present application, obtaining a model to be inferred includes:

[0029] The model to be inferred is obtained from the model training framework, which includes the open source software library TensorFlow, the deep learning framework PyTorch, the deep learning platform PaddlePaddle, and the open source deep learning framework MXNet.

[0030] According to an embodiment of the present application, the model to be inferred is obtained by training the model training framework using the following operations:

[0031] Obtain the data to be labeled, and label the data to be labeled to obtain training data;

[0032] Dividing the training data into a training set and a test set;

[0033] Use the training set to train multiple models to be tested;

[0034] The model to be inferred is determined from the multiple models to be tested using a test set.

[0035] According to a second aspect of the present application, an artificial intelligence platform is provided, the platform comprising:

[0036] An acquisition module, used to acquire the model to be inferred and model data of the model to be inferred, wherein the model data includes model source data, model requirement data and model data conversion rules;

[0037] A first format conversion module, configured to determine an open format converter of the model to be inferred according to model source data of the model to be inferred, and convert the model to be inferred into an open format model according to the open format converter corresponding to the model to be inferred;

[0038] A second format conversion module, configured to convert the open format model into an object model corresponding to the model format based on a conversion tool according to the model requirement data, the model data conversion rules, and the model format supported by the inference chip of the artificial intelligence platform;

[0039] The reasoning module is used to reason about the object model and obtain a reasoning result when receiving a reasoning request of the object model.

[0040] According to a third aspect of the present application, an electronic device is provided, including:

[0041] at least one processor; and

[0042] a memory communicatively connected to the at least one processor; wherein,

[0043] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present application.

[0044] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method described in the present application.

[0045] The method of the embodiment of the present application obtains the model to be inferred and the model data of the model to be inferred, wherein the model data includes model source data, model requirement data and model data conversion rules; according to the model source data of the model to be inferred, the open format converter of the model to be inferred is determined, and according to the open format converter corresponding to the model to be inferred, the model to be inferred is converted into an open format model; according to the model requirement data, the model data conversion rules and the model format supported by the inference chip of the artificial intelligence platform, the open format model is encapsulated into an object model corresponding to the model format based on a conversion tool; when receiving an inference request for the object model, the object model is inferred to obtain an inference result. By first converting the model to an open format and then encapsulating it into a format supported by the inference chip, the use of the original framework of the AI ​​platform is retained, the compatibility of the model with the AI ​​platform and its inference chip is enhanced, and the performance and efficiency degradation problems that may be caused by the use of other frameworks are avoided.

[0046] It should be understood that the teachings of the present application are not required to achieve all of the beneficial effects described above, but specific technical solutions can achieve specific technical effects, and other embodiments of the present application can also achieve beneficial effects not mentioned above. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:

[0048] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0049] Figure 1 A schematic diagram of the implementation process of the model reasoning method based on the artificial intelligence platform provided in the embodiment of the present application is shown;

[0050] Figure 2 A schematic diagram of the implementation flow of the reasoning operation of the model reasoning method based on the artificial intelligence platform provided in an embodiment of the present application is shown;

[0051] Figure 3 A schematic diagram of the composition structure of the artificial intelligence platform provided in the embodiment of the present application is shown;

[0052] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0054] Figure 1 A schematic diagram of the implementation flow of the model reasoning method based on the artificial intelligence platform provided in an embodiment of the present application is shown.

[0055] refer to Figure 1 , the embodiment of the present application provides a model reasoning method based on an artificial intelligence platform, the method comprising:

[0056] Operation 101 , obtaining a model to be inferred and model data of the model to be inferred, wherein the model data includes model source data, model requirement data and model data conversion rules.

[0057] In one embodiment of the present application, obtaining the model to be inferred includes: obtaining the model to be inferred from a model training framework, the model training framework including the open source software library TensorFlow, the deep learning framework PyTorch, the deep learning platform PaddlePaddle, and the open source deep learning framework MXNet.

[0058] In one embodiment of the present application, the model to be inferred is trained through a model training framework using the following operations: obtaining data to be labeled, and labeling the data to be labeled to obtain training data; dividing the training data into a training set and a test set; using the training set to train multiple models to be tested; and using the test set to determine the model to be inferred from multiple models to be tested.

[0059] Specifically, the existing model training frameworks include the open source software library TensorFlow, the deep learning framework PyTorch, the deep learning platform PaddlePaddle, and the open source deep learning framework MXNet. The model to be inferred can be trained in the above model training frameworks and migrated to the AI ​​platform after training.

[0060] The model training process may include: obtaining the data to be labeled, that is, the original data, then labeling the original data to obtain training data, and using the training data to train the model to be inferred.

[0061] Among them, the training data may include only the labeled data, or may include both the data to be labeled and the labeled data, depending on the type of the basic model of the model to be inferred. For example, in the case where the basic model is a supervised learning model, the training data only includes the labeled data. In the case where the basic model is a self-supervised learning or semi-supervised learning model, the training data may include both the data to be labeled and the labeled data. It should be noted that the present application does not make any specific limitation on the basic model of the model to be inferred. For example, the basic model may be a neural network model.

[0062] When using training data for training, in order to ensure the accuracy of the model to be inferred after the final training, the training data can be divided into a training set and a test set. The training set is used to train multiple models to be tested, and then the test set is used to select the model to be tested with the highest accuracy as the final model to be inferred.

[0063] In one embodiment of the present application, the AI ​​platform may be the Huawei Ascend AI platform.

[0064] Operation 102 : determining an open format converter of the model to be inferred according to model source data of the model to be inferred, and converting the model to be inferred into an open format model according to the open format converter corresponding to the model to be inferred.

[0065] The model source data usually includes the model training framework of the model to be inferred and the model format required by its model training framework. As a universal open format, onnx can be adapted to various model training frameworks, so the model to be inferred can be converted into an onnx model to facilitate the AI ​​platform to flexibly process models from different model training frameworks.

[0066] The AI ​​platform can be configured with an onnx converter that adapts to different model training frameworks. When receiving a model to be inferred, the model to be inferred is converted into an onnx model according to the model training framework and model format corresponding to the model to be inferred.

[0067] Operation 103, based on the model requirement data, model data conversion rules and the model format supported by the inference chip of the artificial intelligence platform, encapsulate the open format model into an object model corresponding to the model format based on the conversion tool.

[0068] In order to ensure the original framework of the AI ​​platform, it is also necessary to convert the onnx model into a model format adapted to the AI ​​platform to obtain the corresponding om (Object Model) model.

[0069] The om model usually cannot achieve dynamic adaptation. Therefore, in order to ensure the dynamic adaptability of the model to be inferred, the model requirement data and model data conversion rules mined in advance for the model to be inferred need to be encapsulated into the om model, so that the dynamic adaptation function of the om model can be realized by modifying the underlying logic of the om model during encapsulation. The dynamic adaptation function mainly refers to the dynamic preprocessing of the model input and output.

[0070] The model requirement data may be input and output requirements and performance requirements during model reasoning, and the model data conversion rules may be data format conversion or preprocessing rules.

[0071] In one embodiment of the present application, the conversion tool is ATC (Ascend Tensor Compiler, model conversion tool). Based on the use of ATC, the onnx model and single operators such as the model requirement data and model conversion rules of the onnx model can be converted into a model format supported by the inference chip of the AI ​​platform.

[0072] ATC is a model conversion tool under the CANN (Compute Architecture for Neural Networks) architecture system. Therefore, in order to ensure the use of ATC, it is necessary to configure the CANN driver package on the AI ​​platform in advance. Among them, the CANN driver package can be configured based on the construction of the ACL (Access Control List) development environment. The construction of the ACL development environment mainly includes the installation of the ACL development environment software toolkit ascend-toolkit and related tools.

[0073] Operation 104 , when receiving a reasoning request for the object model, reasoning is performed on the object model to obtain a reasoning result.

[0074] After the om model conversion is completed, when the reasoning request corresponding to the om model is received, the om model is called to complete the reasoning and obtain the reasoning result. Among them, om model reasoning can be regarded as using the om model to perform tasks supported by the om model. For example, when the om model is a text recognition model, om model reasoning can be regarded as using the om model to perform text recognition.

[0075] In one embodiment of the present application, the reasoning of the om model is performed based on the ACL development environment. The CANN driver suite can be regarded as a set of standard programming interfaces. When the om model is reasoned based on the ACL development environment, the CANN driver suite accelerates the om reasoning process to improve the efficiency of om model reasoning.

[0076] Therefore, the embodiment of the present application retains the use of the original framework of the AI ​​platform by first converting the model into an open format and then encapsulating it into a format supported by the inference chip, thereby enhancing the compatibility of the model with the AI ​​platform and its inference chip, and avoiding the performance and efficiency degradation that may be caused by the use of other frameworks.

[0077] In one embodiment of the present application, the model to be inferred includes multiple sub-models with logical relationships. Accordingly, the above operation 102 converts the model to be inferred into an open format model according to the open format converter corresponding to the model to be inferred, including: converting each sub-model into a sub-open format model according to the open format converter corresponding to the model to be inferred.

[0078] The model to be inferred can be a multi-stage model, including multiple sub-models with logical relationships, each sub-model corresponding to a stage of the model to be inferred. Then, converting the model to be inferred can be regarded as converting each sub-model of the model to be inferred to obtain a sub-onnx model corresponding to each sub-model. The conversion of each sub-model is performed using a unified open format converter.

[0079] In one embodiment of the present application, the model requirement data includes sub-model requirement data of each sub-model, and the model data conversion rules include sub-model data conversion rules of each sub-model; accordingly, the above operation 103 encapsulates the open format model into an object model corresponding to the model format according to the model requirement data, the model data conversion rules and the model format supported by the inference chip of the artificial intelligence platform, including: encapsulating each sub-open format model into a sub-object model corresponding to the model format according to the sub-model requirement data corresponding to each sub-open format model, the sub-model data conversion rules and the model format supported by the inference chip of the artificial intelligence platform.

[0080] Similar to the conversion process of sub-models, when the model to be inferred includes multiple sub-models, the model requirement data of the model to be inferred needs to include the sub-model requirement data of each sub-model, and the model data conversion rules need to include the sub-model data conversion rules of each sub-model. During encapsulation, a conversion tool is required to encapsulate each sub-onnx model into a sub-om model according to the corresponding sub-model requirement data and sub-model data conversion rules.

[0081] In one embodiment of the present application, when an inference request for an object model is received, the object model is inferred, including: inferring multiple sub-object models according to the logical relationship between the multiple sub-models.

[0082] When the model to be inferred is composed of multiple sub-models, the reasoning of the model to be inferred can be regarded as reasoning on the sub-om models corresponding to each sub-model. Among them, the reasoning of the multi-stage model is logical. When reasoning on multiple sub-om models, it is necessary to reason according to the logical relationship between the multiple sub-models corresponding to the multiple sub-om models. Among them, the logical relationship can refer to the reasoning order of multiple sub-models.

[0083] In one embodiment of the present application, the model to be inferred is an optical character recognition model, and the multiple sub-models include a text box detection model, a detection box direction model, and a text recognition model.

[0084] The model to be inferred in the embodiment of the present application may be an OCR (Optical Character Recognition) model. The OCR model generally includes a text box detection model, a detection box direction model, and a text recognition model. The three models correspond to the three stages of the OCR model, and process the input data in sequence.

[0085] In one embodiment of the present application, the sub-model requirement data of the text box detection model includes size requirements, and the sub-model data conversion rules of the text box detection model include size conversion rules; the sub-model requirement data of the detection box orientation model includes box size requirements, and the sub-model data conversion rules of the detection box orientation model include box size conversion rules and a multi-box detection mechanism; the sub-model requirement data of the text recognition model includes recognition size requirements, and the sub-model data conversion rules of the text recognition model include recognition size conversion rules and a multi-box recognition mechanism.

[0086] The OCR model has clear requirements on the size of the input data, and it cannot be an arbitrarily variable size. Therefore, in order to ensure that the effect of the OCR model is not affected after the OCR model is encapsulated as an OM model, it is necessary to determine the model data requirements and model data conversion rules of the three-stage OCR model in advance, so that when encapsulating OM, the model data requirements and model data conversion rules of the three-stage OCR model can be used to dynamically adapt the size of the preprocessing and retain the original effect of the OCR model.

[0087] Among them, the sub-model data conversion rules of the text box detection model include size conversion rules, and the sub-model data conversion rules of the text box detection model include size conversion rules.

[0088] For the text detection box model, the sub-model data conversion requirements are mainly related to its input size, that is, the size requirements, and the sub-model data conversion rules are the conversion rules for its input size, that is, the size conversion rules.

[0089] According to the input size characteristics of the input image in the actual use scenario of the text detection model, several sizes of different scales can be prepared from small to large. The image size can vary, but the aspect ratio of the image needs to be scaled proportionally. For example, various sizes such as (length: width) can be selected, such as (1:0.5), (1:1), (1:1.5), (1:2), (1:2.5), etc. In this way, by obtaining various sizes supported by the text box detection model, the various sizes supported by it can be packaged together during packaging, so that the packaged om model supports all sizes.

[0090] The om model has its own input requirements. During inference, the input images it receives are usually of the input size it supports, which is usually not suitable for the corresponding text box detection model. Therefore, the size conversion rules of the text box detection model need to be configured in advance so that the om model can convert the input images to the size supported by the corresponding text box detection model after receiving them.

[0091] The sub-model requirement data of the detection box direction model includes box size requirements, and the sub-model data conversion rules include box size conversion rules and multi-box detection mechanism.

[0092] The detection box direction model is mainly used to identify the specific direction of the text. The box size requirement can be configured as a fixed size. The configuration of its box size conversion rules can refer to the configuration of the text box detection model, which will not be repeated here. For the detection box direction model, since the image may usually include multiple detection boxes, it is also necessary to configure a multi-box detection mechanism so that when encapsulating the om model, the multi-box detection mechanism can be encapsulated together, so that the corresponding om model has the ability to reason about multiple boxes simultaneously.

[0093] The sub-model requirement data of the text recognition model includes recognition size requirements, and the sub-model data conversion rules include recognition size conversion rules and multi-box recognition mechanism.

[0094] The conversion principle of the text recognition model is the same as the above two models, but it needs to combine the advantages of both. The first point is that it also needs to support the ability to reason about multiple frames at the same time, because there are problems with text recognition in multiple detection frames at the same time. The second point is that it also needs to support multiple sizes for conversion, that is, it is necessary to configure the recognition size requirements and recognition size conversion rules, because the length and shape of the recognized text boxes are different. The recognition size conversion rule is to formulate the size to be converted according to the actual application scenario and convert it. For example, in the license plate recognition scenario, the size is relatively fixed, and it only needs to be converted according to the general shape recognized in the actual scenario.

[0095] Figure 2 A schematic diagram of the implementation flow of the reasoning operation of the model reasoning method based on the artificial intelligence platform provided in an embodiment of the present application is shown.

[0096] refer to Figure 2 In one embodiment of the present application, the inference result includes the text recognition position and the text recognition content; accordingly, when receiving the inference request of the object model, the object model is inferred to obtain the inference result, including:

[0097] Operation 201, when receiving an inference request of an object model, obtaining the to-be-inferred graphic and text data corresponding to the inference request;

[0098] Operation 202, converting the size of the image and text data to be inferred into a target size according to the size requirement and size conversion rule of the text box detection model;

[0099] Operation 203, inputting the image and text data to be inferred after the size conversion into the sub-object model corresponding to the text box detection model to obtain the text box detection result;

[0100] Operation 204, converting the frame size of the text frame shown in the text frame detection result into a first target frame size according to the frame size requirement of the detection frame direction model and the frame size conversion rule;

[0101] Operation 205, inputting the text box detection result after the box size conversion into the sub-object model corresponding to the detection box direction model, so that the sub-object model corresponding to the detection box direction model performs box direction detection on the text box detection result after the box size conversion according to the multi-box detection mechanism to obtain a box direction detection result;

[0102] Operation 206, converting the sizes of the multiple detection frames shown in the frame direction detection result into a second target frame size according to the recognition size requirement and recognition size conversion rule of the text recognition model;

[0103] Operation 207 , input the frame direction detection result after the frame size conversion into the sub-object model corresponding to the text recognition model, so that the sub-object model corresponding to the text recognition model performs text recognition on the frame direction detection result according to the multi-frame recognition mechanism to obtain the text recognition position and text recognition content.

[0104] When receiving the image and text data to be inferred, the sub-OM models corresponding to the text box detection model, the detection box direction model and the text recognition model are used to process the image and text data to be inferred in sequence, that is, to obtain the final text recognition position and text recognition content. In the input preprocessing stage of each sub-OM model, the sub-OM model converts the input it receives based on the corresponding sub-model requirement data and sub-data conversion rules. In the sub-OM model inference stage, the sub-OM model with a multi-box detection or recognition mechanism performs inference based on the multi-box detection or recognition mechanism.

[0105] The process of inferring the image and text data to be inferred into text recognition content and text recognition position can refer to the inference process of the conventional OCR model, which will not be repeated here.

[0106] Figure 3 A schematic diagram of the composition structure of the artificial intelligence platform provided in an embodiment of the present application is shown.

[0107] refer to Figure 3 Based on the above-mentioned model reasoning method based on the artificial intelligence platform, the embodiment of the present application further provides an artificial intelligence platform including:

[0108] An acquisition module 301 is used to acquire the model to be inferred and the model data of the model to be inferred, wherein the model data includes model source data, model requirement data and model data conversion rules;

[0109] A first format conversion module 302 is used to determine an open format converter of the model to be inferred according to the model source data of the model to be inferred, and convert the model to be inferred into an open format model according to the open format converter corresponding to the model to be inferred;

[0110] The second format conversion module 303 is used to convert the open format model into an object model corresponding to the model format based on a conversion tool according to the model requirement data, the model data conversion rules and the model format supported by the inference chip of the artificial intelligence platform;

[0111] The reasoning module 304 is used to reason about the object model and obtain a reasoning result when receiving a reasoning request of the object model.

[0112] It should be noted that the description of the device in the embodiment of the present application is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment, so it will not be repeated. Figure 1 to Figure 2 The present invention can be understood by referring to the description of any one of the accompanying drawings.

[0113] According to an embodiment of the present application, the present application also provides an electronic device and a non-transitory computer-readable storage medium.

[0114] Figure 4A schematic block diagram of an example electronic device 40 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0115] like Figure 4 As shown, the electronic device 40 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 40 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0116] A number of components in the electronic device 40 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0117] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as a model reasoning method based on an artificial intelligence platform. For example, in some embodiments, the model reasoning method based on an artificial intelligence platform may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 40 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the model reasoning method based on the artificial intelligence platform described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to execute a model reasoning method based on an artificial intelligence platform in any other appropriate manner (e.g., by means of firmware).

[0118] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0119] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0120] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0121] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0122] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0123] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0124] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0125] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model reasoning method based on an artificial intelligence platform, characterized in that: The method comprises: Acquire a model to be inferred and model data of the model to be inferred, wherein the model data includes model source data, model requirement data and model data conversion rules; Determining an open format converter of the model to be inferred according to the model source data of the model to be inferred, and converting the model to be inferred into an open format model according to the open format converter corresponding to the model to be inferred; According to the model requirement data, the model data conversion rules, and the model format supported by the inference chip of the artificial intelligence platform, based on a conversion tool, the open format model is encapsulated into an object model corresponding to the model format; When receiving the inference request of the object model, the object model is inferred to obtain an inference result.

2. The method according to claim 1, characterized in that The model to be inferred includes multiple sub-models with logical relationships. According to the open format converter corresponding to the model to be inferred, converting the model to be inferred into an open format model includes: Each sub-model is converted into a sub-open format model according to the open format converter corresponding to the model to be inferred.

3. The method according to claim 2, characterized in that The model requirement data includes sub-model requirement data of each sub-model, and the model data conversion rules include sub-model data conversion rules of each sub-model; accordingly, According to the model requirement data, the model data conversion rules and the model format supported by the inference chip of the artificial intelligence platform, the open format model is encapsulated into an object model corresponding to the model format, including: According to the sub-model requirement data corresponding to each sub-open format model, the sub-model data conversion rules and the model format supported by the inference chip of the artificial intelligence platform, each sub-open format model is encapsulated as a sub-object model corresponding to the model format.

4. The method according to claim 3, characterized in that The step of inferring the object model upon receiving the inference request of the object model comprises: Reasoning is performed on multiple sub-object models based on the logical relationships between the multiple sub-models.

5. The method according to claim 3, characterized in that: The model to be inferred is an optical character recognition model, and the multiple sub-models include a text box detection model, a detection box direction model, and a text recognition model.

6. The method according to claim 5, characterized in that The sub-model requirement data of the text box detection model includes size requirements, and the sub-model data conversion rules of the text box detection model include size conversion rules; the sub-model requirement data of the detection box orientation model includes box size requirements, and the sub-model data conversion rules of the detection box orientation model include box size conversion rules and multi-box detection mechanisms; the sub-model requirement data of the text recognition model includes recognition size requirements, and the sub-model data conversion rules of the text recognition model include recognition size conversion rules and multi-box recognition mechanisms.

7. The method according to claim 6, characterized in that The inference result includes the text recognition position and the text recognition content; accordingly, The step of, upon receiving the inference request of the object model, inferring the object model to obtain an inference result, comprises: When receiving an inference request of the object model, obtaining the to-be-inferred graphic and text data corresponding to the inference request; According to the size requirements and size conversion rules of the text box detection model, the size of the image and text data to be inferred is converted into a target size; Input the size-converted image and text data to be inferred into the sub-object model corresponding to the text box detection model to obtain a text box detection result; Converting the frame size of the text frame shown in the text frame detection result into a first target frame size according to the frame size requirement and frame size conversion rule of the detection frame direction model; Inputting the text frame detection result after the frame size conversion into the sub-object model corresponding to the detection frame direction model, so that the sub-object model corresponding to the detection frame direction model performs frame direction detection on the text frame detection result after the frame size conversion according to the multi-frame detection mechanism to obtain a frame direction detection result; According to the recognition size requirement and recognition size conversion rule of the text recognition model, the sizes of the multiple detection frames shown in the frame direction detection result are converted into the second target frame size; The frame direction detection result after the frame size conversion is input into the sub-object model corresponding to the text recognition model, so that the sub-object model corresponding to the text recognition model performs text recognition on the frame direction detection result according to the multi-frame recognition mechanism to obtain the text recognition position and text recognition content.

8. The method according to claim 1, characterized in that Get the model to be inferred, including: The model to be inferred is obtained from the model training framework, which includes the open source software library TensorFlow, the deep learning framework PyTorch, the deep learning platform PaddlePaddle, and the open source deep learning framework MXNet.

9. The method according to claim 8, characterized in that The model to be inferred is trained by the model training framework using the following operations: Obtain the data to be labeled, and label the data to be labeled to obtain training data; Dividing the training data into a training set and a test set; Use the training set to train multiple models to be tested; The model to be inferred is determined from the multiple models to be tested using a test set.

10. An artificial intelligence platform, characterized in that: The platform includes: An acquisition module, used to acquire the model to be inferred and model data of the model to be inferred, wherein the model data includes model source data, model requirement data and model data conversion rules; A first format conversion module, configured to determine an open format converter of the model to be inferred according to model source data of the model to be inferred, and convert the model to be inferred into an open format model according to the open format converter corresponding to the model to be inferred; A second format conversion module, configured to convert the open format model into an object model corresponding to the model format based on a conversion tool according to the model requirement data, the model data conversion rules, and the model format supported by the inference chip of the artificial intelligence platform; The reasoning module is used to reason about the object model and obtain a reasoning result when receiving a reasoning request of the object model.

Images