AI reasoning packaging method for mobile terminal and related device

Abstract

The invention relates to an artificial intelligence (AI) reasoning packaging method for a mobile terminal and a related device. The AI reasoning packaging method for the mobile terminal comprises the steps that an AI reasoning model is initialized based on the specified condition of the mobile terminal, and the specified condition is used for determining the calculation requirement of AI reasoning; converting the original data called by the AI inference model into a feature data format to form a feature mapping base class, and providing a data access interface associated with the feature mapping base class; obtaining model input data through the data access interface, and filling the model input data into the AI reasoning model to execute reasoning; and extracting model output data of the AI reasoning model to be applied to the model reasoning result.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to an artificial intelligence (AI) reasoning packaging method for a mobile terminal, an electronic device, a non-transitory storage medium, and a computer program product. Background Art

[0002] At present, it is difficult to reach a consensus on the choice of artificial intelligence (AI) reasoning framework on mobile phones. The general practice in the industry is to choose a general reasoning framework open sourced by Internet giants, such as Alibaba's MNN and Tencent's TNN. The advantage is that it can be used on multiple terminals, but the disadvantage is that it cannot give full play to the computing power of the NPU processor of different chips. Another approach is to adapt the exclusive closed-source reasoning framework created by the chip factory based on the chip's own neural network processing unit (NPU) according to the chips of different mobile phones, such as Qualcomm's QNN, Huawei HiSilicon's HiAI, Apple's CoreML, etc. Although this method can maximize the speed and energy efficiency of AI reasoning, the data structure and data transmission logic between the frameworks are not the same at present. In addition, the application programming interface (API: Application Programming Interface) calling method is flourishing. For developers of mobile applications (APP: Application) that need to be adapted to multiple terminals and multiple models, each operating system and each mobile phone chip needs to be adapted separately, which consumes huge manpower costs. Summary of the invention

[0003] A brief overview of the disclosure is given below in order to provide a basic understanding of some aspects of the disclosure. However, it should be understood that this overview is not an exhaustive overview of the disclosure. It is not intended to identify the key or important parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is simply to give some concepts of the disclosure in a simplified form as a prelude to a more detailed description given later.

[0004] According to a first aspect of the present disclosure, there is provided an AI reasoning encapsulation method for a mobile terminal, comprising: initializing an AI reasoning model based on specified conditions of the mobile terminal, wherein the specified conditions are used to determine the computing requirements of the AI ​​reasoning; converting the original data called by the AI ​​reasoning model into a feature data format to form a feature mapping base class, and providing a data access interface associated with the feature mapping base class; acquiring model input data via the data access interface, thereby filling the model input data into the AI ​​reasoning model to perform reasoning; and taking out the model output data of the AI ​​reasoning model to apply to the model reasoning result.

[0005] According to a second aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory storing computer executable instructions, wherein when the computer executable instructions are executed by the processor, the processor executes the AI ​​reasoning encapsulation method for a mobile terminal according to the first aspect of the present disclosure.

[0006] According to a third aspect of the present disclosure, a non-volatile storage medium having computer executable instructions stored thereon is provided. When the computer executable instructions are executed by a processor, the processor executes the AI ​​reasoning encapsulation method for a mobile terminal according to the first aspect of the present disclosure.

[0007] According to a fourth aspect of the present disclosure, a computer program product is provided. The computer program product includes instructions, and when the instructions are executed by a processor, they implement the AI ​​reasoning encapsulation method for a mobile terminal according to the first aspect of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The foregoing and other features and advantages of the present disclosure will become apparent from the following description of the embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are incorporated herein and form a part of the specification, and are further used to explain the principles of the present disclosure and enable those skilled in the art to make and use the present disclosure. Among them:

[0009] Figure 1 A flowchart of an AI reasoning encapsulation method for a mobile terminal according to some embodiments of the present disclosure is shown;

[0010] Figure 2 A flowchart of an abstract base class of an AI reasoning encapsulation method for a mobile terminal according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram showing data format conversion in an AI reasoning encapsulation method for a mobile terminal according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A schematic block diagram of an electronic device according to some embodiments of the present disclosure is shown;

[0013] Figure 5 A schematic block diagram of a computer system is shown on which embodiments of the present disclosure may be implemented.

[0014] Note that in the embodiments described below, sometimes the same reference numerals are used in common between different drawings to represent the same parts or parts with the same functions, and their repeated descriptions are omitted. In some cases, similar numbers and letters are used to represent similar items, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0015] For ease of understanding, the position, size, range, etc. of each structure shown in the drawings and the like may not represent the actual position, size, range, etc. Therefore, the present disclosure is not limited to the position, size, range, etc. disclosed in the drawings and the like. DETAILED DESCRIPTION

[0016] Various exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0017] The following description of at least one exemplary embodiment is in fact merely illustrative and is in no way intended to limit the present disclosure and its application or use. That is, the structures and methods herein are shown in an exemplary manner to illustrate different embodiments of the structures and methods in the present disclosure. However, those skilled in the art will appreciate that they merely illustrate exemplary ways of the present disclosure that can be implemented, rather than exhaustive ways. In addition, the drawings need not be drawn to scale, and some features may be enlarged to illustrate the details of specific components.

[0018] In addition, technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0019] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0020] It is difficult for existing mobile terminals to unify the selection of AI reasoning frameworks. The main solution includes selecting an open source general reasoning framework to make the application universal on multiple terminals. Taking the general reasoning framework MNN as an example, it has access to closed-source reasoning frameworks such as HiAI and CoreML built for dedicated chips at its bottom layer, aiming to allow mobile APP developers to achieve corresponding NPU reasoning acceleration on different device chips by simply calling MNN. However, due to the underlying compatibility, MNN needs to translate and splice the operation (op: operation) layer of the model when actually calling the closed-source NPU framework, which is not as efficient as directly calling the original reasoning framework in terms of AI reasoning speed. In addition, the underlying compatibility form strongly depends on the development rhythm of the MNN framework developer, and it is not possible to quickly access emerging frameworks in a portable and efficient manner, and the scalability is very poor. On the other hand, for the selection of AI reasoning frameworks, applications can also be designed separately according to the closed-source reasoning frameworks corresponding to the chips of different mobile terminals. Different API calling methods may be used for each reasoning framework, resulting in a huge workload for APP developers to adapt to multiple terminals and multiple models.

[0021] To solve such problems, the present disclosure relates to a unified AI reasoning encapsulation method for mobile terminals. This method provides a terminal-side AI reasoning framework encapsulation interface for callers such as APP developers, which can undertake various existing reasoning frameworks downward, and can enable callers to always only need to adapt a set of APIs upward, in which different reasoning frameworks are specified in the form of parameters during actual reasoning. At the same time, based on this encapsulation method, the AI ​​reasoning process and data transmission logic of the reasoning framework are highly abstracted, and subsequent new reasoning frameworks can also be quickly adapted according to this abstract logic.

[0022] The cloth animation generation method for a virtual character according to various embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings. It is understood that the actual cloth animation generation method for a virtual character may also include other steps, but in order to avoid blurring the main points of the present disclosure, these other steps are not discussed herein and are not shown in the accompanying drawings.

[0023] Figure 1 A flowchart of an AI reasoning encapsulation method for a mobile terminal according to some embodiments of the present disclosure is shown. Figure 1As shown, method 100 includes steps S101 to S104. At step S101, the AI ​​reasoning model is initialized based on the specified conditions of the mobile terminal, wherein the specified conditions are used to determine the computing requirements of the AI ​​reasoning. At step S102, the raw data of the AI ​​reasoning is converted into a feature data format to form a feature mapping base class, and a data access interface associated with the feature mapping base class is provided. At step S103, the model input data is obtained via the data access interface, thereby filling the model input data into the AI ​​reasoning model to perform reasoning. At step S104, the model output data of the AI ​​reasoning model is taken out to be applied to the model reasoning result.

[0024] Next, combine Figure 2 Embodiments of the present disclosure are described. Figure 2 A flowchart of an abstract base class of an AI reasoning encapsulation method for a mobile terminal according to some embodiments of the present disclosure is shown. Among them, the basic process of terminal-side AI reasoning can be abstracted into a unified pipeline base class, wherein the class has 6 basic functions, corresponding to the six steps of AI reasoning, namely initializing the model 210, obtaining the model input data block 220, filling in the data 230, executing the reasoning 240, obtaining the model output data block 250, and retrieving the data 260, and each of these steps forms an API that can be called by upper-level developers. The upper-level caller only needs to use a set of interfaces, specify any reasoning framework that has been connected to the pipeline base class, and execute the above 6 steps in sequence to complete the corresponding AI reasoning. Among them, in the steps of filling in the data 230 and retrieving the data 260, the caller only needs to interact with the feature mapping class according to the original data structure, without having to care about the different data formats defined by each reasoning framework. Furthermore, retrieving the data 260 also includes using the model output data to apply the model reasoning results 2620 (corresponding to Figure 1 Step S104).

[0025] In some non-limiting embodiments, when initializing model 220 (corresponding to Figure 1In step S101), the upper-layer caller can initialize the required AI reasoning model based on the specified conditions of the mobile terminal, where the specified conditions may include but are not limited to the specified reasoning framework 2110, the specified computing chip 2120, and the specified computing precision 2130. Specifically, the specified reasoning framework to be used, such as MNN, TNN, HiAI, CoreML, etc., can be input by adjusting the parameters; the computing chip used for reasoning, such as CPU, GPU, NPU, etc.; and the computing precision of the specified AI reasoning model, such as single precision, half precision, etc. In particular, if the required AI reasoning model supports dynamic shape, the parameters in these specified conditions can also be set after the initialization step, and thus the reasoning can be completed and the reasoning results can be obtained according to the remaining five steps mentioned above.

[0026] In some non-limiting embodiments, in the Get Model Input Data block 220 (corresponding to Figure 1 In step S102), the model input data can be obtained by converting the data format of the original data called by the AI ​​reasoning model. Generally, the original data used for AI reasoning is mostly images 2310, and also includes data types such as audio 2320 and text 2330. These data types are converted into a series of values ​​(ie, data blocks) that can be input into the AI ​​reasoning model through corresponding pre-processing algorithms (such as image 2310 for image processing, audio 2320 for audio processing, text 2330 for text processing, etc.), and the converted values ​​are stored using the agreed data format.

[0027] Specifically, please refer to Figure 3 , Figure 3 A schematic diagram of data format conversion in an AI reasoning encapsulation method for a mobile terminal according to some embodiments of the present disclosure is shown. Figure 3Two common data storage formats NCHW (310) and NHWC (320) are shown in the figure, where N represents the batch size of the data block, C represents the channel, H represents the height, and W represents the width. Taking an RGB image with a height of h and a width of w as an example, its data storage format is generally h×w data groups, each of which is three 8-bit unsigned integers (i.e., unsigned int8), which correspond to the values ​​of the three primary colors of red, green, and blue of the pixel point in sequence, that is, it can be expressed in NHWC format, where N=1 and C=3. Accordingly, the convolution algorithm currently dominates the neural network, making its related data format NCHW the mainstream of model input. In addition, in order to improve computing efficiency, the MNN reasoning framework has introduced a new NC4HW4 format for instruction set computing, while QNN prefers the NHWC data format. This means that the data formats that AI inference models need to call are different. If you want to use a certain inference framework, you must first convert the original data into the data format it specifies, which undoubtedly increases the development difficulty for the caller.

[0028] Based on this, the present disclosure provides an abstract method for the aforementioned data format. Among them, the original data called by the AI ​​reasoning model is converted into an abstract feature data format to form a feature map FeatureMap base class, which has no definite data format, but provides a data access interface associated with the feature map base class (such as the at interface for calling the ai function). The data access interface is used to implement the input of a feature map vector in a unified format so that the AI ​​reasoning model accesses the model storage data corresponding to the feature map vector, such as by inputting a specified (N, H, W, C) value to accurately obtain and / or modify the data at the corresponding position in the data block.

[0029] Additionally, in response to the AI ​​reasoning model in the initialization model 210 being initialized based on specified conditions, the data access interface is rewritten to execute the interaction between the calling data format of the AI ​​reasoning model and the feature mapping base class. Among them, the feature mapping base class supports classes inherited as other data formats, and only needs to rewrite the data access interface according to the characteristics of the data format. That is, for the caller, the operation of filling in data 230 and / or retrieving data 260 only involves interaction with the feature mapping base class, without considering the specific data structure of the underlying model. Specifically, the data access interface internally supports algorithms for a variety of different data structures, thereby automatically matching the corresponding algorithm model according to the required calling data format.

[0030] In some non-limiting embodiments, the at interface is used to access the value of a specific position in the tensor FeatureMap. For example, when the image data is input into the model, the at function needs to be called to assign each pixel value in an image to the FeatureMap in turn. Taking the FeatureMap in NCHW format as an example, the length of the cube data structure is n, the width is w, and the height is h. The small cube at each position stores a number of data values ​​of c. Considering that these data are stored continuously, if you want to obtain the second value in the small cube with a height of 5 in the 9th row, 7th column, and height of the cube data structure, you need to calculate the offset of this data relative to the first data in the cube data structure. As mentioned above, due to the difference in reasoning frameworks and / or computing chips, the calculation method of the offset will vary depending on the data format, but for the feature mapping base class in the present disclosure, you only need to use FeatureMap.at(9, 7, 5, 2) to get the corresponding value. In other words, the caller does not need to consider the specific underlying data structure that needs to be called, whether it is FeatureMap in the form of NCHW, NHWC or NC4HW4, as long as the ".at(9,7,5,2)" interface is used, the data at the corresponding location can be accurately obtained or modified.

[0031] Additionally or alternatively, the AI ​​reasoning encapsulation method disclosed herein also provides a data interaction function, which can implement the interaction between the original data format and the feature mapping base class based on the original data passed in by the caller and / or the original data format explicitly informed by the caller, thereby improving the computing efficiency.

[0032] In some non-limiting embodiments, in order to optimize performance, in addition to using basic C++ implementation, data interaction functions can also be implemented by the NEON instruction set unique to the Advanced Reduced Instruction Set Computer Machine (ARM) architecture, and its data processing performance is improved by about 3 times. Taking the conversion of an RGB unsigned int8 image with the original data type into NCHW float32 data as an example, the original data is R0G0B0R1G1B1R2G2B2R3G3B3...(unsigned int8), and the data on the three channels are read crosswise to the corresponding three NEON registers through the "vld3q_u8" instruction, and then the RGB unsigned int8 data is converted into float32 data using instruction sets such as "vcvtq_f32_s32" and "vreinterpretq_s16_u16", and then filled back in sequence without crosswise, obtaining NCHW data such as R0R1R2R3...G0G1G2G3...B0B1B2B3...(float32).

[0033] Additionally, a data interaction function is used to directly convert the original data format of the original data into the calling data format of the AI ​​reasoning model. For some data formats unique to the reasoning framework, the existing processing method is to convert the special format into the mainstream NCHW format first, and then interact with the original data. Taking MNN's unique NC4HW4 format as an example, in order to lower the usage threshold, MNN provides a function for converting NC4HW4 to NCHW; at this time, if the original data format is RGB format (which can be understood as NHWC format with C=3), the data interaction process includes multiple conversions of RGB->NCHW->NC4HW4, which will increase the reasoning delay. The data interaction function disclosed in the present invention may include the underlying implementation of direct conversion of RGB to NC4HW4. Among them, the original data is R0G0B0R1G1B1R2G2B2R3G3B3...(unsigned int8), and the target calling data format is R0G0B00R1G1B10R2G2B20R3G3B30...(float32). According to the understanding of the NC4HW4 format and the characteristic of RGB format C=3, the conversion from RGB to NC4HW4 only needs to read 3 unsigned int8 data in succession, convert them into float32 and add a 0 immediately after filling them back, without the need for multiple data format conversions.

[0034] Similarly, for other special formats, the AI ​​reasoning encapsulation method disclosed in the present invention also supports additional expansion thereof. It is only necessary to rewrite the corresponding data interaction function in the above manner, wherein the specific acceleration means can be selected according to the actual situation.

[0035] Additionally or alternatively, the AI ​​reasoning encapsulation method of the present disclosure also provides an integration method for a new reasoning framework. In response to the addition of a first reasoning framework, a first feature mapping class associated with the first reasoning framework and inherited from the feature mapping base class is added according to the first data format for performing interaction with the first reasoning framework. Furthermore, the data access interface is rewritten according to the first data format to implement input of a first feature mapping vector in a unified format so that the first AI reasoning model associated with the first reasoning framework accesses the data corresponding to the first feature mapping vector.

[0036] In some non-limiting embodiments, the extensibility of the AI ​​reasoning encapsulation method disclosed in the present invention is mainly realized based on the concreteness of data types and the concreteness of reasoning processes. Taking the addition of a QNN framework to the encapsulation method as an example, it is known that the data format of QNN's external interaction is NHWC. First, a QNNFeatureMap class inherited from the feature mapping base class is added, and the at interface is rewritten according to the data arrangement characteristics of NHWC (such as the data proximity of channels), so that the corresponding data can be found by inputting the specified feature mapping vector (N, H, W, C).

[0037] Therefore, the embodiments of the present disclosure provide a unified end-side AI reasoning encapsulation method that can be applied to multiple reasoning frameworks and multiple computing chips, so that the upper-layer caller can use the underlying AI reasoning framework as a black box algorithm, that is, only need to focus on providing data and retrieving data, without having to invest a lot of time and energy in adapting various frameworks and various data structures. For mobile phones with different operating systems (Android, iOS, Hongmeng, etc.), APP developers can use the same set of codes to achieve fast and efficient deployment.

[0038] In another aspect, the present disclosure further provides an electronic device, which may include a processor and a memory storing computer executable instructions, wherein when the computer executable instructions are executed by the processor, the processor executes the AI ​​reasoning encapsulation method for a mobile terminal according to any of the aforementioned embodiments of the present disclosure. Figure 4 , which shows a schematic block diagram of an electronic device 400 according to some embodiments of the present disclosure. Figure 4As shown, the electronic device 400 includes a processor 402 and a memory 404 storing computer executable instructions, which, when executed by the processor 402, causes the processor 402 to perform the AI ​​reasoning encapsulation method for a mobile terminal according to any of the foregoing embodiments of the present disclosure. The processor 402 may be, for example, a central processing unit (CPU) of the electronic device 400. The processor 402 may be any type of general-purpose processor, or may be a processor specifically designed for an AI reasoning framework, such as an application-specific integrated circuit (“ASIC”). The memory 404 may be coupled to the processor 402 and may include various computer-readable media accessible by the processor 402. In various embodiments, the memory 404 described herein may include volatile and non-volatile media, removable and non-removable media. For example, the memory 404 may include any combination of the following: random access memory (“RAM”), dynamic RAM (“DRAM”), static RAM (“SRAM”), read-only memory (“ROM”), flash memory, cache memory, and / or any other type of non-transient computer-readable media. The memory 404 may store instructions that, when executed by the processor 402, enable the processor 402 to execute the AI ​​reasoning encapsulation method for a mobile terminal according to any of the aforementioned embodiments of the present disclosure.

[0039] In a non-limiting embodiment, the aforementioned AI reasoning encapsulation method is abstracted into a pipeline base class. Then, in response to the addition of the first reasoning framework, a first pipeline class associated with the first reasoning framework and inherited from the pipeline base class is added according to the first AI reasoning model associated with the first reasoning framework. Taking the addition of the QNN framework to the encapsulation method as an example, when the QNNFeatureMap class is added, a QNNPipeline class inherited from the pipeline base class is added. At this time, it is necessary to classify the actual reasoning steps of the QNN into Figure 2 The encapsulation can be completed by rewriting the corresponding six basic functions according to the six abstract steps shown.

[0040] The present disclosure also provides a non-volatile storage medium having computer executable instructions stored thereon, which, when executed by a processor, enables the processor to execute the method for modulating a signal according to any of the foregoing embodiments of the present disclosure or the AI ​​reasoning encapsulation method for a mobile terminal according to any of the foregoing embodiments.

[0041] The present disclosure also provides a computer program product, which may include instructions, which, when executed by a processor, may implement the method for modulating a signal according to any of the foregoing embodiments of the present disclosure or the method for an AI reasoning framework according to any of the foregoing embodiments. The instructions may be any set of instructions to be executed directly by a processor, such as machine code, or any set of instructions to be executed indirectly, such as a script. The instructions may be stored in an object code format for direct processing by a processor, or in any other computer language, including scripts or collections of independent source code modules that are interpreted on demand or compiled in advance.

[0042] Figure 5A schematic block diagram of a computer system 500 on which embodiments of the present disclosure may be implemented is shown. The computer system 500 includes a bus 502 or other communication mechanism for transmitting information, and a processing device 504 coupled to the bus 502 for processing information. The computer system 500 also includes a memory 506 coupled to the bus 502 for storing instructions to be executed by the processing device 504, which may be a random access memory (RAM) or other dynamic storage device. The memory 506 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processing device 504. The computer system 500 also includes a read-only memory (ROM) 508 or other static storage device coupled to the bus 502 for storing static information and instructions for the processing device 504. A storage device 510 such as a magnetic disk or optical disk is provided and coupled to the bus 502 for storing information and instructions. The computer system 500 may be coupled to an output device 512 via the bus 502 for providing output to a user, such as, but not limited to, a display (such as a cathode ray tube (CRT) or a liquid crystal display (LCD)), a speaker, etc. An input device 514 such as a keyboard, a mouse, a microphone, etc. is coupled to the bus 502 for transmitting information and command selections to the processing device 504. The computer system 500 may perform embodiments of the present disclosure. Consistent with certain implementations of the present disclosure, the results are provided by the computer system 500 in response to the processing device 504 executing one or more sequences of one or more instructions contained in the memory 506. Such instructions may be read into the memory 506 from another computer-readable medium such as the storage device 510. The execution of the sequence of instructions contained in the memory 506 causes the processing device 504 to perform the methods described herein. Alternatively, hard-wired circuitry may be used instead of software instructions or in combination with software instructions to implement the present teachings. Therefore, implementations of the present disclosure are not limited to any specific combination of hardware circuitry and software. In various embodiments, computer system 500 can be connected to one or more other computer systems like computer system 500 across a network via network interface 516 to form a networked system. The network can include a private network or a public network such as the Internet. In a networked system, one or more computer systems can store data and supply data to other computer systems. The term "computer-readable medium" as used herein refers to any medium that participates in providing instructions to processing device 504 for execution. Such media can take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical disks or disks such as storage device 510. Volatile media include dynamic memories such as memory 506. Transmission media include coaxial cables, copper wires, and optical fibers, including wiring that includes bus 502.Common forms of computer-readable media or computer program products include, for example, floppy disks, flexible disks, hard disks, tapes, or any other magnetic media, CD-ROMs, digital video disks (DVDs), Blu-ray disks, any other optical media, thumb drives, memory cards, RAMs, PROMs and EPROMs, flash EPROMs, any other memory chips or boxes, or any other tangible media from which a computer can read. Various forms of computer-readable media may be involved when carrying one or more sequences of one or more instructions to processing device 504 for execution. For example, instructions may initially be carried on a disk of a remote computer. The remote computer may load instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 500 may receive data over a telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector coupled to bus 502 may receive the data carried in the infrared signal and place the data on bus 502. Bus 502 carries the data to memory 506, and processing device 504 retrieves instructions from memory 506 and executes the instructions. Optionally, the instructions received by memory 506 may be stored on storage device 510 either before or after execution by processing device 504 .

[0043] According to various embodiments, instructions configured to be executed by a processing device to perform a method are stored on a computer-readable medium. A computer-readable medium may be a device that stores digital information. For example, a computer-readable medium includes a compact disk read-only memory (CD-ROM) as known in the art for storing software. The computer-readable medium is accessed by a processor adapted to execute the instructions configured to be executed.

[0044] One or more exemplary embodiments of the present disclosure are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0045] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a server system. Of course, the present disclosure does not exclude that with the development of computer technology in the future, the computer that implements the functions of the above embodiments may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0046] Although one or more embodiments of the present disclosure provide method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment).

[0047] The terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, product or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, product or apparatus. In the absence of further restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or apparatus comprising the elements. For example, if the words "first", "second" and the like are used to indicate names, they do not indicate any particular order.

[0048] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing one or more embodiments of the present disclosure, the functions of each module can be implemented in the same or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0049] The present disclosure is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0050] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a particular manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device that implements the functions specified in one or more flows of a flowchart and / or one or more blocks of a block diagram. These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operating steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows of a flowchart and / or one or more blocks of a block diagram.

[0051] Those skilled in the art should understand that one or more embodiments of the present disclosure may be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present disclosure may be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0052] One or more embodiments of the present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0053] The same or similar parts between the various embodiments of the present disclosure can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment. In the description of the present disclosure, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present disclosure, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in the present disclosure and the features of the different embodiments or examples without contradiction.

[0054] In addition, when used in the present disclosure, the words "herein," "above," "below," "hereunder," "above," and words of similar meaning shall refer to the present disclosure as a whole rather than to any particular portion of the present disclosure. Furthermore, unless expressly stated otherwise or understood otherwise in the context of use, conditional language used herein, such as "may," "might," "for example," "such as," and the like, is generally intended to express that certain embodiments include, while other embodiments do not include, certain features, elements, and / or states. Thus, such conditional language is generally not intended to imply that one or more embodiments require features, elements, and / or states in any way, or whether such features, elements, and / or states are included or performed in any particular embodiment.

[0055] The above description is only an example of one or more embodiments of the present disclosure, and is not intended to limit one or more embodiments of the present disclosure. For those skilled in the art, one or more embodiments of the present disclosure may have various changes and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the claims.

Claims

1. An artificial intelligence (AI) reasoning encapsulation method for a mobile terminal, comprising: Initializing an AI reasoning model based on specified conditions on the mobile terminal, wherein the specified conditions are used to determine computing requirements for AI reasoning; Converting the raw data called by the AI ​​reasoning model into a feature data format to form a feature mapping base class, and providing a data access interface associated with the feature mapping base class; Acquiring model input data via the data access interface, thereby filling the model input data into the AI ​​reasoning model to perform reasoning; as well as The model output data of the AI ​​reasoning model is taken out to be applied to the model reasoning result.

2. The AI ​​reasoning packaging method according to claim 1, wherein: The specified conditions include one or more of a specified inference framework, a specified computing chip, and a specified computing accuracy.

3. The AI ​​reasoning packaging method according to claim 1, wherein: In response to the AI ​​reasoning model being initialized, the data access interface is rewritten to perform interaction between the calling data format of the AI ​​reasoning model and the feature mapping base class.

4. The AI ​​reasoning packaging method according to claim 3, wherein: The data access interface is used to implement input of feature mapping vectors in a unified format so that the AI ​​reasoning model accesses model storage data corresponding to the feature mapping vectors.

5. The AI ​​reasoning packaging method according to claim 3, wherein: The data interaction function is used to realize the interaction between the original data format of the original data and the feature mapping base class, wherein The data interaction function is implemented by the NEON instruction set of the advanced reduced instruction set machine ARM architecture.

6. The AI ​​reasoning packaging method according to claim 1, wherein: The original data format of the original data is directly converted into the calling data format of the AI ​​reasoning model using a data interaction function.

7. The AI ​​reasoning packaging method according to claim 1, further comprising: In response to the addition of the first reasoning framework, a first feature mapping class associated with the first reasoning framework is added according to a first data format for performing interaction with the first reasoning framework, wherein the first feature mapping class inherits from the feature mapping base class; as well as The data access interface is rewritten according to the first data format to implement input of a first feature mapping vector in a unified format so that a first AI reasoning model associated with a first reasoning framework accesses data corresponding to the first feature mapping vector.

8. An electronic device comprising: processor; as well as A memory storing computer executable instructions, wherein when the computer executable instructions are executed by the processor, the processor executes the artificial intelligence (AI) reasoning packaging method for a mobile terminal according to claim 1, wherein Abstract the AI ​​reasoning encapsulation method into a pipeline base class, and In response to the newly added first reasoning framework, a first pipeline class associated with the first reasoning framework is added according to a first AI reasoning model associated with the first reasoning framework, wherein the first pipeline class inherits from the pipeline base class.

9. A non-volatile storage medium having computer executable instructions stored thereon, wherein when the computer executable instructions are executed by a processor, the processor executes the artificial intelligence (AI) reasoning encapsulation method for a mobile terminal according to any one of claims 1 to 7.

10. A computer program product, comprising instructions, which, when executed by a processor, implement the artificial intelligence (AI) reasoning encapsulation method for a mobile terminal according to any one of claims 1 to 7.

Images