Electronic device and method of handling artificial intelligence (AI) model switching of electronic device

Abstract

The invention provides an electronic device. The electronic device comprises a storage device; a memory; and processing circuitry configured to execute the instructions: compile a first artificial intelligence (AI) model and at least one second AI model into a first compiled file and at least one second compiled file, respectively; generating at least one lightweight file for the at least one second AI model according to the first compiling file and the at least one second compiling file; the first compiled file and the at least one lightweight file are stored in the storage device; when the first AI model is deduced, loading the first compiled file to the memory; loading the at least one lightweight file to the memory during inferring the at least one second AI model; generating at least one second data set according to the first data set and at least one data patch in the memory; and executing the at least one second AI model according to the generated at least one second data set and at least one second instruction set in the memory.

Description

Background Art

[0002] An Artificial Intelligence (AI) model is a program that is trained with a set of data to recognize specific patterns or make specific decisions without human intervention. An AI model consists of two main elements: instructions executed by the hardware of an electronic device, such as a neural network processing unit (NPU) or a graphics processing unit (GPU); and data used as weights and quantization parameters of the AI ​​model. The size of some AI models has become larger due to the growth of AI models, and these AI models with similar weights are retrained or fine-tuned to cater to different applications and provide users with various styles. Therefore, electronic devices need to switch between these AI models to meet various application requirements. Summary of the invention

[0003] The present invention provides an electronic device and a method for processing an artificial intelligence (AI) model switching of the electronic device.

[0004] In one embodiment, the present invention provides an electronic device, comprising: a storage device; a memory; and a processing circuit coupled to the storage device and the memory and configured to execute instructions: respectively compile a first artificial intelligence (AI) model and at least one second AI model into a first compiled file and at least one second compiled file, wherein the first compiled file includes a first data set and a first instruction set, and the at least one second compiled file includes at least one second data set and at least one second instruction set; generate at least one lightweight version file for the at least one second AI model according to the first compiled file and the at least one second compiled file, wherein the at least one lightweight version file includes the at least one second instruction set and at least one data patch; store the first compiled file and the at least one lightweight version file in the storage device; during inference of the first AI model, load the first compiled file from the storage device to the memory; during inference of the at least one second AI model, load the at least one lightweight version file from the storage device to the memory; generate the at least one second data set according to the first data set and the at least one data patch in the memory; and execute the at least one second AI model according to the generated at least one second data set and the at least one second instruction set in the memory.

[0005] In another embodiment, the present invention provides another electronic device, comprising: a storage device; a memory; and a processing circuit coupled to the storage device and the memory and configured to execute instructions: respectively compile a first artificial intelligence (AI) model and at least one second AI model into a first compiled file and at least one second compiled file, wherein the first compiled file includes a first data set and a first instruction set, the at least one second compiled file includes at least one second data set and at least one second instruction set, and the first data set and the at least one second data set are the same; generate at least one lightweight version file for the at least one second AI model according to the at least one second compiled file, wherein the at least one lightweight version file includes the at least one second instruction set; store the first compiled file and the at least one lightweight version file in the storage device; during inference of the first AI model, load the first compiled file from the storage device to the memory; during inference of the at least one second AI model, load the at least one lightweight version file from the storage device to the memory; obtain the first data set in the memory; and execute the at least one second AI model according to the first data set and the at least one second instruction set in the memory.

[0006] In another embodiment, the present invention provides a method for processing artificial intelligence (AI) model switching of an electronic device, comprising: compiling a first AI model and at least one second AI model into a first compilation file and at least one second compilation file, respectively, wherein the first compilation file includes a first data set and a first instruction set, and the at least one second compilation file includes at least one second data set and at least one second instruction set; generating at least one lightweight version file for the at least one second AI model according to the first compilation file and the at least one second compilation file, wherein the at least one lightweight version file includes the at least one second instruction set and at least one data patch; storing the first compilation file and the at least one lightweight version file in a storage device in the electronic device; during inference of the first AI model, loading the first compilation file from the storage device into a memory of the electronic device; during inference of the at least one second AI model, loading the at least one lightweight version file from the storage device into the memory; generating the at least one second data set according to the first data set and the at least one data patch in the memory; and executing the at least one second AI model according to the generated at least one second data set and the at least one second instruction set in the memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a schematic diagram of an electronic device 10 according to an embodiment of the present invention.

[0008] Figure 2is a flow chart of a process 20 for processing artificial intelligence (AI) model switching in an electronic device according to an embodiment of the present invention.

[0009] Figure 3 is a schematic diagram of scenario 30 according to one embodiment of the present invention.

[0010] Figure 4 is a schematic diagram of scenario 40 according to one embodiment of the present invention.

[0011] Figure 5 is a schematic diagram of a scenario 50 according to one embodiment of the present invention.

[0012] Figure 6 is a schematic diagram of scenario 60 according to one embodiment of the present invention.

[0013] Figure 7 is a flowchart of a process 70 for processing AI model switching for an electronic device according to an embodiment of the present invention.

[0014] Figure 8 is a schematic diagram of scenario 80 according to one embodiment of the present invention.

[0015] Fig. 9 is a schematic diagram of scenario 90 according to one embodiment of the present invention. DETAILED DESCRIPTION

[0016] Certain terms are used in the following description and claims that refer to specific components. As will be appreciated by those skilled in the art, electronic equipment manufacturers may refer to the same component by different names. This document does not intend to distinguish between components that differ in name but function the same. In the following description and claims, the terms "include" and "comprising" are used in an open-ended manner and should be interpreted as "including, but not limited to...". In addition, the term "coupled" is intended to mean an indirect or direct electrical connection. Thus, if one device is coupled to another device, that connection may be through a direct electrical connection, or an indirect electrical connection through other devices and connections.

[0017] Figure 11 is a schematic diagram of an electronic device 10 according to an embodiment of the present invention. The electronic device 10 includes a storage device (storage) 100, a memory (memory) 110 and a processing circuit 120. The storage device 100 is configured to store data for a long time. Examples of the storage device 100 include, but are not limited to, an optical data storage device, a hard disk (e.g., a solid state disk (SSD)), a USB flash drive, a subscriber identity module (Subscriber Identity Module, SIM) card, a secure digital (Secure Digital, SD) card, etc. The memory 110 is configured to temporarily store data. Examples of the memory 110 include, but are not limited to, flash memory (e.g., cache), random access memory (Random-Access Memory, RAM) (e.g., dynamic random access memory (Dynamic RAM, DRAM), static random access memory (Static RAM, SRAM), synchronous DRAM (Synchronous DRAM, SDRAM), double data rate SDRAM (Double Data Rate SDRAM, DDR SDRAM), etc.). The processing circuit 120 is coupled to the storage device 100 and the memory 110, and is configured to execute instructions to perform various tasks (e.g., compiling programs, basic software operations, and / or complex calculations). In some embodiments of the present invention, the processing circuit 120 is configured to process the switching of the AI ​​model in the electronic device 10 (hereinafter referred to as the processes 20, 70 or their variants).

[0018] The electronic device 10 may be a user equipment (UE), a device-to-device (D2D) communication device, a peripheral component interconnect express (PCIe) device, an Internet of Things (IoT) device, a mobile phone, a laptop computer, a tablet computer, a smart watch, an e-book, a portable computer system, a vehicle, a ship or an aircraft, but is not limited thereto.

[0019] Figure 2 According to an embodiment of the present invention, an electronic device (eg, Figure 1 The electronic device 10 in the embodiment, more specifically, the processing circuit 120 in the electronic device 10) is a flowchart of a process 20 for processing an artificial intelligence (AI) model switching. As long as the results are substantially the same, there is no need to follow Figure 2 The steps are performed in the exact order shown. Process 20 includes the following steps:

[0020] Step S200: Start.

[0021] Step S202: Compile a first AI model (e.g., an original / main AI model) and at least one second AI model (e.g., at least one replacement AI model) into a first compilation file and at least one second compilation file, respectively, wherein the first compilation file includes a first data set and a first instruction set, and the at least one second compilation file includes at least one second data set and at least one second instruction set.

[0022] Step S204: Generate at least one lightweight version file for at least one second AI model according to the first compiled file and the at least one second compiled file, wherein the at least one lightweight version file includes at least one second instruction set and at least one data patch.

[0023] Step S206: Store the first compiled file and at least one lightweight version file in a storage device.

[0024] Step S208: During inference (eg, reasoning) of the first AI model, load the first compilation file from the storage device to the memory.

[0025] Step S210: During inference (eg, reasoning) of at least one second AI model, at least one lightweight version file is loaded from a storage device to a memory.

[0026] Step S212: Generate (or obtain) at least one second data set based on the first data set and at least one data patch in the memory, for example, if it is necessary to switch the first AI model to at least one second AI model.

[0027] Step S214: storing the generated at least one second data set into the memory to replace the first data set.

[0028] Step S216: Execute at least one second AI model according to at least one second data set and at least one second instruction set generated in the memory.

[0029] Step S218: End.

[0030] The processing circuit in the electronic device is configured to perform the steps of process 20. According to process 20, the processing circuit in the electronic device loads at least one data patch contained in at least one lightweight archive instead of at least one second data set. Since the size of at least one data patch is smaller than the size of at least one second data set (e.g., weight), the overhead of the electronic device switching AI models can be reduced. It should be noted that step S214 can be skipped or performed after step S216.

[0031] The implementation of process 20 is not limited to the above description. The disclosure of the following embodiments can be applied to implement process 20.

[0032] In one embodiment of the present invention, a first AI model (e.g., an original / main AI model) includes a first data set, a first size list, and a first instruction set. In one embodiment of the present invention, at least one second AI model includes at least one second data set, at least one second size list, and at least one second instruction set. Specifically, each of the at least one second AI model includes a data set, a size list, and an instruction set, and if the number of at least one second AI model exceeds one, different second AI models may include different data sets or instruction sets. If the number of at least one second AI model is one, at least one second AI model includes a data set, a size list, and an instruction set. In one embodiment of the present invention, the first instruction set and the at least one second instruction set are the same. In one embodiment of the present invention, each of the first instruction set and the at least one second instruction set is different.

[0033] In one embodiment of the present invention, the first size list indicates the size of the data in the first data set. In one embodiment of the present invention, each at least one second size list indicates the size of the data in at least one second data set. In one embodiment of the present invention, the first size list and the at least one second size list are used to compare the data in the first data set with the data in at least one second data set. In one embodiment of the present invention, the data in the first data set respectively correspond to the instructions in the first instruction set. In one embodiment of the present invention, the data in the at least one second data set respectively correspond to the instructions in the at least one second instruction set.

[0034] In one embodiment of the present invention, a processing circuit in an electronic device generates at least one data patch based on a first data set and at least one second data set during inference (e.g., reasoning) of at least one second AI model. In one embodiment of the present invention, at least one data patch includes at least one first data patch (e.g., a first binary data patch). In one embodiment of the present invention, at least one first data patch indicates at least one first difference between a first data set and at least one second data set, and is used to transform the first data set into at least one second data set. In one embodiment of the present invention, at least one data patch further includes at least one second data patch (e.g., a second binary data patch), and at least one second data patch indicates at least one second difference between at least one second data set and the first data set, and is used to transform at least one second data set into the first data set.

[0035] In one embodiment of the present invention, at least one data patch (i.e., at least one first data patch and / or at least one second data patch) does not indicate any difference between the first data set and at least one second data set when the first data set is identical to the at least one second data set.

[0036] In one embodiment of the present invention, the first data set and at least one second data set share the same memory space. That is, the memory space stores one of the first data set and at least one second data set during different runtimes of inferring the first AI model and at least one second AI model. For example, before switching the first AI model to at least one second AI model, the first data set is stored in the memory space during the first runtime (inferring the first AI model during the first runtime), and after generating at least one second data set, at least one second data set is stored in the same memory space during the second runtime to replace the first data set (inferring at least one second AI model during the second runtime). In one embodiment of the present invention, the first instruction set and at least one second instruction set are stored (or loaded) separately. For example, the first instruction set is stored in a space in the memory, and each of the at least one second instruction set is stored in a different space in the memory. In one embodiment of the present invention, the first runtime of inferring the first AI model is different from the at least one second runtime of inferring at least one second AI model.

[0037] In one embodiment of the present invention, after executing at least one second AI model, the processing circuit in the electronic device generates (or restores) a first data set based on at least one second data set and at least one second data patch, for example, when it is necessary to switch back to the first AI model from at least one second AI model. In one embodiment of the present invention, after generating a first data set based on at least one second data set and at least one second data patch, the processing circuit in the electronic device stores the first data set to the memory to replace at least one second data set; and executes the first AI model based on the first data set and the first instruction set stored in the memory. That is, when it is necessary to switch back to the first AI model from at least one second AI model, the first data set is restored and replaces at least one second data set.

[0038] Figure 3 is a schematic diagram of scenario 30 according to one embodiment of the present invention. Figure 3In , there are two models M_A and M_B. Model M_A can be regarded as the original / main AI model, while model M_B can be regarded as the replacement AI model. Model M_A includes dataset Data_A with data d1-dn, size list Size_A with sizes s1-sn and instruction set Command_A with instructions Read1(d1)-Readn(dn). Model M_B includes dataset Data_B with data d1'-dn', size list Size_B with sizes s1'-sn' and instruction set Command_B with instructions Read1(d1')-Readn(dn'). The dataset Data_A is different from the dataset Data_B, while the instruction set Command_A is the same as the instruction set Command_B. The slashed part of the dataset Data_B represents the difference between the dataset Data_A and the dataset Data_B. In Figure 3 , four memory spaces SP1 - SP4 and two operating periods RT1 and RT2 of the memory in the electronic device are shown.

[0039] First, the electronic device (for example, a processing circuit in the electronic device) compiles the models M_A and M_B into compiled archives File_A and File_B, respectively. The compiled archive File_A includes a data set Data_A and an instruction set Command_A, and the compiled archive File_B includes a data set Data_B and an instruction set Command_B. The electronic device generates data patches P(A, B) and P(B, A) based on the data set Data_A and the data set Data_B (for example, the oblique line portion of the data set Data_B). The data patch P(A, B) is a binary data patch used to convert the data set Data_A into the data set Data_B. The data patch P(B, A) is a binary data patch used to convert the data set Data_B into the data set Data_A. The electronic device generates a lightweight version archive File_LV based on the compiled archives File_A and File_B. The lightweight version archive File_LV includes the data patch P(A, B) and the instruction set Command_B. The electronic device stores the compiled file File_A, the lightweight version file File_LV and the data patch P(B, A) in a storage device (not shown) in the electronic device.

[0040] Then, during the runtime RT1, the electronic device loads the compiled file File_A from the storage device into the memory when inferring the model M_A. Specifically, the data set Data_A and the instruction set Command_A are loaded into (or stored in) the memory spaces SP1 and SP2, respectively. The electronic device optionally executes the model M_A according to the data set Data_A and the instruction set Command_A. During the runtime RT2, the electronic device loads the lightweight version file File_LV from the storage device into the memory when inferring the model M_B. Specifically, the data patch P(A, B) and the instruction set Command_B are loaded into (or stored in) the memory spaces SP3 and SP4, respectively. If it is necessary to switch the model M_A to the model M_B, the electronic device generates the data set Data_B according to the loaded data set Data_A and the data patch P(A, B). The electronic device replaces the data set Data_A in the memory space SP1 with the data set Data_B, and executes the model M_B according to the data set Data_B in the memory space SP1 and the instruction set Command_B in the memory space SP4.

[0041] In addition, as an alternative, if it is necessary to switch from model M_B back to model M_A, the electronic device loads the data patch P(B, A) from the storage device to the memory space SP5 (not shown) of the memory, and generates (or restores) the data set Data_A according to the data set Data_B in the memory space SP1 and the data patch P(B, A) in the memory space SP5. The electronic device replaces the data set Data_B in the memory space SP1 with the data set Data_A, and executes the model M_A according to the data set Data_A in the memory space SP1 and the instruction set Command_A in the memory space SP2.

[0042] Figure 4 is a schematic diagram of scenario 40 according to one embodiment of the present invention. Figure 4 There are two models, M_A and M_B. Model M_A can be considered the original / main AI model, and model M_B can be considered the replacement AI model. Models M_A and M_B can be referenced Figure 3 For the sake of brevity, the models M_A and M_B in are not described here. Figure 4 , five memory spaces SP1 - SP5 and two operating periods RT1 and RT2 in the memory of the electronic device are shown.

[0043] First, the electronic device (for example, a processing circuit in the electronic device) compiles the models M_A and M_B into compiled files File_A and File_B, respectively. The compiled file File_A includes a data set Data_A and an instruction set Command_A, and the compiled file File_B includes a data set Data_B and an instruction set Command_B. The electronic device generates data patches P(A, B) and P(B, A) based on the data set Data_A and the data set Data_B (for example, the oblique line portion of the data set Data_B). The data patch P(A, B) is a binary data patch used to change the data set Data_A to the data set Data_B. The data patch P(B, A) is a binary data patch used to change the data set Data_B to the data set Data_A. The electronic device generates a lightweight version file File_LV based on the compiled files File_A and File_B. The lightweight version file File_LV includes data patches P(A, B) and P(B, A) and an instruction set Command_B. The electronic device stores the compiled file File_A and the lightweight version file File_LV in a storage device (not shown) in the electronic device.

[0044] Then, during the runtime RT1, the electronic device loads the compiled file File_A from the storage device into the memory while inferring the model M_A. Specifically, the data set Data_A and the instruction set Command_A are loaded (or stored) into the memory spaces SP1 and SP2, respectively. The electronic device may optionally execute the model M_A according to the data set Data_A and the instruction set Command_A. During the runtime RT2, the electronic device loads the lightweight version file File_LV from the storage device into the memory while inferring the model M_B. Specifically, the data patches P(A, B) and P(B, A) and the instruction set Command_B are loaded (or stored) into the memory spaces SP3, SP4, and SP5, respectively. If it is necessary to switch the model M_A to the model M_B, the electronic device generates the data set Data_B according to the data set Data_A and the data patch P(A, B). The electronic device replaces the data set Data_A in the memory space SP1 with the data set Data_B, and executes the model M_B according to the data set Data_B in the memory space SP1 and the instruction set Command_B in the memory space SP5.

[0045] In addition, as an alternative, if it is necessary to switch back from model M_B to model M_A, the electronic device generates (or restores) data set Data_A based on data set Data_B in memory space SP1 and data patch P(B, A) in memory space SP4. The electronic device replaces data set Data_B in memory space SP1 with data set Data_A, and executes model M_A based on data set Data_A in memory space SP1 and instruction set Command_A in memory space SP2.

[0046] exist Figure 3 and Figure 4 In the embodiment, the electronic device loads at least one data patch (e.g., data patch P(A, B)) instead of a data set (e.g., data set Data_B), and the at least one data patch is contained in at least one lightweight archive. Since the size of at least one data patch is smaller than the size of the data set, the overhead of the electronic device switching AI models (e.g., switching from model M_A to model M_B) can be reduced. In addition, data sets Data_A and Data_B share the same memory space SP1 to further reduce costs. That is, if the AI ​​model needs to be switched, the data in the memory space SP1 may be replaced by new data.

[0047] Figure 5 is a schematic diagram of a scenario 50 according to one embodiment of the present invention. Figure 5In the AI ​​​​model, there are three models M_A, M_B, and M_C. Model M_A can be regarded as the original / main AI model, while models M_B and M_C can be regarded as replacement AI models. Model M_A includes a dataset Data_A containing data d1-dn, a size list Size_A containing sizes s1-sn, and an instruction set Command_A containing instructions Read1(d1)-Readn(dn). Model M_B includes a dataset Data_B containing data d1'-dn', a size list Size_B containing sizes s1'-sn', and an instruction set Command_B containing instructions Read1(d1')-Readn(dn'). Model M_C includes a dataset Data_C containing data d1"-dn", a size list Size_C containing sizes s1"-sn", and an instruction set Command_C containing instructions Read1(d1")-Readn(dn"). The data sets Data_A, Data_B, and Data_C are different, while the command sets Command_A, Command_B, and Command_C are the same. The slashed part of the data set Data_B represents the difference between the data sets Data_A and Data_B. The slashed part of the data set Data_C represents the difference between the data sets Data_A and Data_C. Figure 5 , eight memory spaces SP1 - SP8 and three operating periods RT1 - RT3 in the memory of the electronic device are shown.

[0048] First, an electronic device (e.g., a processing circuit in the electronic device) compiles models M_A, M_B, and M_C into compiled files File_A, File_B, and File_C, respectively. Compiled file File_A includes a data set Data_A and an instruction set Command_A. Compiled file File_B includes a data set Data_B and an instruction set Command_B. Compiled file File_C includes a data set Data_C and an instruction set Command_C. The electronic device generates data patches P(A, B) and P(B, A) (e.g., the slashed portion of data set Data_B) based on data sets Data_A and Data_B. Data patch P(A, B) is a binary data patch used to change data set Data_A to data set Data_B. Data patch P(B, A) is a binary data patch used to change data set Data_B to data set Data_A. The electronic device generates data patches P(A, C) and P(C, A) (e.g., the slashed portion of data set Data_C) based on data sets Data_A and Data_C. The data patch P(A,C) is a binary data patch used to change the data set Data_A to the data set Data_C. The data patch P(C,A) is a binary data patch used to change the data set Data_C to the data set Data_A.

[0049] Then, the electronic device generates a lightweight version file File_LV_B for the model M_B according to the compiled files File_A and File_B, and generates a lightweight version file File_LV_C for the model M_C according to the compiled files File_A and File_C. The lightweight version file File_LV_B includes data patches P(A, B) and P(B, A) and a command set Command_B, and the lightweight version file File_LV_C includes data patches P(A, C) and P(C, A) and a command set Command_C. The electronic device stores the compiled file File_A and the lightweight version files File_LV_B and File_LV_C in a storage device (not shown) in the electronic device.

[0050] Then, during the runtime RT1, while inferring the model M_A, the electronic device loads the compiled file File_A from the storage device into the memory. Specifically, the data set Data_A and the instruction set Command_A are loaded (or stored) into the memory spaces SP1 and SP2, respectively. The electronic device optionally executes the model M_A according to the data set Data_A and the instruction set Command_A. During the runtime RT2, while inferring the model M_B, the electronic device loads the lightweight version file File_LV_B from the storage device into the memory. Specifically, the data patches P(A, B) and P(B, A) are loaded (or stored) into the memory spaces SP3 and SP4, respectively, and the instruction set Command_B is loaded (or stored) into the memory space SP5. If it is necessary to switch the model M_A to the model M_B, the electronic device generates the data set Data_B according to the data set Data_A and the data patch P(A, B). The electronic device replaces the data set Data_A with the data set Data_B in the memory space SP1 , and executes the model M_B according to the data set Data_B in the memory space SP1 and the instruction set Command_B in the memory space SP5 .

[0051] In addition, as an alternative, if it is necessary to switch from model M_B back to model M_A or from model M_B to model M_C, the electronic device generates (or restores) data set Data_A according to data set Data_B in memory space SP1 and data patch P(B, A) in memory space SP4. The electronic device replaces data set Data_B with data set Data_A in memory space SP1, and executes model M_A according to data set Data_A in memory space SP1 and instruction set Command_A in memory space SP2.

[0052] In addition, during the running period RT3, the electronic device loads the lightweight version file File_LV_C from the storage device into the memory while inferring the model M_C. Specifically, the data patches P(A, C) and P(C, A) are loaded (or stored) into the memory spaces SP6 and SP7, respectively, and the instruction set Command_C is loaded (or stored) into the memory space SP8. If it is necessary to switch the model M_A to the model M_C, the electronic device generates the data set Data_C based on the data set Data_A and the data patch P(A, C). The electronic device replaces the data set Data_A in the memory space SP1 with the data set Data_C, and executes the model M_C according to the data set Data_C in the memory space SP1 and the instruction set Command_C in the memory space SP8.

[0053] If it is necessary to switch from model M_C back to model M_A or from model M_C to model M_B, the electronic device generates (or restores) data set Data_A according to data set Data_C in memory space SP1 and data patch P(C, A) in memory space SP7. The electronic device replaces data set Data_C with data set Data_A in memory space SP1, and executes model M_A according to data set Data_A in memory space SP1 and instruction set Command_A in memory space SP2.

[0054] Figure 6 is a schematic diagram of scenario 60 according to one embodiment of the present invention. Figure 6 In this example, model M_A can be considered the original / main AI model, while models M_B and M_C can be considered replacement AI models. Models M_A, M_B, and M_C can be referenced in Figure 5 The models M_A, M_B and M_C in , and for the sake of brevity, are not described here. Figure 6 , six memory spaces SP1 - SP6 and three operating periods RT1 - RT3 in the memory of the electronic device are shown.

[0055] Then, the electronic device generates a lightweight version file File_LV_B for the model M_B based on the compiled files File_A and File_B, and generates a lightweight version file File_LV_C for the model M_C based on the compiled files File_A and File_C. The lightweight version file File_LV_B includes a data patch P(A, B) and an instruction set Command_B, and the lightweight version file File_LV_C includes a data patch P(A, C) and an instruction set Command_C. The electronic device stores the compiled file File_A, the lightweight files File_LV_B and File_LV_C, and the data patches P(B, A) and P(C, A) in a storage device (not shown) in the electronic device. During the running period RT1, the electronic device loads the compiled file File_A from the storage device into the memory during the inference of the model M_A. Specifically, the data set Data_A and the instruction set Command_A are loaded (or stored) into the memory spaces SP1 and SP2, respectively. The electronic device optionally executes the model M_A according to the data set Data_A and the instruction set Command_A.

[0056] During the running period RT2, the electronic device loads the lightweight version file File_LV_B from the storage device into the memory while inferring the model M_B. Specifically, the data patch P(A, B) and the instruction set Command_B are loaded (or stored) into the memory spaces SP3 and SP4, respectively. If it is necessary to switch the model M_A to the model M_B, the electronic device generates the data set Data_B based on the data set Data_A and the data patch P(A, B). The electronic device replaces the data set Data_A in the memory space SP1 with the data set Data_B, and executes the model M_B according to the data set Data_B in the memory space SP1 and the instruction set Command_B in the memory space SP4.

[0057] In addition, as an alternative, if it is necessary to switch from model M_B back to model M_A or from model M_B to model M_C, the electronic device loads the data patch P(B, A) from the storage device into the memory space SP7 (not shown) in the memory, and generates (or restores) the data set Data_A according to the data set Data_B in the memory space SP1 and the data patch P(B, A) in the memory space SP7. The electronic device replaces the data set Data_B in the memory space SP1 with the data set Data_A, and executes the model M_A according to the data set Data_A in the memory space SP1 and the instruction set Command_A in the memory space SP2.

[0058] In addition, during the running period RT3, the electronic device loads the lightweight version file File_LV_C from the storage device into the memory during the inference of the model M_C. Specifically, the data patch P(A, C) and the instruction set Command_C are loaded (or stored) in the memory spaces SP5 and SP6, respectively. If it is necessary to switch the model M_A to the model M_C, the electronic device generates the data set Data_C based on the data set Data_A and the data patch P(A, C). The electronic device replaces the data set Data_A in the memory space SP1 with the data set Data_C, and executes the model M_C according to the data set Data_C in the memory space SP1 and the instruction set Command_C in the memory space SP6.

[0059] The electronic device loads the data patch P(C, A) from the storage device into the memory space SP8 (not shown) in the memory, and generates (or restores) the data set Data_A according to the data set Data_C and the data patch P(C, A) in the memory space SP1, and switches from the model M_C back to the model M_A or from the model M_C to the model M_B if necessary. The electronic device replaces the data set Data_C in the memory space SP1 with the data set Data_A, and executes the model M_A according to the data set Data_A in the memory space SP1 and the instruction set Command_A in the memory space SP2.

[0060] exist Figure 5 and Figure 6 In the embodiment, the electronic device loads at least one data patch (for example, data patch P(A,B) or P(A,B)), which is contained in at least one lightweight archive, instead of a data set (for example, data set Data_B or Data_C). Since the size of at least one data patch is smaller than the size of the data set, the overhead of the electronic device switching AI models (for example, switching from model M_A to model M_B, or from model M_A to model M_C) can be reduced. In addition, data sets Data_A, Data_B, and Data_C share the same memory space SP1 to further reduce costs. That is, if the AI ​​model needs to be switched, the data in the memory space SP1 may be replaced by new data. In the embodiment, the data set Data_A, Data_B, and Data_C share the same memory space SP1 to further reduce costs. That is, if the AI ​​model needs to be switched, the data in the memory space SP1 may be replaced by new data. Figure 5 and Figure 6 In the example, if it is necessary to switch from model M_B to model M_C, the electronic device needs to first switch from model M_B back to model M_A, and then switch from model M_A to model M_C. In other words, model M_C cannot be directly switched from model M_B. Similarly, model M_B cannot be directly switched from model M_C.

[0061] Figure 7 According to an embodiment of the present invention, an electronic device (eg, Figure 1 The electronic device 10 in the embodiment, more specifically, the processing circuit 120 in the electronic device 10) is a flowchart of a process 70 for processing AI model switching. As long as the results are basically the same, there is no need to follow Figure 7 The steps are performed in the exact order shown. Process 70 includes the following steps:

[0062] Step S700: Start.

[0063] Step S702: Compile a first AI model (e.g., an original / main AI model) and at least one second AI model (e.g., at least one replacement AI model) into a first compilation file and at least one second compilation file, respectively, wherein the first compilation file includes a first data set and a first instruction set, the at least one second compilation file includes at least one second data set and at least one second instruction set, and the first data set and the at least one second data set are the same.

[0064] Step S704: Generate at least one lightweight version file for at least one second AI model according to at least one second compiled file, wherein the at least one lightweight version file includes at least one second instruction set.

[0065] Step S706: Store the first compiled file and at least one lightweight version file in a storage device.

[0066] Step S708: During inference (eg, reasoning) of the first AI model, load the first compilation file from the storage device into the memory.

[0067] Step S710: During inference (eg, reasoning) of at least one second AI model, at least one lightweight version file is loaded from a storage device into a memory.

[0068] Step S712: Obtain (or acquire) a first data set in the memory, for example, if it is necessary to switch the first AI model to at least one second AI model.

[0069] Step S714: Execute at least one second AI model according to the first data set and at least one second instruction set in the memory.

[0070] Step S716: End.

[0071] The processing circuit is configured in the electronic device to perform the steps of process 70. According to process 70, when the first data set and the at least one second data set are the same, the processing circuit in the electronic device does not load any data patch and the at least one second data set. This can reduce the additional burden of the electronic device when switching AI models.

[0072] The implementation of process 70 is not limited to the above description. The following embodiments can be used to implement process 70.

[0073] In one embodiment of the present invention, a first AI model (e.g., an original / main AI model) includes a first data set, a first size list, and a first instruction set. In one embodiment of the present invention, at least one second AI model includes at least one second data set, at least one second size list, and at least one second instruction set. Specifically, each of the at least one second AI model includes a data set, a size list, and an instruction set, and if the number of at least one second AI model exceeds one, different second AI models may include different data sets or instruction sets. If the number of at least one second AI model is one, at least one second AI model includes a data set, a size list, and an instruction set. In one embodiment of the present invention, the first instruction set and the at least one second instruction set are the same. In one embodiment of the present invention, the first instruction set and the at least one second instruction set are different.

[0074] In one embodiment of the present invention, the first size list indicates the size of data in the first data set. In one embodiment of the present invention, the at least one second size list indicates the size of data in the at least one second data set. In one embodiment of the present invention, the first size list and the at least one second size list are used to compare data in the first data set and the at least one second data set. In one embodiment of the present invention, the first size list and the at least one second size list are the same. In one embodiment of the present invention, the data in the first data set respectively correspond to instructions in the first instruction set. In one embodiment of the present invention, the data in the at least one second data set respectively correspond to instructions in the at least one second instruction set.

[0075] In one embodiment of the present invention, since the first data set and the at least one second data set are the same, the processing circuit in the electronic device uses the first data set as the at least one second data set in the memory. In one embodiment of the present invention, the first instruction set and the at least one second instruction set are stored (or loaded) separately. For example, the first instruction set is stored in a space in the memory, and each at least one second instruction set is stored in a different space in the memory. In one embodiment of the present invention, the first runtime of the inferred first AI model is different from the at least one second runtime of the inferred at least one second AI model.

[0076] In one embodiment of the present invention, the processing circuit in the electronic device directly obtains the first data set after executing at least one second AI model, for example, if it is necessary to switch back to the first AI model from at least one second AI model. In one embodiment of the present invention, the processing circuit in the electronic device executes the first AI model according to the first data set and the first instruction set after obtaining the first data set and the first instruction set stored in the memory.

[0077] Figure 8is a schematic diagram of scenario 80 according to one embodiment of the present invention. Figure 8 In , there are two models M_A and M_B. Model M_A can be regarded as the original / main AI model, while model M_B can be regarded as the replacement AI model. Model M_A includes a dataset Data_A containing data d1-dn, a size list Size_A containing sizes s1-sn, and an instruction set Command_A containing instructions Read1(d1)-Readn(dn). Model M_B includes a dataset Data_B containing data d1-dn, a size list Size_B containing sizes s1-sn, and an instruction set Command_B containing instructions Read1'(d1)-Readn'(dn). The dataset Data_A is the same as the dataset Data_B, while the instruction set Command_A is different from the instruction set Command_B. In Figure 8 , three memory spaces SP1-SP3 and two runtimes RT1 and RT2 are shown.

[0078] First, the electronic device (e.g., a processing circuit in the electronic device) compiles the models M_A and M_B into compiled files File_A and File_B respectively. The compiled file File_A includes a data set Data_A and a command set Command_A, and the compiled file File_B includes a data set Data_B and a command set Command_B. The electronic device generates a lightweight version file File_LV according to File_B. The lightweight version file File_LV includes the command set Command_B. The electronic device stores the compiled file File_A and the lightweight version file File_LV in a storage device (not shown) in the electronic device.

[0079] Then, during the runtime RT1, the electronic device loads the compiled file File_A from the storage device into the memory while inferring the model M_A. Specifically, the data set Data_A and the instruction set Command_A are loaded (or stored) into the memory spaces SP1 and SP2, respectively. The electronic device may optionally execute the model M_A according to the data set Data_A and the instruction set Command_A. During the runtime RT2, the electronic device loads the lightweight version file File_LV from the storage device into the memory while inferring the model M_B. Specifically, the instruction set Command_B is loaded (or stored) into the memory space SP3. Since the data set Data_A and the data set Data_B are the same, if it is necessary to switch the model M_A to the model M_B, the electronic device obtains the data set Data_A in the memory space SP1 and uses it as Data_B. The electronic device executes the model M_B according to the data set Data_A in the memory space SP1 and the instruction set Command_B in the memory space SP3.

[0080] In addition, as an alternative, since the data sets Data_A and Data_B are the same, if it is necessary to switch back from model M_B to model M_A, the electronic device can directly execute model M_A according to the data set Data_A in the memory space SP1 and the instruction set Command_A in the memory space SP2 without restoring the data set Data_A.

[0081] exist Figure 8 In the present invention, since no data patch or data set is loaded into the memory when switching the AI ​​model, the extra burden of the electronic device when switching the AI ​​model (for example, switching from model M_A to model M_B) can be reduced.

[0082] Fig. 9 is a schematic diagram of scenario 90 according to one embodiment of the present invention. Fig. 9In the AI ​​​​model, there are three models M_A, M_B, and M_C. Model M_A can be regarded as the original / main AI model, while models M_B and M_C can be regarded as replacement AI models. Model M_A includes a dataset Data_A containing data d1-dn, a size list Size_A containing sizes s1-sn, and an instruction set Command_A containing instructions Read1(d1)-Readn(dn). Model M_B includes a dataset Data_B containing data d1'-dn', a size list Size_B containing sizes s1'-sn', and an instruction set Command_B containing instructions Read1(d1')-Readn(dn'). Model M_C includes a dataset Data_C containing data d1-dn, a size list Size_C containing sizes s1-sn, and an instruction set Command_C containing instructions Read1'(d1)-Readn'(dn). Data set Data_A is different from data set Data_B, while command set Command_A is the same as command set Command_B. The slashed part of data set Data_B represents the difference between data set Data_A and data set Data_B. Data set Data_A is the same as data set Data_C, while command set Command_A is different from command set Command_C. Fig. 9 , six memory spaces SP1 - SP6 in the memory of the electronic device and three operating periods RT1 - RT3 are shown.

[0083] First, the electronic device (for example, a processing circuit in the electronic device) compiles the models M_A, M_B, and M_C into compiled files File_A, File_B, and File_C, respectively. The compiled file File_A includes a data set Data_A and an instruction set Command_A. The compiled file File_B includes a data set Data_B and an instruction set Command_B. The compiled file File_C includes a data set Data_C and an instruction set Command_C. The electronic device generates data patches P(A, B) and P(B, A) based on the data set Data_A and the data set Data_B (for example, the slashed portion of the data set Data_B). The data patch P(A, B) is a binary data patch used to change the data set Data_A to the data set Data_B. The data patch P(B, A) is a binary data patch used to change the data set Data_B to the data set Data_A. The electronic device generates a lightweight file File_LV_B of the model M_B according to the compilation files File_A and File_B, and generates a lightweight file File_LV_C of the model M_C according to the compilation files File_A and File_C. The lightweight file File_LV_B includes data patches P(A, B) and P(B, A) and a command set Command_B, while the lightweight file File_LV_C includes a command set Command_C. The electronic device stores the compilation file File_A and the lightweight files File_LV_B and File_LV_C in a storage device (not shown) in the electronic device.

[0084] Then, during the runtime RT1, the electronic device loads the compiled file File_A from the storage device into the memory while inferring the model M_A. Specifically, the data set Data_A and the instruction set Command_A are loaded (or stored) into the memory spaces SP1 and SP2, respectively. The electronic device may optionally execute the model M_A according to the data set Data_A and the instruction set Command_A. During the runtime RT2, the electronic device loads the lightweight version file File_LV_B from the storage device into the memory while inferring the model M_B. Specifically, the data patches P(A, B) and P(B, A) are loaded (or stored) into the memory spaces SP3 and SP4, and the instruction set Command_B is loaded (or stored) into the memory space SP5. If it is necessary to switch the model M_A to the model M_B, the electronic device generates the data set Data_B according to the loaded data set Data_A and the data patch P(A, B). The electronic device replaces the data set Data_A in the memory space SP1 with the data set Data_B, and executes the model M_B according to the data set Data_B in the memory space SP1 and the instruction set Command_B in the memory space SP5.

[0085] In addition, as an alternative, if it is necessary to switch from model M_B back to model M_A or from model M_B to model M_C, the electronic device generates (or restores) data set Data_A according to data set Data_B in memory space SP1 and data patch P(B, A) in memory space SP4. The electronic device replaces data set Data_B in memory space SP1 with data set Data_A, and executes model M_A according to data set Data_A in memory space SP1 and instruction set Command_A in memory space SP2.

[0086] In addition, during the running period RT3, the electronic device loads the lightweight version file File_LV_C from the storage device into the memory during the inference of the model M_C. Specifically, the instruction set Command_C is loaded (or stored) into the memory space SP6. Since the data set Data_A and the data set Data_C are the same, if it is necessary to switch from the model M_A to the model M_C, the electronic device directly obtains the data set Data_A in the memory space SP1. The electronic device executes the model M_C according to the data set Data_A in the memory space SP1 and the instruction set Command_C in the memory space SP6. Since the data set Data_A and the data set Data_C are the same, if it is necessary to switch from the model M_C back to the model M_A, the electronic device directly executes the model M_A according to the data set Data_A in the memory space SP1 and the instruction set Command_A in the memory space SP2 without restoring the data set Data_A.

[0087] exist Fig. 9 In the embodiment, the electronic device loads at least one data patch (e.g., data patch P(A,B)) instead of a data set (e.g., data set Data_B), and the at least one data patch is contained in at least one lightweight archive. Since the size of at least one data patch is smaller than the size of the data set, the overhead of the electronic device switching AI models (e.g., switching from model M_A to model M_B) can be reduced. In addition, the data set Data_A and the data set Data_B share the same memory space RT1_1. That is, if the AI ​​model needs to be switched, the data in the memory space SP1 may be replaced by new data. In addition, since no data patch or data set is loaded into the memory when switching from model M_A to model M_C, the overhead of the electronic device can be further reduced.

[0088] exist Fig. 9In the example, if it is necessary to switch from model M_B to model M_C, the electronic device needs to first switch from model M_B back to model M_A, and then switch from model M_A to model M_C. In other words, model M_C cannot be directly switched from model M_B. Similarly, model M_B cannot be directly switched from model M_C.

[0089] In summary, the present invention provides an electronic communication device and a method for processing AI model switching. The electronic communication device generates a data patch representing the difference between the original / main AI model and the replacement AI model. When the AI ​​model is switched, if the replacement AI model has a different data set from the original AI model, the electronic communication device loads a data patch instead of a data set, and generates a data set for the replacement AI model based on the data patch. When the replacement AI model has a different instruction set from the original AI model, the electronic communication device does not load any data patch or data set during the AI ​​model switching. Therefore, the data / instruction set of the AI ​​model can be effectively reused, and the overhead of switching AI models of the electronic device can be reduced, making switching AI models on the electronic device more efficient. The present invention ensures that when switching AI models, the overhead is negligible and the memory usage is minimal, without the need for developers to intervene to modify the AI ​​model.

[0090] Those skilled in the art will readily appreciate that numerous modifications and variations of the apparatus and methods can be made while retaining the teachings of the present invention. Therefore, the above disclosure should be limited only by the scope and metes and bounds of the appended claims.

Claims

1. An electronic device, characterized in that: include: Storage devices; Memory; as well as A processing circuit is coupled to the storage device and the memory and configured to execute instructions: Compiling a first artificial intelligence (AI) model and at least one second AI model into a first compiled file and at least one second compiled file, respectively, wherein the first compiled file includes a first data set and a first instruction set, and the at least one second compiled file includes at least one second data set and at least one second instruction set; Generating at least one lightweight version file for the at least one second AI model according to the first compiled file and the at least one second compiled file, wherein the at least one lightweight version file includes the at least one second instruction set and at least one data patch; storing the first compiled file and the at least one lightweight version file in the storage device; During inferring the first AI model, loading the first compiled file from the storage device to the memory; During inferring the at least one second AI model, loading the at least one lightweight version file from the storage device to the memory; generating the at least one second data set based on the first data set and the at least one data patch in the memory; as well as The at least one second AI model is executed according to the generated at least one second data set and the at least one second instruction set in the memory.

2. The electronic device according to claim 1, wherein: The directive further includes: The generated at least one second data set is stored in the memory to replace the first data set.

3. The electronic device according to claim 1, wherein: The first AI model includes the first data set, the first size list and the first instruction set, and the at least one second AI model includes the at least one second data set, the at least one second size list and the at least one second instruction set.

4. The electronic device according to claim 1, wherein: The at least one data patch includes at least one first data patch.

5. The electronic device as claimed in claim 4, characterized in that: The at least one first data patch indicates at least one first difference between the first data set and the at least one second data set, and is used to transform the first data set into the at least one second data set.

6. The electronic device as claimed in claim 4, characterized in that: The at least one data patch further includes at least one second data patch indicating at least one second difference between the at least one second data set and the first data set and used to transform the at least one second data set into the first data set.

7. The electronic device according to claim 6, wherein: The directive further includes: After executing the at least one second AI model, the first data set is generated according to the at least one second data set and the at least one second data patch.

8. The electronic device as claimed in claim 7, characterized in that: The directive further includes: After generating the first data set according to the at least one second data set and the at least one second data patch, storing the first data set in the memory to replace the at least one second data set; and The first AI model is executed according to the first data set and the first instruction set stored in the memory.

9. The electronic device as claimed in claim 1, wherein: The size of the at least one data patch is smaller than the size of the at least one second data set.

10. An electronic device, characterized in that: include: Storage devices; Memory; as well as A processing circuit is coupled to the storage device and the memory and configured to execute instructions: Compiling a first artificial intelligence (AI) model and at least one second AI model into a first compilation file and at least one second compilation file respectively, wherein the first compilation file includes a first data set and a first instruction set, the at least one second compilation file includes at least one second data set and at least one second instruction set, and the first data set and the at least one second data set are the same; Generating at least one lightweight version file for the at least one second AI model according to the at least one second compiled file, wherein the at least one lightweight version file includes the at least one second instruction set; storing the first compiled file and the at least one lightweight version file in the storage device; During inferring the first AI model, loading the first compiled file from the storage device to the memory; During inferring the at least one second AI model, loading the at least one lightweight version file from the storage device to the memory; Obtaining the first data set in the memory; as well as The at least one second AI model is executed according to the first data set and the at least one second instruction set in the memory.

11. The electronic device according to claim 10, wherein: The first AI model includes the first data set, the first size list, and the first instruction set, and the at least one second AI model includes the at least one second data set, the at least one second size list, and the at least one second instruction set.

12. The electronic device according to claim 10, wherein: The directive further includes: After executing at least one second AI model, the first data set is directly obtained from the memory.

13. The electronic device according to claim 12, wherein: The directive further includes: After obtaining the first data set and the first instruction set from the memory, the first AI model is executed according to the first data set and the first instruction set.

14. A method for processing artificial intelligence (AI) model switching of an electronic device, characterized in that: include: Compiling the first AI model and the at least one second AI model into a first compilation file and at least one second compilation file respectively, wherein the first compilation file includes a first data set and a first instruction set, and the at least one second compilation file includes at least one second data set and at least one second instruction set; Generating at least one lightweight version file for the at least one second AI model according to the first compiled file and the at least one second compiled file, wherein the at least one lightweight version file includes the at least one second instruction set and at least one data patch; storing the first compiled file and the at least one lightweight version file in a storage device in the electronic device; During inferring the first AI model, loading the first compiled file from the storage device into the memory of the electronic device; During inferring the at least one second AI model, loading the at least one lightweight version file from the storage device into the memory; generating the at least one second data set according to the first data set and at least one data patch in the memory; And execute the at least one second AI model according to the generated at least one second data set and the at least one second instruction set in the memory.

15. The method according to claim 14, characterized in that The directive further includes: The generated at least one second data set is stored in the memory to replace the first data set.

16. The method according to claim 14, characterized in that The at least one data patch includes at least one first data patch.

17. The method according to claim 16, characterized in that The at least one first data patch indicates at least one first difference between the first data set and the at least one second data set, and is used to transform the first data set into the at least one second data set.

18. The method according to claim 16, characterized in that The at least one data patch further includes at least one second data patch, and the at least one second data patch indicates at least one second difference between the at least one second data set and the first data set, and is used to change the at least one second data set into the first data set.

19. The method according to claim 18, characterized in that Further including: After executing the at least one second AI model, the first data set is generated according to the at least one second data set and the at least one second data patch.

20. The method of claim 19, wherein: Further including: After generating the first data set according to the at least one second data set and the at least one second data patch, storing the first data set in the memory to replace the at least one second data set; And execute the first AI model according to the first data set and the first instruction set stored in the memory.

Images