Optimization method for loading AI model on low-power-consumption electronic product

Abstract

The invention provides an optimization method for loading an AI model on a low-power-consumption electronic product, which utilizes the characteristics that a DDR (Double Data Rate) does not power down and data is reserved after low-power-consumption equipment is dormant, and directly addresses and accesses data on the DDR after the equipment is started, and comprises the following steps: S1, starting the equipment; s2, starting the system; s3, judging whether starting is carried out for the first time or not; if so, performing S4; if yes, performing S5; s4, loading the AI model to the DDR, wherein the process needs to consume time; then continuing to carry out the step S6; s5, using a zboost function, wherein the process does not need to consume time; according to the method, a zboost function is designed on hardware, after the function is started, a CPU is in a power-off state when equipment is in a dormant state, namely a standby state, but a DDR is not powered off, and data generated when the equipment is started and runs last time is reserved on the DDR; after the CPU is started in the next starting process, the corresponding program can be executed by directly addressing the access data at the address corresponding to the DDR; s6, executing an AI algorithm model on the DDR; and S7, finishing execution, and continuing to wait for next start.

Description

Technical Field

[0001] The present invention belongs to the technical field of AI for low-power electronic products, and in particular relates to an optimization method for loading an AI model for a low-power electronic product. Background Art

[0002] With the current development of science and technology, various terminal products have higher and higher requirements for AI performance, and AI demand has become a current trend. Taking low-power AI electronic products as an example, due to market demand, users have higher and higher requirements for algorithm accuracy and variety. AI model files are naturally large. In the actual use of electronic products, the download and decompression time of Flash algorithm model data is long, so the algorithm intervention time is also long, resulting in a poor product experience.

[0003] In order to achieve fast loading of AI models, existing solutions generally optimize from two aspects: the selection of storage materials and the method of loading software models.

[0004] In terms of material selection, NOR type flash with fast reading speed is generally selected. The advantage is that its reading speed is relatively fast among non-volatile storage media, which is suitable for such products. Its disadvantages are small capacity and high price.

[0005] In terms of software loading, it is not recommended to compress the model file. It is a binary file that has been processed and has basically no redundant data. Even if the highest compression format is used, the size of the compressed data will basically not change. In addition, it is generally not recommended to use the file system for loading model files. The existence of the file system is to facilitate users to operate and manage files. However, for AI model files, they are generally composed of one or a few files, and there is no need for a file system to manage them. The most important thing is that once the file system is involved, due to the complex management mechanism of the file system and the compression mechanism that may exist in different file systems, it will take a long time to mount the file system containing the AI ​​model after the root file system is up. The different file system types will also lead to different time consumption. For the reading of such binary files with basically no redundant data, the MTD raw partition reading method is generally adopted to avoid the time consumption caused by the intervention of the file system.

[0006] The above operations are to optimize the loading of AI models in terms of solutions and software. However, in actual products, due to the uncertainty of model size caused by different businesses and the bottleneck limitation of nor flash reading data, the time consumption when the model file is large is generally at the second level, which has a great impact on the user experience of the product.

[0007] For other low-power products, the above operations can basically meet certain market demands, but they still do not have an advantage in the high-end market. The most time-consuming problem of the existing technology lies in the loading process of model data. Loading model data from the storage device to DDR for execution will be limited by the maximum read speed of the storage device flash no matter how the software is optimized. Even if the model file is loaded at the maximum read speed, it still takes a certain amount of time to complete the operation. The specific process is as follows Figure 1 As shown:

[0008] The device starts, the system starts, the AI ​​model is loaded into DDR (time-consuming), the AI ​​algorithm is executed on DDR, and the execution ends. Wait for the next startup.

[0009] In addition, the commonly used terms in the prior art include:

[0010] Flash: A type of memory device, it is a non-volatile memory. It is also called non-volatile random access memory, which is characterized by the fact that data does not disappear after power failure. It is mainly divided into NOR type and NAND type. Different storage media have different data reading rates. Generally speaking, compared with NAND flash, NOR flash has a smaller capacity, generally 1-16MByte. In terms of price, NOR is higher than NAND. In terms of read operation, NOR is faster than NAND. Taking comprehensive considerations to improve product performance, NOR flash is generally selected.

[0011] Cold start: Just like a computer can only work after being connected to the power supply, the same is true for any electronic device. From the time the device is connected to the power supply to the time it is ready to work, a series of initialization operations must be performed. This process is called a cold start.

[0012] MTD subsystem: (Memory Technology Device) is commonly referred to as Flash and other storage devices that use memory chips. The MTD subsystem corresponds to the device driver layer in the block device driver framework. It can be said that MTD is a standardized hardware driver framework designed for Flash devices. Using the MTD subsystem to write flash device drivers is very fast.

[0013] Embedded Linux system startup process: After power-on, SOC executes the solidified bootrom code, loads the bootloader segment from the flash to the RAM for operation, and then loads the kernel to the RAM for operation in the bootloader stage. The process is the same for NOR and NAND. The battery solution generally chooses NOR flash because the reading speed of NOR flash is faster than that of NAND flash, which can achieve faster startup time. Summary of the invention

[0014] In order to solve the above problems, the purpose of this application is to completely solve the problems in the prior art by designing a zboost function.

[0015] Specifically, the present invention provides an optimization method for loading an AI model on a low-power electronic product. The method utilizes the characteristics that DDR does not lose power and data is retained after the low-power device goes into sleep mode, and directly addresses and accesses data on DDR after the device starts up, and includes the following steps:

[0016] S1, equipment startup;

[0017] S2, system startup;

[0018] S3, determine whether it is the first start? If yes, proceed to step S4; if no, proceed to step S5;

[0019] The determination further includes: writing a flag bit on the storage medium DDR, and the CPU will read the flag bit during the chip startup phase, if the flag bit is enabled, it means it is not the first startup, if it is not enabled, it means it is the first startup;

[0020] S4, load AI model to DDR, this process is time-consuming; then proceed to step S6; S5, use zboost function, this process is not time-consuming, that is, a function named zboost is designed in the hardware. After this function is turned on, the CPU of the device is in a power-off state in the sleep state, that is, in the standby state, but the DDR is not powered off. The DDR retains the data from the last device startup. After the CPU is started during this startup process, the corresponding program can be executed by directly addressing and accessing the data at the address corresponding to the DDR;

[0021] S6, executes AI algorithm model on DDR;

[0022] S7, execution ends, and continues to wait for the next startup.

[0023] In step S4, when the device is cold-started for the first time, the CPU will load the model file from the Flash memory to the DDR through the Flash controller.

[0024] In step S5, the zboost function further comprises:

[0025] The zboost technology hardware is designed with LP-DDR. DDR itself does not lose power and maintains self-refresh. When the device loses power, the SOC is in a power-off state, but the DDR is not powered off. At this time, the data on the DDR still exists and is in a self-refresh state. When the device is started again, there is no need for the traditional method of reloading data to the DDR. The CPU can directly jump to the corresponding address of the DDR to continue executing instructions.

[0026] The step S5 further comprises:

[0027] After the first startup, the data on the DDR will only be updated with the user program. When the business execution after the startup is completed, the CPU will power off and the device will enter the sleep state, but the DDR is in the self-refresh state without power failure. When the next event wakes up the device, the CPU will power on and directly address and execute on the DDR.

[0028] Under the zboost design, when it is found that the current boot is not the first boot, the CPU will return to the state before the hibernation and continue the instructions before the hibernation. Because the DDR is not powered off, the data on the DDR is always there, and the CPU can run normally.

[0029] The same is true for AI model file data, that is, the CPU only needs to execute it at the specified location of DDR. For developers, there is a file system on the operating system to assist developers in managing files. When developing this function, the model data is placed in a specified path directory of the file system. When the file needs to be used, just go to the directory to read the file, thus omitting the step of loading the corresponding model from the flash memory to the DDR after the conventional electronic product is cold started.

[0030] The updating of the above data with the user program includes the operation of updating the AI ​​model file.

[0031] The method is applicable to a chip that uses a DDR that supports self-refresh without power failure in hardware design.

[0032] The method is applied to the embedded startup low-power device quick start product industry.

[0033] Therefore, the advantage of this application is that no matter what operating system, the algorithm model needs to be loaded to execute the AI ​​algorithm. The operation of loading the model into the DDR is necessarily time-consuming. This technology solves the bottleneck that the AI ​​algorithm cannot be executed quickly due to the loading of the model every time the device starts. After using this technology, the system can obtain the model data on the DDR at the first time every time it is not started for the first time, so that the corresponding model data can be obtained on the DDR at the first time of startup, thereby quickly executing the AI ​​algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0035] Figure 1 It is a schematic diagram of the startup process of low-power electronic products in the prior art.

[0036] Figure 2 It is a schematic diagram of the startup process in this application. DETAILED DESCRIPTION

[0037] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0038] This application proposes an optimization method for loading AI models on low-power electronic products, which belongs to IPC security and consumer electronic products.

[0039] This embodiment uses the T41 chip platform of Beijing Ingenic Integrated Circuit Co., Ltd. (hereinafter referred to as Ingenic), which has a zboost function designed in hardware. After this function is turned on, the CPU of the device is in a power-off state in the sleep state (standby state), but the DDR is not powered off, and the data from the last time the device was started is retained on the DDR. After the CPU is started during the next startup process, the corresponding program can be executed by directly addressing and accessing the data at the address corresponding to the DDR.

[0040] When the device is cold started for the first time, the CPU will load the model file from the flash memory to the DDR through the flash controller. After that, the data on the DDR will only be updated with the user program, such as updating the model file. When the business execution after this startup is completed, the CPU will power off the device and enter a sleep state, but the DDR is in a self-refresh state without power failure. When the next event wakes up the device, the CPU will power on and directly address and execute on the DDR. The same is true for the AI ​​model file data. The CPU only needs to execute at the specified location in the DDR. This completely avoids the process of loading the corresponding model from the flash memory to the DDR after a conventional product is cold started. The model read execution time is optimized to the extreme, achieving extremely fast AI. The specific process is as follows: Figure 2 As shown, the method utilizes the characteristics that DDR does not lose power and retains data after the low-power device is dormant, and directly addresses and accesses data on DDR after the device is started, including the following steps:

[0041] S1, equipment startup;

[0042] S2, system startup;

[0043] S3, determine whether it is the first start? If yes, proceed to step S4; if no, proceed to step S5;

[0044] The determination further includes: writing a flag bit on the storage medium DDR, and the CPU will read the flag bit during the chip startup phase, if the flag bit is enabled, it means it is not the first startup, if it is not enabled, it means it is the first startup;

[0045] S4, load the AI ​​model to DDR. This process is time-consuming. Then proceed to step S6. When the device is cold-started for the first time, the CPU will load the model file from the Flash memory to the DDR through the Flash controller.

[0046] S5, use the zboost function. This process does not require time. That is, a function named zboost is designed in the hardware. After this function is turned on, the CPU of the device is in a power-off state in the sleep state, that is, in the standby state, but the DDR is not powered off. The DDR retains the data from the last device startup. After the CPU is started during this startup process, the corresponding program can be executed by directly addressing and accessing the data at the address corresponding to the DDR.

[0047] S6, executes AI algorithm model on DDR;

[0048] S7, execution ends, and continues to wait for the next startup.

[0049] In the prior art, in normal device startup, since DDR is a volatile medium, the data on the medium will be lost when the power is off. Therefore, when the device is restarted, the data in the flash storage medium needs to be reloaded to the DDR and the CPU will jump to execute the relevant generation instructions. This loading process is generally time-consuming.

[0050] Therefore, in step S5 of the method, the zboost function further includes:

[0051] The zboost technology hardware is designed with LP-DDR. DDR itself does not lose power and maintains self-refresh. When the device loses power, the SOC is in a power-off state, but the DDR is not powered off. At this time, the data on the DDR still exists and is in a self-refresh state. When the device is started again, there is no need for the traditional method of reloading data to the DDR. The CPU can directly jump to the corresponding address of the DDR to continue executing instructions.

[0052] Normal process in the prior art: After the device is powered on, the SOC boots and loads the kernel in the bootloader stage, loads the kernel firmware from the storage medium flash to the DDR, and after loading is completed, jumps to the kernel start address on the DDR to execute the kernel start code;

[0053] The step S5 further includes that after the first startup is completed, the data on the DDR will only be updated with the user program. When the service execution after the startup is completed, the CPU will be powered off and the device will enter a dormant state, but the DDR is in a self-refresh state without power failure; when the next event wakes up the device, the CPU will be powered on and directly address the DDR for execution;

[0054] Under the zboost design, when it is found that the current boot is not the first boot, the CPU will return to the state before the hibernation and continue the instructions before the hibernation. Because the DDR is not powered off, the data on the DDR is always there, and the CPU can run normally.

[0055] The same is true for AI model file data, that is, the CPU only needs to execute at the specified location of DDR. For developers, the operating system has a file system to assist developers in managing files. When developing this function, the model data is placed in a specified path directory of the file system. When the file needs to be used, it can be read from the directory. In the Linux system, the file can be fopened in the path, where fopen is the operation of opening the file handle in Linux, thus omitting the step of loading the corresponding model from the flash memory to the DDR after the conventional electronic products are cold-started. For AI model data, there is a file system on the operating system to assist us users in development and use, but it is essentially a bunch of binary numbers stored on the DDR, so as long as the DDR is not powered off, the data will always exist.

[0056] Therefore, updating the above data with the user program includes the operation of updating the AI ​​model file.

[0057] The method is applied to embedded startup low-power device quick-start products.

[0058] In particular, the method is applicable to the Ingenic T41 chip platform, that is, the zboost function is applicable to the Ingenic T41 platform. If the hardware design of other platforms uses DDR that supports self-refresh without power failure, it can also be supported.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An optimization method for loading an AI model on a low-power electronic product, characterized in that: The method utilizes the characteristics that DDR does not lose power and retains data after the low-power device goes into sleep mode, and directly addresses and accesses data on DDR after the device starts up, and includes the following steps: S1, equipment startup; S2, system startup; S3, determine whether it is the first start? If yes, proceed to step S4; if no, proceed to step S5; The determination further includes: writing a flag bit on the storage medium DDR, and the CPU will read the flag bit during the chip startup phase, if the flag bit is enabled, it means it is not the first startup, if it is not enabled, it means it is the first startup; S4, load the AI ​​model to DDR, this process is time-consuming; then proceed to step S6; S5, use the zboost function. This process does not require time. That is, a function named zboost is designed in the hardware. After this function is turned on, the CPU of the device is in a power-off state in the sleep state, that is, in the standby state, but the DDR is not powered off. The DDR retains the data from the last device startup. After the CPU is started during this startup process, the corresponding program can be executed by directly addressing and accessing the data at the address corresponding to the DDR. S6, executes AI algorithm model on DDR; S7, execution ends, and continues to wait for the next startup.

2. According to claim 1, a method for optimizing a low-power electronic product loaded with an AI model is characterized in that: In step S4, when the device is cold-started for the first time, the CPU will load the model file from the Flash memory to the DDR through the Flash controller.

3. According to claim 1, a method for optimizing a low-power electronic product loaded with an AI model is characterized in that: In step S5, the zboost function further comprises: The zboost technology hardware is designed with LP-DDR. DDR itself does not lose power and maintains self-refresh. When the device loses power, the SOC is in a power-off state, but the DDR is not powered off. At this time, the data on the DDR still exists and is in a self-refresh state. When the device is started again, there is no need for the traditional method of reloading data to the DDR. The CPU can directly jump to the corresponding address of the DDR to continue executing instructions.

4. The optimization method for loading an AI model on a low-power electronic product according to claim 1, characterized in that: The step S5 further comprises: After the first startup, the data on the DDR will only be updated with the user program. When the business execution after the startup is completed, the CPU will power off and the device will enter the sleep state, but the DDR is in the self-refresh state without power failure. When the next event wakes up the device, the CPU will power on and directly address and execute on the DDR. Under the zboost design, when it is found that the current boot is not the first boot, the CPU will return to the state before the hibernation and continue the instructions before the hibernation. Because the DDR is not powered off, the data on the DDR is always there, and the CPU can run normally. The same is true for AI model file data, that is, the CPU only needs to execute it at the specified location of DDR. For developers, there is a file system on the operating system to assist developers in managing files. When developing this function, the model data is placed in a specified path directory of the file system. When the file needs to be used, just go to the directory to read the file, thus omitting the step of loading the corresponding model from the flash memory to the DDR after the conventional electronic product is cold started.

5. The optimization method for loading an AI model on a low-power electronic product according to claim 4, characterized in that: The updating of the above data with the user program includes the operation of updating the AI ​​model file.

6. The optimization method for loading an AI model on a low-power electronic product according to claim 1, characterized in that: The method is applicable to a chip that uses a DDR that supports self-refresh without power failure in hardware design.

7. The optimization method for loading an AI model on a low-power electronic product according to claim 1, characterized in that: The method is applied to the embedded startup low-power device quick start product industry.

Images