AI partitioning method, system, device and storage medium in resource-constrained environment

Abstract

The application provides an AI partitioning method, system, device and storage medium in a resource-limited environment, the method comprising: acquiring the memory usage and CPU usage of a device in real time, and calculating a comprehensive resource score; determining the optimal partitioning size according to the calculated comprehensive resource score; and processing input data using the optimal partitioning size to achieve optimal partitioning of data. The method of the application has significant advantages in resource utilization, performance optimization and energy consumption reduction, and exhibits strong technical value and market potential.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to AI block segmentation methods, systems, devices and storage media in resource-constrained environments. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, AI applications such as Large Language Models (LLM) and Retrieval Augmentation (RAG) systems have been widely applied in various fields. However, in resource-constrained environments (such as mobile devices, edge computing devices, or IoT devices), the performance and efficiency of AI models face significant challenges. Chunking technology, as an important means of processing large datasets, is crucial in AI applications. Traditional chunking methods typically employ fixed-size chunks or sentence-based segmentation, which cannot adapt to the dynamic changes in device resource availability, leading to the following problems: Low resource utilization: Fixed block size may lead to memory overflow when resources are scarce, and may not fully utilize computing power when resources are abundant.

[0003] Performance degradation: In resource-constrained environments, excessively large blocks may cause processing delays, while excessively small blocks may increase computational overhead.

[0004] Existing technologies primarily employ chunking strategies based on tokens, sentences, or semantics; however, these methods lack adaptability in resource-constrained scenarios. Research indicates that fixed chunking strategies perform poorly on devices with limited memory and computing power. While dynamic chunking has been applied in media streaming, no patented technologies specifically targeting AI data processing have been found. Therefore, a solution capable of dynamically adjusting chunk sizes to optimize AI performance in resource-constrained environments is urgently needed. Summary of the Invention

[0005] This application provides an AI block partitioning method, system, device, and storage medium for resource-constrained environments, which addresses the performance limitations of traditional block partitioning methods in resource-constrained environments.

[0006] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0007] According to a first aspect of the embodiments of this application, an AI block partitioning method in a resource-constrained environment is provided, including: Real-time acquisition of device memory and CPU usage; Calculate the optimal block size based on the memory usage and CPU usage; Process the input data using the optimal block size to achieve optimal data blocking.

[0008] In one embodiment of this application, the device is a mobile AI device, an edge computing device, or an Internet of Things (IoT) device.

[0009] In one embodiment of this application, the comprehensive resource score calculation method is as follows: Comprehensive resource score = memory utilization rate × 0.7 + CPU utilization rate × 0.3.

[0010] In one embodiment of this application, determining the optimal block size based on the comprehensive resource score specifically includes: When the overall resource score is greater than 80%, the optimal block size is set to 100 tokens. When the overall resource score is between 50% and 80%, the optimal block size is set to 300 tokens. When the overall resource score is less than 50%, the optimal block size is set to 500 tokens.

[0011] In one embodiment of this application, the memory usage and CPU usage of the device are obtained at a preset fixed time interval.

[0012] According to a second aspect of the embodiments of this application, an AI block-segmentation system for resource-constrained environments is provided, including: The resource monitoring module is used to obtain the device's memory usage and CPU usage in real time and calculate a comprehensive resource score; The block calculation module is used to determine the optimal block size based on the comprehensive resource score. The AI ​​processing unit is used to process input data according to the optimal block size to ensure the efficient operation of the AI ​​model.

[0013] In one embodiment of this application, the comprehensive resource score in the resource monitoring module is calculated as follows: Comprehensive resource score = memory utilization rate × 0.7 + CPU utilization rate × 0.3.

[0014] In one embodiment of this application, the process of calculating the optimal block size in the block calculation module includes: When the overall resource score is greater than 80%, the optimal block size is set to 100 tokens. When the overall resource score is between 50% and 80%, the optimal block size is set to 300 tokens. When the overall resource score is less than 50%, the optimal block size is set to 500 tokens.

[0015] According to a third aspect of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in the first aspect for the AI ​​block division method under resource-constrained conditions.

[0016] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer program instructions are stored, wherein when executed by a processor, the program instructions are used to implement the process corresponding to the AI ​​block segmentation method in a resource-constrained environment described in the first aspect.

[0017] Compared with existing technologies, the beneficial effects of adopting the above technical solution are as follows: (1) Resource adaptability: It can dynamically adjust the block size according to the real-time resource status of the device to avoid memory overflow and waste of computing resources.

[0018] (2) Performance optimization: By allocating weights (memory weight 0.7, CPU weight 0.3), the impact of memory constraints on AI inference tasks is fully considered to ensure the rationality of block size.

[0019] (3) Reduced energy consumption and reduced monitoring costs while ensuring accurate assessment of resource status.

[0020] (4) Wide adaptability, applicable to various resource-constrained scenarios such as mobile AI devices, edge computing devices, and IoT devices.

[0021] This invention has significant advantages in resource utilization, performance optimization and energy consumption reduction, demonstrating strong technical value and market potential. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0023] Figure 1 This is a flowchart of an AI block-segmentation system under resource-constrained conditions, as described in an embodiment of this application.

[0024] Figure 2 This is a schematic diagram of an AI block-segmentation system under resource-constrained conditions, as described in an embodiment of this application.

[0025] Figure 3 This is a schematic diagram of an electronic device according to an embodiment of this application.

[0026] Figure 4 This is a schematic diagram of the structure of a computer system suitable for implementing the embodiments of this application.

[0027] Figure label: 200-AI block system, 201-Resource monitoring module, 202-Block calculation module, 203-AI processing unit; 300 - Electronic device; 301 - Memory; 302 - Processor; 400 - Computer system, 401 - CPU, 402 - ROM, 403 - RAM, 404 - Bus, 405 - I / O interface, 406 - Input section, 407 - Output section, 408 - Storage section, 409 - Communication section, 410 - Driver, 411 - Removable media. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0029] To address the performance limitations of traditional block-based methods in resource-constrained environments, this application proposes an AI block-based method for such environments. This method optimizes resource utilization and performance of the AI ​​model by dynamically adjusting block sizes in real-time monitoring of device resources. Please refer to [link / reference]. Figure 1 The specific plan is as follows: S101. Obtain the device's memory usage and CPU usage in real time, and calculate the overall resource score.

[0030] In this embodiment, the device's memory usage and CPU usage are obtained by using a preset fixed time interval. In one embodiment, the preset fixed time interval is set to 5 seconds.

[0031] In one embodiment, resource monitoring uses lightweight system calls to obtain memory and CPU usage, avoiding excessive consumption of device resources by the monitoring process itself. When the change in three consecutive sampling results is less than 5%, the resource status is considered stable, and the sampling interval can be appropriately extended to 10 seconds to reduce monitoring overhead. When a sudden change in resource usage is detected (the difference between memory or CPU usage in two adjacent samples exceeds 15%), a recalculation of the block size is immediately triggered without waiting for the next fixed time interval.

[0032] It should be noted that the devices involved in this embodiment refer to resource-constrained devices, such as mobile AI devices, edge computing devices, or IoT devices.

[0033] Furthermore, this embodiment also provides a specific method for calculating the comprehensive resource score, namely, constructing the comprehensive resource score with memory utilization as the primary weight and CPU utilization as the secondary weight. Specifically, the comprehensive resource score = memory utilization × 0.7 + CPU utilization × 0.3. The weight of memory utilization is set to 0.7, and the weight of CPU utilization is set to 0.3, because memory overflow has a more direct impact on AI inference tasks, while CPU load fluctuations are relatively mild. Simultaneously, boundary checks are performed on the calculated block size. When the calculation result is lower than the preset minimum block threshold (50 tokens), the minimum block threshold is used; when the calculation result is higher than the preset maximum block threshold (500 tokens), the maximum block threshold is used. S102, Determine the optimal block size based on the comprehensive resource score.

[0034] This embodiment provides a specific method for determining the optimal block size: When the overall resource score is greater than 80%, the optimal block size is set to 100 tokens. When the overall resource score is between 50% and 80%, the optimal block size is set to 300 tokens. When the overall resource score is less than 50%, the optimal block size is set to 500 tokens.

[0035] S103. Process the input data using the optimal block size to achieve optimal data block partitioning.

[0036] After obtaining the optimal block size, the input data can be processed, such as text embedding and model inference.

[0037] Specifically, when processing input data, the system maintains a block queue, sequentially dividing the input data according to the optimal block size before enqueuing it. If the overall resource score changes across a threshold during the processing of a current block (i.e., the score jumps from one level to another), subsequent data enqueuing is paused, and the remaining data is processed again with the new block size after the block size is recalculated. The results of blocks that have already been processed are unaffected, ensuring the continuity and integrity of the processing results. In addition, the system has a debouncing mechanism for adjusting the block size: the block size is only switched when the overall resource score falls into the new level twice consecutively, avoiding the additional overhead caused by frequently switching the block size when the resource score fluctuates near the threshold boundary.

[0038] Repeating steps S101 to S103 ensures that the block size dynamically adapts to resource changes.

[0039] The proposed solution in this embodiment ensures that the block size is adjusted in real time as resources change. This method is applicable to the following resource-constrained scenarios: (1) Mobile AI: such as speech recognition or text processing on smartphones and tablets. (2) Edge computing: such as data processing in smart cameras and industrial sensors. (3) Internet of Things: such as AI inference tasks in low-power devices.

[0040] Compared with the prior art, the innovation of this application is as follows: (1) It adopts a dynamic resource evaluation mechanism to monitor the memory and CPU resource status of the device in real time, and avoids the performance fluctuation caused by environmental changes due to static block method; (2) It introduces a weighted comprehensive resource scoring algorithm, with memory utilization rate as the main weight (0.7) and CPU utilization rate as the correction weight (0.3), fully considering the key impact of memory constraints on large language model inference tasks, and ensuring the effectiveness and security of the block strategy; (3) It provides three levels of dynamic block strategy (small blocks are used to reduce memory pressure when the resource score is high, medium blocks are used to balance performance when the score is medium, and large blocks are used to improve throughput when the score is low), which can adapt to various resource-constrained scenarios; (4) By adaptively adjusting the sampling interval, the overhead of resource monitoring itself is reduced, and the impact on the system is reduced while ensuring accuracy and timeliness.

[0041] Please refer to Figure 2 This application embodiment also provides an AI block segmentation system 200 for resource-constrained environments, including: a resource monitoring module 201, used to acquire the memory usage and CPU usage of the device in real time and calculate a comprehensive resource score; a block calculation module 202, used to determine the optimal block size based on the comprehensive resource score; and an AI processing unit 203, used to process input data according to the optimal block size to ensure the efficient operation of the AI ​​model.

[0042] In the resource monitoring module, the device's memory usage and CPU usage are obtained at preset fixed time intervals.

[0043] Furthermore, in the resource monitoring module, the calculation method for the comprehensive resource score is: Comprehensive resource score = memory utilization × 0.7 + CPU utilization × 0.3.

[0044] In the block calculation module, the method for determining the optimal block size is as follows: When the overall resource score is greater than 80%, the optimal block size is set to 100 tokens. When the overall resource score is between 50% and 80%, the optimal block size is set to 300 tokens. When the overall resource score is less than 50%, the optimal block size is set to 500 tokens.

[0045] The technical effects brought about by this invention include: (1) Adaptive performance: The block size is dynamically adjusted according to the real-time resource status, so that the same application can obtain the best performance on different devices and at different times, avoiding the fixed block strategy of one-size-fits-all; (2) Memory security: By setting the memory utilization weight to 0.7 (compared to 0.3 for CPU), the sensitivity of large language models to memory is fully considered, effectively reducing the risk of memory overflow and ensuring the stability of the inference process; (3) Resource efficiency: By dynamically adjusting the sampling interval (extending to 10 seconds when stable, and recalculating immediately when there is a sudden change), the resource monitoring overhead is reduced while ensuring accuracy, and the overall resource utilization rate is 15%-30% higher than that of the fixed block method; (4) Reliability: By using boundary threshold verification (50-500 token range), the block size is always within the available range, avoiding the impact of abnormal calculation results on the system; (5) Scalability: This scheme is easy to extend to other resource dimensions (such as GPU memory and network bandwidth), and supports multi-dimensional comprehensive scoring strategies.

[0046] Please refer to Figure 3 An electronic device 300 according to an embodiment of this application includes a memory 301 and a processor 302. The memory 301 stores a computer program that can be loaded by the processor 302 and executed corresponding to the aforementioned beam interval determination method for satellite-to-ground communication. It should be noted that the electronic device also has a display screen for displaying a user interface (UI). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen is a touch screen, it also has the ability to collect touch signals on or above the surface of the display screen. The touch signals can be input to the processor as control signals for processing. In this case, the display screen can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, the display screen can be a single screen, the front panel of the electronic device; in other embodiments, there can be at least two screens, respectively disposed on different surfaces of the electronic device or in a folded design; in still other embodiments, the display screen can be a flexible screen, disposed on a curved surface or folded surface of the electronic device. Furthermore, the display screen can also be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. The display screen can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0047] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0048] It should be noted that, Figure 4The computer system 400 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0049] like Figure 4 As shown, the computer system 400 includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 402 or programs loaded from storage portion 408 into Random Access Memory (RAM) 403, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 403. The CPU 401, ROM 402, and RAM 403 are interconnected via bus 404. An Input / Output (I / O) interface 405 is also connected to bus 404.

[0050] The following components are connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to I / O interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0051] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs various functions defined in the system of this application.

[0052] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0053] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0054] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0055] In another aspect, this application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the AI ​​block-segmentation method under resource-constrained conditions described in the above embodiments.

[0056] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the AI ​​block-segmentation method under resource-constrained conditions described in the above embodiments.

[0057] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0058] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0059] For those skilled in the art, the specific meanings of the above terms in this invention can be understood according to the specific circumstances; the accompanying drawings in the embodiments are used to clearly and completely describe the technical solutions in the embodiments of this invention. Obviously, the described embodiments are some embodiments of this invention, but not all embodiments. Generally, the components of the embodiments of this invention described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0060] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. An AI block-segmentation method for resource-constrained environments, characterized in that, include: Real-time acquisition of device memory and CPU usage, and calculation of comprehensive resource score; The optimal block size is determined based on the comprehensive resource score. Process the input data using the optimal block size to achieve optimal data blocking.

2. The AI ​​block partitioning method under resource-constrained environments according to claim 1, characterized in that, The device is a mobile AI device, an edge computing device, or an IoT device.

3. The AI ​​block partitioning method under resource-constrained environments according to claim 1 or 2, characterized in that, The comprehensive resource score calculation method is as follows: Comprehensive resource score = memory utilization rate × 0.7 + CPU utilization rate × 0.

3.

4. The AI ​​block partitioning method under resource-constrained environments according to claim 3, characterized in that, The process of determining the optimal block size based on the comprehensive resource score specifically includes: When the overall resource score is greater than 80%, the optimal block size is set to 100 tokens. When the overall resource score is between 50% and 80%, the optimal block size is set to 300 tokens. When the overall resource score is less than 50%, the optimal block size is set to 500 tokens.

5. The AI ​​block partitioning method under resource-constrained environments according to claim 1, characterized in that, The device's memory usage and CPU usage are obtained at preset fixed time intervals.

6. An AI block-segmentation system for resource-constrained environments, characterized in that, include: The resource monitoring module is used to obtain the device's memory usage and CPU usage in real time and calculate a comprehensive resource score; The block calculation module is used to determine the optimal block size based on the comprehensive resource score. The AI ​​processing unit is used to process input data according to the optimal block size to ensure the efficient operation of the AI ​​model.

7. The AI ​​block-segmentation system under resource-constrained environments according to claim 6, characterized in that, In the resource monitoring module, the comprehensive resource score is calculated as follows: Comprehensive resource score = memory utilization rate × 0.7 + CPU utilization rate × 0.

3.

8. The AI ​​block-segmentation system under resource-constrained environments according to claim 6 or 7, characterized in that, In the block calculation module, the process of determining the optimal block size includes: When the overall resource score is greater than 80%, the optimal block size is set to 100 tokens. When the overall resource score is between 50% and 80%, the optimal block size is set to 300 tokens. When the overall resource score is less than 50%, the optimal block size is set to 500 tokens.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed corresponding to the AI ​​block division method in a resource-constrained environment as described in any one of claims 1 to 8.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they are used to implement the process corresponding to the AI ​​block division method in a resource-constrained environment as described in any one of claims 1 to 8.

Images