Multi-modal large model caching system and method for reducing computing power loss

Abstract

The invention provides a multi-mode large model caching system and method capable of reducing calculation power loss, and belongs to the technical field of artificial intelligence. Cue words are preprocessed, original cue words are converted into fixed formats, results and the converted cue words are cached, and if the same cue words exist next time, the cue words are cached. Therefore, the consumption of computing power is greatly reduced, the request response is accelerated, and a series of objectives of reducing the operation cost, reducing the energy consumption, promoting the model popularization, improving the user experience and the like are realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a multi-modal large model caching system and method for reducing computing power loss. Background Art

[0002] With the rapid development of artificial intelligence technology, large models have been widely used in natural language processing, image recognition, recommendation systems and other fields. Large models usually refer to deep learning models with hundreds of millions or even billions of parameters, such as OpenAI's GPT-4 and Google's Transformer. These models can handle complex tasks and provide high-quality prediction results, but they also require huge computing resources.

[0003] In the reasoning application of large models, every time a user asks a query or enters a prompt, the model needs to go through a complete reasoning process, including data processing, model reasoning, and result generation. This process consumes a lot of computing resources, and the cost of each reasoning may be very high, especially in multimodal scenarios such as images, videos, and audios. The consumption of computing power, time, and energy is even greater. This repeated calculation not only wastes computing power, but also increases operating costs, limiting the widespread application of the model.

[0004] To solve this problem, researchers have proposed various methods to optimize the model's reasoning process. One common method is model pruning, which reduces the computational complexity by reducing the number of parameters in the model. Another method is knowledge distillation, which transfers the knowledge of a large model to a small model, thereby reducing resource consumption during reasoning. In addition, some work has attempted to accelerate the model's reasoning process through hardware optimization, such as using dedicated artificial intelligence chips. However, the above methods all have their limitations. Model pruning and knowledge distillation may lead to a decline in model performance, while hardware optimization requires additional investment and a complex deployment process. Therefore, it is necessary to explore a new method that can effectively reduce the computing power loss and operating costs of large models without sacrificing model performance. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a multi-modal large model caching system that reduces computing power loss.

[0006] The technical solution of the present invention is:

[0007] A multi-modal large model caching system for reducing computing power loss, comprising:

[0008] A prompt formatting module is used to receive and pre-process the original prompt input by the user and convert it into a standardized new prompt word;

[0009] The cache storage module is used to store the formatted prompts and their corresponding multi-modal generation results, and provide external search capabilities;

[0010] The cache query module is used to search for new prompt words before each reasoning. If the corresponding multimodal reasoning result is found, it will be returned directly, otherwise it will enter the model reasoning process;

[0011] The cache evaluation module is used to evaluate the data in the cache and provide a basis for the cache update module to make judgments;

[0012] The cache update module is used to update or delete the data in the cache according to the evaluation results of the cache evaluation module.

[0013] Furthermore,

[0014] The prompt formatting module converts the original prompt into a standardized format through predefined rules or algorithms, ensuring that prompts in different forms can be mapped to the same identifier after processing.

[0015] The prompt formatting module further includes functions for removing irrelevant characters, unifying upper and lower cases, word segmentation, extracting keywords, and vectorizing and hashing.

[0016] Furthermore,

[0017] The cache storage module uses a key-value pair storage system, where the key is the new prompt word and the value corresponds to the multimodal reasoning result.

[0018] Furthermore,

[0019] The evaluation criteria of the cache evaluation module include data access frequency, freshness, accuracy, hit rate, and user satisfaction. The cache update module updates or deletes the data in the cache according to the evaluation results of the cache evaluation module.

[0020] In addition, the present invention also provides a multimodal large model caching method for reducing computing power loss, by analyzing the input original prompt words and summarizing them into new prompt words of set length and fixed format, and then binding the generated multimodal results with the converted new prompt words and storing them in a storage medium. In subsequent reasoning, the new prompt words will also be converted before each reasoning, and then searched in the storage medium. If the search is found, the generated multimodal results can be directly returned. If not found, reasoning generation is performed and the generated results are placed in the storage medium as a subsequent cache.

[0021] Furthermore,

[0022] The original prompt is converted into a standardized format through predefined rules or algorithms to ensure that different forms of prompts can be mapped to the same identifier after processing. The process of converting the original prompt includes removing irrelevant characters, unifying upper and lower cases, word segmentation, extracting keywords, vectorization representation and hash coding.

[0023] Furthermore,

[0024] A key-value pair storage system is used, where the key is the new prompt word and the value corresponds to the multimodal reasoning result.

[0025] Furthermore,

[0026] Among them, the evaluation basis of the cache evaluation module includes data access frequency, freshness, accuracy, hit rate, and user satisfaction; according to the evaluation results of the cache evaluation module, the data in the cache is updated or deleted.

[0027] The beneficial effects of the present invention are

[0028] The present invention can not only significantly reduce computing power loss and improve data processing efficiency, but also dynamically adjust cache strategies according to real-time data and user needs, ensuring data accessibility and maximizing user experience. This innovative cache method and system has broad application prospects in the field of multimodal data processing and can bring significant economic and social benefits to related industries. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] The present invention analyzes the complex input words and summarizes them into input words of a certain length and fixed format, then binds the generated multimodal results with the converted input words and stores them in a storage medium. In subsequent reasoning, the input words will be converted before each reasoning, and then searched in the storage medium. If the search is found, the generated multimodal results can be directly returned. If not found, reasoning is performed and the generated results are placed in the storage medium as a cache. The implementation scheme of the system includes the following key components:

[0031] It consists of five parts: prompt formatting module, cache storage module, cache query module, cache evaluation module and cache update module.

[0032] 1. Prompt formatting module:

[0033] The core function of this module is to deeply understand and convert the original prompts input by the user. The original prompts may contain multiple forms and expressions. In order to ensure that they can be correctly identified and retrieved in the subsequent cache matching process, this module converts the original prompts into a standardized format through predefined rules or algorithms to ensure that different forms of prompts can be mapped to the same identifier after processing, while maintaining sufficient differentiation to avoid mismatching.

[0034] During the formatting process, irrelevant characters such as punctuation marks and special symbols are first removed, and then the upper and lower case characters are unified to ensure the uniformity of the prompts. Next, word segmentation is performed to break down the prompts into words or phrases to better understand their meanings. Keywords are then extracted to represent the core content of the prompts. A natural language big model can also be used. Through the capabilities of the big model, a suitable general prompt can be written to convert the input prompts according to the format.

[0035] In order to build an efficient and low-redundancy cache key-value system, the module may also use technical means such as vectorized representation and hash coding. Vectorized representation converts prompts into numerical vectors, thereby reducing the need for storage space; hash coding maps prompts into hash values ​​to improve cache query efficiency.

[0036] Different prompt formats can also be set for different multimodal scenarios. For example, the prompts for picture scenes can include fixed elements such as "painting style", "tone", "background", and "perspective". The prompts for audio scenes can include fixed elements such as "music style", "rhythm", "instrument", and "voice". The prompts for video scenes can include fixed elements such as "role", "object", "place", and "action". Such a design can make the final output formatted prompts more refined and improve the effect of the final generated results.

[0037] 2. Cache storage module:

[0038] The cache storage module is responsible for storing the formatted prompts and their corresponding multimodal generation results, and provides high-speed search capabilities. This module uses efficient data structures such as key-value storage systems, where the key is the new prompt and the value is the corresponding multimodal reasoning result. This storage method can quickly perform read and write operations and reduce access delays.

[0039] In order to adapt to different performance and capacity requirements, the storage medium can be a solid-state drive (SSD), a distributed file system, or an in-memory database. SSDs have high read and write speeds and stable performance, making them suitable as cache storage media; distributed file systems can provide high availability and scalability; in-memory databases can achieve extremely low access latency; object storage is low-cost and is a natural key-value storage method. However, it should be noted that if the storage used does not support key-value pair queries, a key-value database will need to be added for metadata management, because in this application, fast key-value pair queries and searches are required.

[0040] When designing a cache storage structure, you need to consider storage capacity limitations. In order to effectively utilize limited storage space, you can use cache replacement strategies such as LFU (Least Frequently Used), LRU (Least Recently Used) or other more advanced elimination algorithms. These algorithms can determine which data should be retained and which data should be replaced based on the access frequency and freshness of the data.

[0041] 3. Cache query module:

[0042] The cache query module is the module that controls the entire inference cache process. Before each inference, the prompt formatting module will be called to format the input prompt to obtain a new prompt word. Then, the cache storage module is called to search for the new prompt word. If the search is found, the corresponding multimodal reasoning result is directly returned, thus avoiding repeated model reasoning processes and saving computing resources; if the search is not found, the reasoning process is entered. After the reasoning is completed, the cache query module will call the cache storage module to store the generated result in the cache storage.

[0043] 4. Cache evaluation module:

[0044] This module is responsible for evaluating the data in the cache and providing a basis for the cache update module to determine which data needs to be retained, updated or deleted. The final output evaluation basis can include data access frequency, freshness, accuracy, hit rate, user satisfaction, etc.

[0045] In order to evaluate the effectiveness and efficiency of the cache, the cache evaluation module also includes a cache hit rate monitoring and analysis component, which regularly or dynamically evaluates the effectiveness of the cache system to facilitate the adjustment of cache strategies, such as cache size, expiration strategy, and formatting algorithm.

[0046] The cache evaluation module regularly evaluates the effectiveness and accuracy of cached results. Due to the continuous update and iteration of large models, some cached results may become outdated. This module uses preset evaluation criteria, such as user feedback, model version updates, etc., to help the cache update module update or clear old data in the cache.

[0047] 5. Cache update module:

[0048] Cache storage takes up a lot of storage space, which brings cost pressure. As the cache size increases, large-scale storage may reduce the fast query effect of the cache. Therefore, a cache update module is needed to update and delete the cache in the cache storage module.

[0049] The cache update module needs to update or delete the data in the cache based on the evaluation results of the cache evaluation module. This module also needs to deal with issues such as cache aging and data updates to ensure that the cache content remains up-to-date and valid over time. When the cache evaluation module determines that the cache results need to be updated, the cache update module will delete the previous cache. When the next query comes, new multimodal results will be generated and these results will replace the old results.

[0050] When a user asks a query or enters a prompt, the prompt is first preprocessed by the prompt formatting module to obtain a new prompt. Then, the new prompt is searched in the cache query module. If the corresponding result is found, it is directly returned to the user; if not, the reasoning process is entered, multimodal reasoning is performed through the large model to generate a new result, and the cache storage module will cache the result together with the processed prompt. The cache evaluation module will comprehensively evaluate the cache based on multiple dimensions. The cache update module will use these evaluations to update and delete the existing cache.

[0051] The five modules of the present invention cooperate with each other to form a closed-loop control system. By flexibly and efficiently managing the entire life cycle of large model cache data, the overall computing cost is reduced while ensuring that data accessibility and user experience are maximized.

[0052] The above description is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A multi-modal large model caching system that reduces computing power loss, characterized in that: include: A prompt formatting module is used to receive and pre-process the original prompt input by the user and convert it into a standardized new prompt word; The cache storage module is used to store the formatted prompts and their corresponding multi-modal generation results, and provide external search capabilities; The cache query module is used to search for new prompt words before each reasoning. If the corresponding multimodal reasoning result is found, it will be returned directly, otherwise it will enter the model reasoning process; The cache evaluation module is used to evaluate the data in the cache and provide a basis for the cache update module to make judgments; The cache update module is used to update or delete the data in the cache according to the evaluation results of the cache evaluation module.

2. The system according to claim 1, characterized in that The prompt formatting module converts the original prompt into a standardized format through predefined rules or algorithms, ensuring that prompts in different forms can be mapped to the same identifier after processing; The prompt formatting module further includes functions for removing irrelevant characters, unifying upper and lower cases, word segmentation, extracting keywords, and vectorizing and hashing.

3. The system according to claim 1, characterized in that The cache storage module uses a key-value pair storage system, where the key is the new prompt word and the value corresponds to the multimodal reasoning result.

4. The system according to claim 1, characterized in that in, The evaluation criteria of the cache evaluation module include data access frequency, freshness, accuracy, hit rate, and user satisfaction.

5. The system according to claim 1, characterized in that in, The cache update module updates or deletes the data in the cache according to the evaluation result of the cache evaluation module.

6. A multi-modal large model caching method for reducing computing power loss, characterized in that: The original prompt words are analyzed and summarized into new prompt words of set length and fixed format, and then the generated multimodal results are bound to the converted new prompt words and stored in the storage medium. In subsequent reasoning, the new prompt words will be converted before each reasoning, and then searched in the storage medium. If the search is found, the generated multimodal results can be directly returned. If not found, reasoning generation is performed and the generated results are placed in the storage medium as a cache for the future.

7. The method according to claim 6, characterized in that The original prompts are converted into a standardized format through predefined rules or algorithms to ensure that prompts in different forms can be mapped to the same identifier after processing.

8. The method according to claim 7, characterized in that The process of converting the original prompt includes removing irrelevant characters, unifying upper and lower cases, word segmentation, extracting keywords, and vectorizing and hashing.

9. The method according to claim 6, characterized in that A key-value pair storage system is used, where the key is the new prompt word and the value corresponds to the multimodal reasoning result.

10. The method according to claim 6, characterized in that in, The evaluation criteria of the cache evaluation module include data access frequency, freshness, accuracy, hit rate, and user satisfaction; based on the evaluation results of the cache evaluation module, the data in the cache is updated or deleted.