Large model reasoning method and device based on multi-level cache, storage medium and equipment

Abstract

The invention discloses a large model reasoning method and device based on multi-level cache, a storage medium and equipment, and the method comprises the steps: receiving a reasoning task, and carrying out fixed-length lexical sequence segmentation processing on the reasoning task to determine at least one lexical sequence corresponding to the reasoning task, the lengths of other lexical element sequences except the last lexical element sequence in the lexical element sequences are fixed lengths, and the length of the last lexical element sequence is smaller than or equal to the fixed length; aiming at the lexical element sequence with the fixed length, a target data block corresponding to the lexical element sequence is inquired in a video memory, if the lexical element sequence does not exist in the video memory, the target data block corresponding to the lexical element sequence is inquired in a memory, and when the data blocks in the video memory are evicted out of the video memory, the data blocks are copied into the memory; if the tail lexical element sequence is smaller than the fixed length, the reasoning task is executed according to the target data block and the tail lexical element sequence through the large model; if the tail lexical element sequence is equal to the fixed length, the reasoning task is executed according to the target data block through the large model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of task reasoning, in particular to a large model reasoning method and device based on multi-level cache, storage medium and computer equipment. BACKGROUND

[0002] With the continuous expansion of the parameter scale of large language models and the significant increase in the length of the context, the demand for video memory resources in the reasoning process has risen sharply, which has become a key bottleneck restricting the efficient deployment of large models. In the autoregressive generation process, key-value cache (KVCache) as the core mechanism for storing intermediate states, its memory occupancy increases linearly with the sequence length. In a single machine environment, the capacity of the video memory is limited, and when processing long text input, the accumulated KVCache will soon exceed the carrying capacity of the video memory, forcing the system to frequently trigger the cache replacement and eviction mechanism, resulting in a large number of tensors being expelled from the video memory and being reloaded when needed. This "see-saw effect" between video memory and computation not only causes the idle of GPU computing units, but also causes potential re-computation overhead due to cache misses, which seriously weakens the overall reasoning throughput and forms systematic performance degradation.

[0003] To alleviate the pressure on video memory, current mainstream optimization schemes mostly use distributed cache architecture or block-based memory management mechanism, which identifies reusable context fragments through prefix matching and exchanges data blocks between video memory and memory (or remote storage). Although such methods improve cache utilization to some extent, they still have obvious defects. The core replacement strategy usually only takes cache hit rate as the optimization target, lacking fine-grained evaluation of the trade-off between "exchange cost" and "re-computation cost". Especially for shorter token sequence fragments, the I / O overhead of loading cache blocks from memory or remote storage is often higher than the cost of direct re-computation, resulting in a decrease in system performance. In addition, existing methods generally use fixed padding or coarse-grained blocking strategies when processing variable-length inputs, failing to adaptively divide based on actual sequence structure, further exacerbating resource waste and latency. SUMMARY

[0004] Therefore, the embodiments of the present application provide a large model reasoning method and device based on multi-level cache, storage medium and computer equipment, which balance the time overhead of cache replacement and model re-computation, achieve dynamic optimal balance between video memory occupancy and computing efficiency, and realize efficient and low-latency processing of large model long context reasoning tasks on the premise of ensuring reasoning quality.

[0005] According to one aspect of the present application, a large model reasoning method based on multi-level cache is provided, the method comprising:

[0006] receiving an inference task, performing fixed-length token sequence segmentation processing on the inference task to determine at least one token sequence corresponding to the inference task, wherein the length of each token sequence in the token sequence except for a tail token sequence is the fixed length, and the length of the tail token sequence is less than or equal to the fixed length;

[0007] For the fixed-length token sequence, querying the target data block corresponding to the token sequence in the video memory, and if the target data block does not exist in the video memory, querying the target data block corresponding to the token sequence in the memory, wherein the data block in the video memory is copied to the memory when the data block is evicted from the video memory;

[0008] If the length of the tail token sequence is less than the fixed length, performing the inference task by a large model according to the target data block and the tail token sequence;

[0009] If the length of the tail token sequence is equal to the fixed length, performing the inference task by the large model according to the target data block.

[0010] In the embodiments of the present application, optionally, the method further comprises:

[0011] In the case where the target data block corresponding to the fixed-length token sequence is not acquired, acquiring an intermediate calculation result of the token sequence in the process of performing the inference task by the large model, and generating a data block based on the token sequence and the intermediate calculation result and storing the data block in the video memory;

[0012] If the length of the tail token sequence is less than the fixed length, performing the inference task by a large model according to the target data block and the tail token sequence;

[0013] If the length of the tail token sequence is less than the fixed length, performing the inference task by a large model according to the target data block and the tail token sequence;

[0014] If the length of the tail token sequence is equal to the fixed length, performing the inference task by the large model according to the target data block.

[0015] If the length of the tail token sequence is equal to the fixed length, performing the inference task by the large model according to the target data block.

[0016] In the embodiments of the present application, optionally, the method further comprises:

[0017] Counting the number of associated requests for the data blocks in the video memory, wherein the number of associated requests represents the current number of requests for the corresponding data block;

[0018] When the number of associated requests of any one first data block in the video memory is zero, triggering an eviction mechanism of the first data block, releasing the first data block from the video memory and copying the first data block to the memory.

[0019] In the embodiments of the present application, optionally, the data block includes a key-value pair of a corresponding token sequence, the key information of the key-value pair is the unique identifier of the token sequence, and the value information of the key-value pair is the intermediate calculation result of the token sequence; after the data block generated based on the token sequence and the intermediate calculation result is stored in the video memory, the method further includes:

[0020] updating the metadata in the memory according to the generated data block, wherein the metadata includes the unique identifier of the token sequence corresponding to each data block in the video memory and the memory and the data block position identifier, and the data block position identifier is used to indicate whether the corresponding data block exists in the video memory.

[0021] In the embodiments of the present application, optionally, the querying the target data block corresponding to the token sequence in the video memory, if the target data block does not exist in the video memory, querying the target data block corresponding to the token sequence in the memory includes:

[0022] performing unique identifier calculation on the token sequence, and when the unique identifier calculation result of the token sequence hits the unique identifier in the metadata, querying whether the target data block corresponding to the token sequence exists in the video memory based on the data block position identifier corresponding to the unique identifier calculation result in the metadata;

[0023] if the target data block exists in the video memory, obtaining the target data block corresponding to the token sequence in the video memory;

[0024] if the target data block does not exist in the video memory, querying the target data block corresponding to the token sequence in the memory.

[0025] In the embodiments of the present application, optionally, after querying the target data block corresponding to the token sequence in the memory, the method further includes:

[0026] if the target data block does not exist in the memory, querying the target data block corresponding to the token sequence in the external storage, wherein the external storage includes the data block copied from the memory.

[0027] In the embodiments of the present application, optionally, the method further includes:

[0028] counting the number of copy threads of the data block in the memory, wherein the number of copy threads represents the current number of threads of the corresponding data block copied to the video memory;

[0029] copying the second data block from the memory to the external storage when the number of copy threads of any one of the second data blocks in the memory is zero.

[0030] In the embodiments of the present application, optionally, after the target data block corresponding to the word sequence is queried in the external storage, the method further comprises:

[0031] If the target data block corresponding to the word sequence exists in the external storage, the target data block is copied from the external storage to the video memory by using a GDR exchange technology, and the data block position identifier of the metadata corresponding to the target data block in the memory is updated based on the target data block.

[0032] In the embodiments of the present application, optionally, the method further comprises:

[0033] counting the number of associated requests of the data blocks in the video memory, wherein the number of associated requests represents the current number of requests of the corresponding data blocks;

[0034] copying the third data block to the external storage when the number of associated requests of any one of the third data blocks in the video memory is greater than a preset hot data request number.

[0035] In the embodiments of the present application, optionally, after the inference task is received, the method further comprises:

[0036] allocating the inference task to one node of a plurality of task processing nodes in a task processing cluster;

[0037] The counting the number of associated requests of the data blocks in the video memory comprises:

[0038] counting the number of associated requests of the data blocks in the video memory of each task processing node;

[0039] The copying the third data block to the external storage when the number of associated requests of any one of the third data blocks in the video memory is greater than a preset hot data request number comprises:

[0040] copying the third data block to the external storage when the number of associated requests of any one of the third data blocks in the video memory of any one of the task processing nodes is greater than a preset hot data request number.

[0041] In the embodiments of the present application, optionally, the method further comprises:

[0042] The task processing nodes respectively perform hotspot data sensing on the external storage, and when new hotspot data is sensed in the external storage, the hotspot data is loaded into the video memory of the task processing nodes, wherein the hotspot data includes data blocks with a number of associated requests greater than a preset number of hotspot data requests.

[0043] In the embodiments of the present application, optionally, the method further comprises:

[0044] When any one of the task processing nodes in the task processing cluster is restarted, the any one of the task processing nodes reads a data block list in the external storage, and loads required data blocks from the external storage based on the data block list for data recovery.

[0045] According to another aspect of the present application, a large model inference device based on multi-level cache is provided, the device comprising:

[0046] A token sequence segmentation module is configured to receive an inference task, and perform fixed-length token sequence segmentation processing on the inference task to determine at least one token sequence corresponding to the inference task, wherein the length of each token sequence in the token sequence except for a last token sequence is the fixed length, and the length of the last token sequence is less than or equal to the fixed length.

[0047] A data block acquisition module is configured to, for a fixed-length token sequence, query a target data block corresponding to the token sequence in the video memory, and if the target data block does not exist in the video memory, query the target data block in the memory, wherein the data blocks in the video memory are copied to the memory when they are evicted from the video memory.

[0048] A task inference module is configured to, if the last token sequence is less than the fixed length, execute the inference task according to the target data block and the last token sequence by using a large model; and if the last token sequence is equal to the fixed length, execute the inference task according to the target data block by using the large model.

[0049] In the embodiments of the present application, optionally, the device further comprises:

[0050] A cache module is configured to, in a case where a target data block corresponding to a fixed-length token sequence is not acquired, acquire an intermediate calculation result of the token sequence in a process in which the large model executes the inference task, and generate a data block based on the token sequence and the intermediate calculation result and store the data block in the video memory.

[0051] The task inference module is further configured to: if the tail token sequence is less than the fixed length, performing the inference task according to the intermediate calculation result in the target data block and the tail token sequence by using a large model; and if the tail token sequence is equal to the fixed length, performing the inference task according to the intermediate calculation result in the target data block by using a large model.

[0052] In the embodiments of the present application, optionally, the cache module is further configured to:

[0053] counting the number of associated requests of the data blocks in the video memory, wherein the number of associated requests represents the current number of requests of the corresponding data block;

[0054] when the number of associated requests of any one first data block in the video memory is zero, triggering the eviction mechanism of the first data block to release the first data block from the video memory and copy it to the memory.

[0055] In the embodiments of the present application, optionally, the data block includes a key-value pair of the corresponding token sequence, the key information of the key-value pair is the unique identifier of the token sequence, and the value information of the key-value pair is the intermediate calculation result of the token sequence.

[0056] The cache module is further configured to update the metadata in the memory according to the generated data block, wherein the metadata includes the unique identifier of the token sequence corresponding to each data block in the video memory and the memory and the data block position identifier, and the data block position identifier is used to indicate whether the corresponding data block exists in the video memory.

[0057] In the embodiments of the present application, optionally, the data block obtaining module is further configured to:

[0058] performing unique identifier calculation on the token sequence, and when the unique identifier calculation result of the token sequence hits the unique identifier in the metadata, querying whether the target data block corresponding to the token sequence exists in the video memory based on the data block position identifier corresponding to the unique identifier calculation result in the metadata;

[0059] if the target data block exists in the video memory, obtaining the target data block corresponding to the token sequence in the video memory;

[0060] if the target data block does not exist in the video memory, querying the target data block corresponding to the token sequence in the memory.

[0061] In the embodiments of the present application, optionally, the data block obtaining module is further configured to:

[0062] If the word sequence corresponds to a target data block in the external storage, the target data block is copied from the external storage to the GPU memory by using a GDR exchange technology, and a data block position identifier of metadata corresponding to the target data block in the memory is updated based on the target data block.

[0063] In the embodiments of the present application, the cache module is further configured to:

[0064] counting a number of copy threads of the data block in the memory, wherein the number of copy threads represents a current number of threads for copying the corresponding data block to the GPU memory;

[0065] when the number of copy threads of any one of the second data blocks in the memory is zero, releasing the second data block from the memory and copying the second data block to the external storage.

[0066] In the embodiments of the present application, the cache module is further configured to:

[0067] If the word sequence corresponds to a target data block in the external storage, the target data block is copied from the external storage to the GPU memory by using a GDR exchange technology, and a data block position identifier of metadata corresponding to the target data block in the memory is updated based on the target data block.

[0068] In the embodiments of the present application, the cache module is further configured to:

[0069] counting a number of associated requests of the data block in the GPU memory, wherein the number of associated requests represents a current number of requests of the corresponding data block;

[0070] when the number of associated requests of any one of the third data blocks in the GPU memory is greater than a preset hot data request number, copying the third data block to the external storage.

[0071] In the embodiments of the present application, the word sequence segmentation module is further configured to: assign the inference task to one of a plurality of task processing nodes in a task processing cluster;

[0072] The cache module is further configured to: count a number of associated requests of a data block in a GPU memory of each task processing node; and when the number of associated requests of any one of the third data blocks in the GPU memory of any one of the task processing nodes is greater than a preset hot data request number, copy the third data block to the external storage.

[0073] In the embodiments of the present application, the cache module is further configured to:

[0074] The task processing nodes respectively perform hotspot data sensing on the external storage, and when new hotspot data is sensed in the external storage, the hotspot data is loaded into the video memory of the task processing nodes, wherein the hotspot data includes data blocks with a number of associated requests greater than a preset number of hotspot data requests.

[0075] In the embodiments of the present application, the data block obtaining module is further configured to:

[0076] When any one of the task processing nodes in the task processing cluster is restarted, the any one of the task processing nodes reads a data block list in the external storage, and loads required data blocks from the external storage based on the data block list for data recovery.

[0077] According to another aspect of the present application, a storage medium having a computer program stored thereon is provided, and the program is executed by a processor to implement the above-mentioned large model inference method based on multi-level cache.

[0078] According to another aspect of the present application, a computer device is provided, which includes a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, and the processor implements the above-mentioned large model inference method based on multi-level cache when executing the program.

[0079] Through the above technical solutions, the embodiments of the present application provide a large model inference method and device based on multi-level cache, a storage medium, and a computer device. By dividing an input sequence into fixed-length data blocks and combining the cooperative caching mechanism of video memory and internal memory, the performance bottleneck caused by limited video memory resources is alleviated. Compared with traditional methods, the embodiments of the present application reduce the recalculation and frequent data exchange caused by cache misses when processing long text inputs, and improve inference efficiency and resource utilization. At the same time, the end non-integer multiple length input is optimized, unnecessary padding operations are avoided, and the calculation efficiency and output accuracy are improved. In addition, by actively copying the video memory data to the internal memory when the video memory data is evicted, the reuse and continuity management of cache data are realized, and the overall stability and scalability of the system are enhanced. Finally, under the premise of ensuring inference quality, efficient and low-latency processing of large model long context inference tasks is realized.

[0080] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the embodiments of the present application can be implemented in accordance with the content of the description, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0081] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0082] Figure 1 A flowchart of a large model inference method based on multi-level cache provided by an embodiment of the application is shown;

[0083] Figure 2 A flowchart of another large model inference method based on multi-level cache provided by an embodiment of the application is shown;

[0084] Figure 3 A structural diagram of a large model inference device based on multi-level cache provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0085] The application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments. It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0086] In this embodiment, a large model inference method based on multi-level cache is provided, as shown in the figure, the method comprises: Figure 1

[0087] Step 101, receiving an inference task, performing fixed-length token sequence segmentation processing on the inference task to determine at least one token sequence corresponding to the inference task, wherein the length of each token sequence except the last token sequence is the fixed length, and the length of the last token sequence is less than or equal to the fixed length.

[0088] In the embodiment of the application, the inference task is received, and the fixed-length token sequence segmentation processing is performed on the inference task to determine at least one token sequence. Among them, the length of each token sequence except the last token sequence is the fixed length, and the length of the last token sequence is less than or equal to the fixed length. By dividing the input text into fixed-length blocks, the size of each processing unit can be controlled, so that it is easier to manage and schedule in the video memory, and it is helpful for the unified processing of the subsequent cache mechanism. The length of the last sequence is allowed to be less than the fixed length, which can avoid forced padding of the input, reduce redundant calculation and invalid memory occupation.

[0089] Step 102, for the fixed-length token sequence, querying the target data block corresponding to the token sequence in the video memory, if the target data block does not exist in the video memory, querying the target data block corresponding to the token sequence in the memory, wherein the data block in the video memory is copied to the memory when it is evicted from the video memory. ​

[0090] In the embodiments of the present application, for each fixed-length token sequence, the corresponding target data block is queried in the video memory. If the data block does not exist in the video memory, it is further queried in the memory. In the embodiments of the present application, a multi-level cache mechanism is constructed for the cooperation of the video memory and the memory. When the data block in the video memory is evicted, it is copied to the memory for subsequent reuse. The video memory is used as a high-speed access storage level and is preferentially used to cache the data block required for current processing; the memory is used as a backup cache with larger capacity but slower access speed. Through the step-by-step query mechanism from the video memory to the memory, the cache capacity can be effectively expanded without sacrificing performance, and the recalculation or frequent data loading operations caused by cache miss can be reduced. In addition, when the data block is evicted from the video memory, it is actively copied to the memory, which retains historical cache information and provides support for subsequent reuse, thereby improving the continuity and utilization of the cache.

[0091] In step 103, if the end token sequence is less than the fixed length, the inference task is performed by the large model according to the target data block and the end token sequence.

[0092] In step 104, if the end token sequence is equal to the fixed length, the inference task is performed by the large model according to the target data block.

[0093] In the embodiments of the present application, if the length of the end token sequence is less than the fixed length, the inference task is performed by the large model according to the target data block and the end token sequence, which avoids the invalid padding operation introduced in the traditional method to meet the fixed length requirement. By only calculating the end variable length part, redundant calculation and unnecessary video memory occupation can be reduced, and historical data blocks in the cache are combined to achieve efficient inference. This method is especially suitable for non-integer multiple length inputs commonly used in natural language processing, which improves the model processing flexibility and resource utilization. For shorter token sequences, the efficiency of external exchange is significantly lower than that of recalculation, which limits the system-level optimization efficiency. At this time, it is more efficient to directly recalculate. If the length of the end token sequence is equal to the fixed length, the inference task is performed by the large model according to the target data block, which processes the case where the input length is an integer multiple of the fixed length. At this time, there is no need to additionally process the irregularity of the end part, and the inference process can be completely based on the fixed-length data block in the cache, which further improves the processing efficiency and cache hit rate.

[0094] By applying the technical solution of the embodiment, the performance bottleneck caused by limited graphics memory resources is alleviated by dividing the input sequence into fixed-length data blocks and combining the cooperative caching mechanism of the graphics memory and the memory. Compared with the traditional method, the present embodiment reduces the recalculation and frequent data exchange caused by cache misses when processing long text input, thereby improving the inference efficiency and resource utilization. At the same time, the input of non-integer multiple length at the end is optimized, unnecessary padding operations are avoided, and the calculation efficiency and output accuracy are improved. In addition, by actively copying the graphics memory data to the memory when it is evicted, the reuse and continuity management of cache data are realized, and the overall stability and scalability of the system are enhanced. Finally, under the premise of ensuring the inference quality, efficient and low-delay processing of large model long context inference tasks is realized.

[0095] In the embodiment of the present application, optionally, the method further comprises: in the case where the target data block corresponding to the fixed-length word sequence is not acquired, acquiring the intermediate calculation result of the word sequence in the process of the large model executing the inference task, and generating a data block based on the word sequence and the intermediate calculation result and storing it in the graphics memory.

[0096] In this embodiment, when the target data block corresponding to the fixed-length word sequence is not acquired, the intermediate calculation result corresponding to the word sequence is obtained by actually executing the inference process of the word sequence, and a new data block is generated based on the word sequence and the calculation result, and then the data block is cached in the graphics memory for subsequent inference task reuse. In a specific application scenario, if there is a target data block corresponding to a word sequence that is not acquired, the target data block corresponding to the fixed-length word sequence acquired (if any), the word sequence for which the target data block is not acquired, and the end word sequence (if less than the fixed length) are executed by the large model to perform the inference task, and the intermediate calculation result of the fixed-length word sequence for which the target data block is not acquired is generated together with the word sequence to generate a new data block and store it in the graphics memory. When a certain word sequence does not hit in the graphics memory and the memory, the calculation result of this time is actively cached by generating a new cache data block, which provides fast access support for the same or similar context that may appear subsequently. This "calculate and cache" strategy not only improves the cache hit rate, but also enhances the adaptability and continuity of the cache data of the system.

[0097] In the embodiment of the present application, optionally, the method further comprises: counting the number of associated requests of the data blocks in the graphics memory, wherein the number of associated requests represents the current number of requests of the corresponding data block; when the number of associated requests of any first data block in the graphics memory is zero, triggering the eviction mechanism of the first data block to release the first data block from the graphics memory and copy it to the memory.

[0098] In this embodiment, the number of associated requests of each data block in the video memory is maintained to record the number of inference tasks currently accessing the data block, thereby reflecting the activity level of the data block. When the number of associated requests of a certain data block drops to zero, it indicates that no inference task currently needs to use the data block, that is, it is in an idle state, and the eviction mechanism is triggered to release it from the video memory and copy it completely to the memory for storage, so that it can be loaded and used again by subsequent inference tasks when needed. The design of this mechanism solves the common "mis-eviction" problem in traditional cache management strategies. For example, strategies based on time or access frequency (such as LRU, FIFO) may mistakenly remove the data block when it is still being used, resulting in cache miss during subsequent access, and thus causing additional computational overhead or video memory loading delay. By using reference counting, the eviction is only performed when the data block is no longer used at all, improving the accuracy and security of cache management.

[0099] In the embodiments of the present application, the data block includes a key-value pair of the corresponding token sequence, the key information of the key-value pair is the unique identifier of the token sequence, and the value information of the key-value pair is the intermediate calculation result of the token sequence. After the data block generated based on the token sequence and the intermediate calculation result is stored in the video memory, the method further includes updating the metadata in the memory according to the generated data block, wherein the metadata includes the unique identifier of the token sequence corresponding to each data block in the video memory and the memory, and the data block position identifier, the data block position identifier is used to indicate whether the corresponding data block exists in the video memory.

[0100] In this embodiment, the data block contains a key-value pair structure corresponding to the token sequence, where the key (Key) is the unique identifier of the token sequence, used to uniquely identify the data block in the cache system; the value (Value) is the intermediate calculation result of the token sequence produced in the inference process, such as key-value cache (KVCache) and the like, facilitating quick positioning and reuse of historical calculation results. After generating a new data block and caching it to the video memory, the metadata table in the memory is updated according to the information of the data block. The metadata table records the unique identifier of the token sequence of all cached data blocks and the data block position identifier, wherein the data block position identifier is used to indicate whether the data block currently exists in the video memory, for example, the data block position identifier is_gpu is set to False, indicating that the block is not in the video memory, and is_gpu is set to True, indicating that the block is in the video memory. This metadata updating mechanism ensures that the target data block can be accurately determined to hit the video memory or need to be further loaded from the memory during subsequent query process, thereby improving the overall cache query efficiency and inference scheduling accuracy.

[0101] Further, as a refinement and extension of the above embodiment, in order to fully describe the specific implementation process of the embodiment, another large model inference method based on multi-level cache is provided, as shown in Figure 2 The method comprises the following steps:

[0102] Step 201, receiving an inference task, performing fixed-length token sequence segmentation processing on the inference task to determine at least one token sequence corresponding to the inference task, wherein the length of each token sequence in the token sequence except the last token sequence is the fixed length, and the length of the last token sequence is less than or equal to the fixed length.

[0103] In the embodiment of the application, the input text is divided into a plurality of fixed-length data blocks, so that the size of each processing unit is controllable, facilitating efficient scheduling and cache management in the video memory. At the same time, the length of the last token sequence is allowed to be less than the fixed length, avoiding forced padding of the input, thereby reducing redundant calculation and invalid memory occupation, and improving inference efficiency and accuracy of model output.

[0104] Step 202, for the fixed-length token sequence, performing unique identifier calculation on the token sequence, and when the unique identifier calculation result of the token sequence hits the unique identifier in the metadata, querying whether there is a target data block corresponding to the token sequence in the video memory based on the data block position identifier corresponding to the unique identifier calculation result in the metadata; if there is, obtaining the target data block corresponding to the token sequence in the video memory; if there is not, querying the target data block corresponding to the token sequence in the memory.

[0105] In the embodiment of the application, for each fixed-length token sequence, its unique identifier is calculated, and whether the unique identifier exists in the metadata is found. If it exists, it is determined whether the target data block corresponding to the token sequence exists in the video memory according to the data block position identifier recorded in the metadata. If it exists, the data block is directly obtained from the video memory; if it does not exist, it is further found in the memory. Thus, based on the cache query mechanism of the unique identifier, it is ensured that the existing intermediate calculation results can be efficiently identified and reused. By recording the position information (video memory or memory) of the data block in the metadata, the required data can be quickly located without redundant calculation, reducing the performance loss caused by cache miss, and improving cache hit rate and inference efficiency.

[0106] Step 203, if the memory does not exist, querying the target data block corresponding to the token sequence in the external storage, wherein the external storage includes data blocks copied from the memory.

[0107] In the embodiments of the present application, if the target data block is not found in the memory, the data block corresponding to the token sequence is further searched in the external storage. The external storage saves the data blocks migrated from the memory as the bottom layer of the cache system. By introducing the external storage as an extended layer of the cache system, the historical calculation results can still be retained in the case of limited memory and graphics memory resources, avoiding permanent data loss, providing stronger continuity support for long context or multi-round reasoning tasks, and enhancing the scalability and stability of the cache system.

[0108] In step 204, if the target data block corresponding to the token sequence exists in the external storage, the target data block is copied to the graphics memory through GDR exchange technology, and the data block position identifier of the metadata corresponding to the target data block in the memory is updated based on the target data block.

[0109] In the embodiments of the present application, if the target data block is found in the external storage, the data block is directly copied to the graphics memory through GDR (GPU Direct RDMA) technology, and the metadata in the memory is updated, and the position identifier of the data block is updated to "existing in the graphics memory". The introduction of GDR technology enables data to be directly transmitted from the external storage (such as SSD or remote node) to the graphics memory, bypassing the memory, reducing data transmission delay and system overhead. At the same time, updating the metadata ensures the accurate synchronization of the cache state, and improves the data migration efficiency and system response speed between multi-level caches.

[0110] In step 205, if the end token sequence is less than the fixed length, the reasoning task is performed by the large model according to the intermediate calculation result in the target data block and the end token sequence.

[0111] In step 206, if the end token sequence is equal to the fixed length, the reasoning task is performed by the large model according to the intermediate calculation result in the target data block.

[0112] In the embodiments of the present application, if the length of the end token sequence is less than the fixed length, the reasoning task is performed by the large model according to the intermediate calculation result in the target data block and the end token sequence, avoiding the padding operation introduced in the traditional method to meet the fixed length requirement, thereby reducing redundant calculation and invalid graphics memory occupation. If the length of the end token sequence is equal to the fixed length, the reasoning task is performed by the large model according to the intermediate calculation result in the target data block, and at this time, the irregularity of the end part does not need to be processed additionally, and the reasoning process can be completed based on the fixed length data block in the cache, further improving the processing efficiency and cache hit rate.

[0113] By applying the technical solutions of the embodiment, the input token sequence is divided into fixed-length blocks, and a multi-level cache system of a graphics memory, a memory and an external storage is combined to realize efficient management and reuse of cache data in the large model inference process. Among them, the introduction of unique identifiers, metadata management, GDR accelerated transmission and other technologies improves the cache hit rate and data migration efficiency, reduces the recalculation and data loading delay caused by cache miss. At the same time, the flexible processing mechanism of the tail token sequence avoids the resource waste caused by forced padding in the traditional method, and improves the inference efficiency and output quality. Combined with reference counting, data block generation and eviction mechanism, dynamic scheduling and on-demand release of cache resources can be realized, further relieving the graphics memory pressure and enhancing the adaptability to complex inference tasks such as long context and high concurrency.

[0114] In the embodiment of the application, optionally, the method further comprises: counting the number of copy threads of the data blocks in the memory, wherein the number of copy threads represents the current number of threads for copying the corresponding data blocks to the graphics memory; and when the number of copy threads of any one second data block in the memory is zero, releasing the second data block from the memory and copying it to the external storage.

[0115] In this embodiment, in order to optimize the memory usage efficiency and ensure the effective migration of data blocks in the multi-level storage system, the number of copy threads of the data blocks in the memory is counted. Specifically, it is tracked how many threads are currently trying to copy each data block from the memory to the graphics memory. When the number of copy threads of any one second data block in the data blocks is zero, the second data block is released from the memory and copied to the external storage. That is, if no thread is using or trying to transfer a certain data block (i.e., the number of copy threads is zero), it is considered that the data block will not be used temporarily. At this time, it can be safely removed from the memory to release the memory space and migrated to the external storage (such as SSD) for storage. This strategy is beneficial to manage limited memory resources, ensures that tasks or data blocks that need more memory space can obtain sufficient support, and also ensures that even if the data blocks are temporarily not in the memory, they can be accessed through the external storage, thereby enhancing the overall stability and performance of the system.

[0116] In the embodiment of the application, optionally, the method further comprises: counting the number of associated requests of the data blocks in the graphics memory, wherein the number of associated requests represents the current number of requests of the corresponding data blocks; and when the number of associated requests of any one third data block in the graphics memory is greater than a preset hot data request number, copying the third data block to the external storage.

[0117] In this embodiment, a "associated request number" is maintained for each data block in the video memory, which records the number of inference tasks currently referencing the data block, as a quantitative indicator of its access frequency. When the associated request number of a certain third data block in each data block exceeds the preset hot data request number, the data block is copied from the video memory to the external storage. The high-frequency access data is backed up persistently to support subsequent cross-task and cross-session reuse. By actively storing hot data to the external storage, its long-term accessibility can be ensured, avoiding loss due to pressure from the video memory. This mechanism is especially suitable for scenarios where there is a large amount of shared context or multi-task reuse in the inference process. In addition, this mechanism can also serve as the basis for hot data identification and preloading strategy, providing data preparation for subsequent fast preloading from external storage to memory or video memory.

[0118] In the embodiments of the present application, optionally, after receiving the inference task, the method further comprises: assigning the inference task to one of a plurality of task processing nodes in a task processing cluster; the counting of the associated request number of the data block in the video memory comprises: counting the associated request number of the data block in the video memory of each task processing node; and the copying of the third data block to the external storage when the associated request number of any one third data block in the video memory is greater than the preset hot data request number comprises: copying the third data block to the external storage when the associated request number of any one third data block in the video memory of any one task processing node is greater than the preset hot data request number.

[0119] In this embodiment, after receiving the inference task, it is assigned to a certain task processing node in the cluster for execution, and the data block related to the task is maintained in the video memory of the node. The system counts the associated request number of the data block in the video memory of each node, which is used to quantify its access frequency in the local task. When the access frequency of a certain data block exceeds the preset hot data request number, it is determined to be hot data with high reuse value, and it is actively copied to the shared external storage for reuse by other nodes in the cluster in future inference tasks. Thus, by identifying locally high-frequency access cache data and sharing it to the global storage layer, cross-node and cross-task cache data reuse is achieved, improving the resource utilization efficiency and cache hit rate of the entire inference cluster.

[0120] In the embodiments of the present application, optionally, the method further comprises: each task processing node performs hot data sensing on the external storage, and when new hot data is sensed in the external storage, the hot data is loaded into the video memory of the task processing node, wherein the hot data includes a data block with an associated request number greater than a preset hot data request number.

[0121] In this embodiment, each task processing node in the cluster continuously performs hotspot data awareness on the external storage, and once it detects that a data block marked as "hotspot" (i.e., its associated request number has exceeded the preset threshold and is a high-frequency access data pushed to the external storage by other nodes) is added to the external storage, the hotspot data is immediately loaded into the video memory of the node. This mechanism realizes the automatic discovery and rapid diffusion of hotspot cache data within the cluster, so that high-value cache information generated by any node can be actively acquired and reused by other potentially needed nodes. In this way, the cache hit rate across tasks and sessions is improved, and the repeated computation overhead is reduced. Especially when processing inference requests with similar context or shared knowledge path, the response delay can be greatly shortened and the video memory utilization efficiency can be improved. At the same time, this scheme enhances the cooperative intelligence of the cluster, so that the cache system has a certain adaptive and predictive ability, thereby effectively supporting high-throughput and low-latency service requirements in large-scale concurrent inference scenarios.

[0122] In the embodiments of the present application, optionally, the method further comprises: when any one of the task processing nodes in the task processing cluster is restarted, the any one of the task processing nodes reads the data block list in the external storage, and loads the required data block from the external storage based on the data block list for data recovery.

[0123] In this embodiment, when any task processing node in the cluster is restarted, the node first reads the data block list saved in the external storage, and loads the data block required by the node from the external storage based on the list to recover the cache state of the node before the restart. By persistently storing the cache data in the external storage, it is ensured that even in the case of abnormal or active restart of the node, important intermediate calculation results previously cached by the node will not be lost, thereby ensuring the continuity and stability of the inference task and enhancing the disaster recovery capability of the system and the migratability of the cache data. In a distributed inference scenario, node restart is a common operation and maintenance operation or fault recovery means. If there is no effective cache recovery mechanism, the node needs to start processing the task from the beginning after the restart, resulting in a large amount of repeated calculation and waste of video memory resources. By loading the historical cache data block from the external storage, the restarted node can quickly rebuild its local cache environment, shorten the recovery time, and improve the overall inference efficiency.

[0124] Further, as a specific implementation of the method, the embodiments of the present application provide a large model inference device based on multi-level cache, as shown in Figure 1 FIG. 1, the device comprises: Figure 3

[0125] ​The word sequence segmentation module is configured to receive an inference task, perform fixed-length word sequence segmentation processing on the inference task, and determine at least one word sequence corresponding to the inference task, wherein the length of each word sequence in the word sequences except for a last word sequence is the fixed length, and the length of the last word sequence is less than or equal to the fixed length.

[0126] The data block acquisition module is configured to, for the fixed-length word sequence, query a target data block corresponding to the word sequence in the video memory, and if the target data block does not exist in the video memory, query the target data block in the internal memory, wherein the data blocks in the video memory are copied to the internal memory when the data blocks are evicted from the video memory.

[0127] The task inference module is configured to, if the last word sequence is less than the fixed length, perform the inference task based on the target data block and the last word sequence by using a large model, and if the last word sequence is equal to the fixed length, perform the inference task based on the target data block by using the large model.

[0128] In an embodiment of the present application, the device further includes:

[0129] The cache module is configured to, in a case where the target data block corresponding to the fixed-length word sequence is not acquired, acquire an intermediate calculation result of the word sequence in a process in which the large model performs the inference task, and generate a data block based on the word sequence and the intermediate calculation result and store the data block in the video memory.

[0130] The task inference module is further configured to, if the last word sequence is less than the fixed length, perform the inference task based on the intermediate calculation result in the target data block and the last word sequence by using the large model, and if the last word sequence is equal to the fixed length, perform the inference task based on the intermediate calculation result in the target data block by using the large model.

[0131] In an embodiment of the present application, the cache module is further configured to:

[0132] Count the number of associated requests for the data blocks in the video memory, wherein the number of associated requests represents a current number of requests for a corresponding data block.

[0133] When the number of associated requests for any first data block in the video memory is zero, trigger an eviction mechanism of the first data block, release the first data block from the video memory and copy the first data block to the internal memory.

[0134] In the embodiments of the present application, optionally, the data block comprises a key-value pair of the corresponding token sequence, the key information of the key-value pair is the unique identifier of the token sequence, and the value information of the key-value pair is the intermediate calculation result of the token sequence.

[0135] The cache module is further configured to update the metadata in the internal memory according to the generated data block, wherein the metadata comprises the unique identifier of the token sequence corresponding to each data block in the video memory and the internal memory and a data block position identifier, and the data block position identifier is used to indicate whether the corresponding data block exists in the video memory.

[0136] In the embodiments of the present application, optionally, the data block obtaining module is further configured to:

[0137] perform unique identifier calculation on the token sequence, and when the unique identifier calculation result of the token sequence hits the unique identifier in the metadata, query whether the target data block corresponding to the token sequence exists in the video memory based on the data block position identifier corresponding to the unique identifier calculation result in the metadata;

[0138] if the target data block exists in the video memory, obtain the target data block corresponding to the token sequence in the video memory;

[0139] if the target data block does not exist in the video memory, query the target data block corresponding to the token sequence in the internal memory.

[0140] In the embodiments of the present application, optionally, the data block obtaining module is further configured to:

[0141] if the target data block does not exist in the internal memory, query the target data block corresponding to the token sequence in the external storage, wherein the external storage comprises the data block copied from the internal memory.

[0142] In the embodiments of the present application, optionally, the cache module is further configured to:

[0143] count the number of copy threads of the data block in the internal memory, wherein the number of copy threads represents the current number of threads for copying the corresponding data block to the video memory;

[0144] when the number of copy threads of any one second data block in the internal memory is zero, release the second data block from the internal memory and copy it to the external storage.

[0145] In the embodiments of the present application, optionally, the cache module is further configured to:

[0146] copy, by a GDR exchange technique, a target data block corresponding to the word sequence in the external storage into the display memory, and update a data block position identifier of metadata corresponding to the target data block in the memory based on the target data block.

[0147] In the embodiment of the present application, optionally, the cache module is further configured to:

[0148] count the number of associated requests of the data blocks in the display memory, wherein the number of associated requests represents the current number of requests of the corresponding data blocks;

[0149] copy, into the external storage, any third data block in the display memory when the number of associated requests of the third data block is greater than a preset hot data request number.

[0150] In the embodiment of the present application, optionally, the word sequence segmentation module is further configured to: assign the inference task to one node of a plurality of task processing nodes in a task processing cluster;

[0151] The cache module is further configured to: count the number of associated requests of the data blocks in the display memory of each task processing node; and copy, into the external storage, any third data block in the display memory of any one task processing node when the number of associated requests of the third data block is greater than a preset hot data request number.

[0152] In the embodiment of the present application, optionally, the cache module is further configured to:

[0153] Each task processing node performs hot data sensing on the external storage, and loads hot data into the display memory of the task processing node when the task processing node senses that there is new hot data in the external storage, wherein the hot data includes a data block with a number of associated requests greater than a preset hot data request number.

[0154] In the embodiment of the present application, optionally, the data block obtaining module is further configured to:

[0155] When any one task processing node in the task processing cluster is restarted, the any one task processing node reads a data block list in the external storage, and loads a required data block from the external storage based on the data block list to perform data recovery.

[0156] It should be noted that other corresponding descriptions of the various functional units involved in the large model inference device based on multi-level cache provided in the embodiments of the present application can be referred to the corresponding descriptions in the method, which will not be repeated here. Figures 1 to 2

[0157] ​The embodiments of the present application also provide a computer device, which can be a personal computer, a server, a network device, etc. The computer device comprises a bus, a processor, a memory and a communication interface, and can further comprise an input / output interface and a display device. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store location information. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement the steps in the method embodiments.

[0158] Those skilled in the art can understand that the structure of the computer device described above is only part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. Specifically, the computer device can comprise more or fewer components, or combine certain components, or have a different component arrangement.

[0159] In one embodiment, a computer readable storage medium is provided, which can be non-volatile or volatile, and has a computer program stored thereon. The computer program is executed by the processor to implement the steps in the method embodiments described above.

[0160] In one embodiment, a computer program product is provided, which comprises a computer program. The computer program is executed by the processor to implement the steps in the method embodiments described above.

[0161] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0162] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0163] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0164] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A large model inference method based on multi-level caching, characterized in that, The method includes: Receive a reasoning task, perform fixed-length word sequence segmentation on the reasoning task to determine at least one word sequence corresponding to the reasoning task, wherein the length of the word sequences other than the last word sequence in the word sequence is the fixed length, and the length of the last word sequence is less than or equal to the fixed length; For a fixed-length word sequence, the target data block corresponding to the word sequence is queried in the video memory. If it does not exist in the video memory, the target data block corresponding to the word sequence is queried in the memory. When the data block in the video memory is evicted from the video memory, it is copied to the memory. If the last word sequence is less than the fixed length, then the inference task is performed using a large model based on the target data block and the last word sequence. If the final word sequence is equal to the fixed length, then the inference task is performed based on the target data block using a large model.

2. The method according to claim 1, characterized in that, The method further includes: In the absence of a target data block corresponding to a fixed-length lexical sequence, the intermediate calculation results of the lexical sequence are obtained during the execution of the inference task by the large model, and a data block is generated based on the lexical sequence and the intermediate calculation results and stored in the video memory. If the final word sequence is less than the fixed length, then the inference task is performed using a large model based on the target data block and the final word sequence, including: If the last word sequence is less than the fixed length, then the inference task is performed using a large model based on the intermediate calculation results in the target data block and the last word sequence. If the last word sequence is equal to the fixed length, then the inference task is performed using a large model based on the target data block, including: If the final word sequence is equal to the fixed length, then the inference task is performed using a large model based on the intermediate calculation results in the target data block.

3. The method according to claim 2, characterized in that, The method further includes: The number of associated requests for the data blocks in the video memory is counted, where the number of associated requests represents the current number of requests for the corresponding data block; When the number of associated requests for any first data block in the video memory is zero, the eviction mechanism of the first data block is triggered, releasing the first data block from the video memory and copying it to the main memory.

4. The method according to claim 3, characterized in that, The data block includes key-value pairs of corresponding word sequences, where the key information of the key-value pair is a unique identifier of the word sequence, and the value information of the key-value pair is an intermediate calculation result of the word sequence; After the data block generated based on the lexical sequence and the intermediate calculation results is stored in the video memory, the method further includes: The metadata in the memory is updated according to the generated data blocks, wherein the metadata includes a unique identifier of the lexical sequence corresponding to each data block in the video memory and the memory, and a data block location identifier, the data block location identifier being used to indicate whether the corresponding data block exists in the video memory.

5. The method according to claim 4, characterized in that, The step of querying the target data block corresponding to the word sequence in the video memory, and if it does not exist in the video memory, then querying the target data block corresponding to the word sequence in the memory, includes: A unique identifier is calculated for the word sequence, and when the unique identifier calculation result of the word sequence matches the unique identifier in the metadata, the data block location identifier corresponding to the unique identifier calculation result in the metadata is used to query whether there is a target data block in the video memory that corresponds to the word sequence; If the target data block corresponding to the word sequence exists in the video memory, then the target data block corresponding to the word sequence is obtained from the video memory. If the target data block corresponding to the word sequence is not present in the video memory, then the target data block is queried from the memory.

6. The method according to any one of claims 1 to 5, characterized in that, After querying the target data block corresponding to the lexical sequence in memory, the method further includes: If the target data block corresponding to the lexical sequence is not found in the memory, the target data block is queried in the external storage, wherein the external storage includes data blocks copied from the memory.

7. The method according to claim 6, characterized in that, The method further includes: The number of copy threads is counted for the data blocks in the memory, where the number of copy threads represents the current number of threads copying the corresponding data block to the video memory; When the number of copy threads for any second data block in the memory is zero, the second data block is released from the memory and copied to the external storage.

8. The method according to claim 6, characterized in that, After querying the target data block corresponding to the lexical sequence in external storage, the method further includes: If the target data block corresponding to the lexical sequence exists in the external storage, the target data block in the external storage is copied to the video memory using GDR switching technology, and the data block location identifier of the metadata corresponding to the target data block in the memory is updated based on the target data block.

9. The method according to claim 6, characterized in that, The method further includes: The number of associated requests for the data blocks in the video memory is counted, where the number of associated requests represents the current number of requests for the corresponding data block; When the number of associated requests for any third data block in the video memory exceeds the preset number of hot data requests, the third data block is copied to the external storage.

10. The method according to claim 9, characterized in that, After receiving the inference task, the method further includes: The inference task is assigned to one of the multiple task processing nodes within the task processing cluster; The counting of associated request counts for data blocks in the video memory includes: Count the number of associated requests for data blocks in the video memory of each task processing node; When the number of associated requests for any third data block in the video memory exceeds the preset number of hot data requests, the third data block is copied to the external storage, including: When the number of associated requests for any third data block in the video memory of any task processing node exceeds the preset number of hot data requests, the third data block is copied to the external storage.

11. The method according to claim 10, characterized in that, The method further includes: Each task processing node detects hot data in the external storage. When it detects new hot data in the external storage, it loads the hot data into the video memory of the task processing node. The hot data includes data blocks whose associated request number is greater than the preset hot data request number.

12. The method according to claim 6, characterized in that, The method further includes: When any task processing node in the task processing cluster restarts, the task processing node reads the list of data blocks in the external storage and loads the required data blocks from the external storage based on the list of data blocks to perform data recovery.

13. A large model inference device based on multi-level caching, characterized in that, The device includes: A word sequence segmentation module is used to receive a reasoning task and perform fixed-length word sequence segmentation processing on the reasoning task to determine at least one word sequence corresponding to the reasoning task. The fixed length is the length of the word sequences other than the last word sequence in the word sequence, and the length of the last word sequence is less than or equal to the fixed length. The data block acquisition module is used to query the target data block corresponding to the word sequence in the video memory for a fixed-length word sequence. If the target data block does not exist in the video memory, it is queried in the memory. The data block in the video memory is copied to the memory when it is evicted from the video memory. The task reasoning module is used to perform the reasoning task based on the target data block and the final word sequence using a large model if the final word sequence is less than the fixed length; and to perform the reasoning task based on the target data block using a large model if the final word sequence is equal to the fixed length.

14. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 12.

15. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 12.

Images