Expert model reasoning system and reasoning method based on RRAM in-memory computing

Abstract

This application provides an expert model inference system and method based on RRAM in-memory computation, applicable to the field of artificial intelligence technology. The system includes a GPU client, a communication control module, and an RRAM server. The GPU client performs attention mechanism computation on the input sequence to obtain activation vectors; performs expert routing computation to obtain a target expert index; and encapsulates the activation vectors and target expert indexes into a task to be processed. The shared buffer of the communication control module stores the task to be processed. The RRAM server includes an RRAM in-memory computation array and a controller; the RRAM in-memory computation array has a mapping relationship with multiple expert modules. The controller reads the task to be processed from the shared buffer, controls the RRAM in-memory computation array to perform operations on the activation vectors, and obtains the inference result. The shared buffer also stores the inference result for the GPU client to read. This system improves inference efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an expert model reasoning system and method based on RRAM in-memory computing. BACKGROUND

[0002] In recent years, large language models (LLM) and sparse expert models (Mixture of Experts, MoE) have become important directions for the development of artificial intelligence. The MoE model activates only part of the expert modules through a routing mechanism to improve parameter utilization and model size. However, in the reasoning process of the MoE model, the expert matrix multiplication calculation is large, and the GPU execution of matrix multiplication is limited by the bottleneck of accessing memory, resulting in low reasoning efficiency. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide an expert model reasoning system and method based on RRAM in-memory computing to improve the reasoning efficiency of the system.

[0004] In a first aspect, the embodiments of the present application provide an expert model reasoning system based on RRAM in-memory computing, which includes a GPU client, a communication control module and an RRAM server, and the GPU client is in communication connection with the RRAM server through the communication control module;

[0005] The GPU client is configured to perform attention mechanism calculation on an input sequence to obtain an activation vector of the input sequence, and perform expert routing calculation to obtain a target expert index, and encapsulate the activation vector and the target expert index as a to-be-processed task.

[0006] The communication control module includes a shared buffer, and the shared buffer is configured to store the to-be-processed task.

[0007] The RRAM server includes an RRAM in-memory computing array and a controller, and the RRAM in-memory computing array has a mapping relationship with a plurality of expert modules; the controller is configured to read the to-be-processed task from the shared buffer and control the RRAM in-memory computing array to perform operation on the activation vector to obtain a reasoning result.

[0008] The shared buffer is further configured to store the reasoning result for the GPU client to read the reasoning result.

[0009] The embodiments of the present application perform attention calculation and expert routing calculation through the GPU client, perform matrix multiplication calculation of the expert modules through the RRAM server, and realize asynchronous processing of tasks through the shared buffer between the GPU client and the RRAM server. Therefore, the system can improve the reasoning efficiency.

[0010] In a possible implementation of the first aspect, the shared buffer comprises a state identifier, and the state corresponding to the state identifier comprises an idle state, a writing state, a ready state, a computing state, and a completion state.

[0011] The idle state is used to represent that there is no to-be-processed task and inference result in the shared buffer.

[0012] The writing state is used to represent that the GPU client is writing the to-be-processed task to the shared buffer.

[0013] The ready state is used to represent that the GPU client has completed the writing of the to-be-processed task.

[0014] The computing state is used to represent that the RRAM server has read the to-be-processed task from the shared buffer and is performing inference calculation on the to-be-processed task.

[0015] The completion state is used to represent that the RRAM server writes the inference result of the inference calculation to the shared buffer.

[0016] In the embodiment of the application, the GPU client and the RRAM server perform data reading and processing on the read data through the state identifier of the shared buffer, thereby realizing asynchronous pipeline processing and improving inference efficiency.

[0017] In a possible implementation of the first aspect, the number of to-be-processed tasks is a plurality, and the controller is specifically configured to merge the plurality of to-be-processed tasks based on the target expert index.

[0018] In the embodiment of the application, the plurality of to-be-processed tasks are merged based on the target expert index, so that the same expert module can process multiple to-be-processed tasks at a time, which enables more array units to work simultaneously, improves the utilization rate of the RRAM in-memory array, and reduces the energy consumption of frequent weight loading.

[0019] In a possible implementation of the first aspect, the controller is further configured to program the expert module weight, perform shift-add accumulation on the result output by the RRAM in-memory computing array, and compensate for the conductance drift by using a calibration parameter.

[0020] In the embodiment of the application, the mapping relationship between the expert module and the RRAM in-memory computing array is established by programming the expert module weight, and the conductance drift is compensated for by using a calibration parameter, thereby improving the accuracy of inference.

[0021] In a possible implementation of the first aspect, the controller is further configured to trigger a fallback mechanism to make the GPU client process the to-be-processed task in a digital calculation manner in a case where an abnormality of the RRAM in-memory computing array is monitored.

[0022] The embodiments of the present application monitor the RRAM in-memory computing array, and when the RRAM in-memory computing array is abnormal, a fallback mechanism is triggered to let the GPU client process the pending task, thereby ensuring the accuracy of inference.

[0023] In a possible implementation of the first aspect, the controller is further configured to record access frequencies of the expert modules, and copy an expert module with an access frequency greater than a first preset frequency to the multiple RRAM in-memory computing array copies.

[0024] The embodiments of the present application copy the expert module with the access frequency greater than the first preset frequency to the multiple RRAM in-memory computing array copies, thereby achieving load balancing and live migration.

[0025] In a possible implementation of the first aspect, the controller is further configured to record access frequencies of the expert modules, and set an expert module with an access frequency less than a second preset frequency to a sleep mode; the second preset frequency is less than the first preset frequency.

[0026] The embodiments of the present application sleep the expert module with the low access frequency, thereby reducing power consumption.

[0027] In a possible implementation of the first aspect, the RRAM in-memory computing card further includes a digital-to-analog conversion module and an analog-to-digital conversion module; the digital-to-analog conversion module is configured to convert the activation vector into a voltage signal; and the analog-to-digital conversion module is configured to convert the inference result in the form of an analog voltage signal output by the RRAM in-memory computing array into an inference result in the form of a digital signal.

[0028] The embodiments of the present application convert data through the digital-to-analog conversion module and the analog-to-digital conversion module, so that the CPU client and the RRAM server can both recognize and process the data written to the shared buffer by the other party.

[0029] In a possible implementation of the first aspect, the GPU client and the RRAM server implement a task pipeline through a state machine mechanism of a shared buffer, wherein the GPU client performs attention calculation and expert routing calculation, and the RRAM server performs in-memory matrix multiplication calculation.

[0030] The embodiments of the present application implement a task pipeline through a state machine mechanism of a shared buffer, so that the CPU client and the RRAM server can asynchronously and in parallel process tasks, thereby improving the efficiency of system inference.

[0031] In a possible implementation of the first aspect, the communication control module adopts a GPUDirect Async asynchronous communication protocol, and the GPU client and the RRAM server perform data copy-free transmission through the GPUDirect Async asynchronous communication protocol.

[0032] The embodiments of the present application make the GPU client and the RRAM server perform no-copy transmission of data through the GPUDirect Async asynchronous communication protocol, that is, the RRAM can directly read input data in the GPU display memory and directly write output results back to the display memory without passing through the host CPU or the system memory.

[0033] In this way, communication delay and energy consumption can be significantly reduced, and asynchronous parallel computing is more efficient.

[0034] In a possible implementation of the first aspect, the RRAM server performs matrix multiplication operation on the activation vector in the RRAM in-memory computing array by using a Bit-Slicing and BlockAMC mapping method, to obtain the inference result.

[0035] The embodiments of the present application realize matrix multiplication operation of the expert module by using the RRAM in-memory computing array by using the Bit-Slicing and BlockAMC mapping method, to obtain high-precision matrix calculation.

[0036] In a second aspect, the embodiments of the present application provide an expert model inference method based on RRAM in-memory computing, comprising:

[0037] The GPU client performs attention mechanism calculation on the input sequence to obtain an activation vector of the input sequence, and performs expert routing calculation to obtain a target expert index;

[0038] The GPU client encapsulates the activation vector and the target expert index into a to-be-processed task, and writes the to-be-processed task into a shared buffer of the communication control module;

[0039] The controller in the RRAM server reads the to-be-processed task from the shared buffer, and controls the RRAM in-memory computing array in the RRAM server to perform operation on the activation vector, to obtain an inference result; wherein the RRAM in-memory computing array has a mapping relationship with a plurality of expert modules;

[0040] The controller writes the inference result into the shared buffer;

[0041] The GPU client reads the inference result from the shared buffer.

[0042] The embodiments of the present application perform attention calculation and expert routing calculation by using the GPU client, perform matrix multiplication calculation of the expert module by using the RRAM server, and realize asynchronous processing of tasks by using the shared buffer between the GPU client and the RRAM server, so that the system can improve inference efficiency.

[0043] In a possible implementation of the first aspect, after the to-be-processed task is written into the shared buffer of the communication control module, the method further comprises:

[0044] The GPU client performs attention mechanism and expert routing calculation on the next batch of input sequences.

[0045] In the embodiment of the application, after the GPU client writes the to-be-processed task into the shared buffer, the GPU client does not wait for the RRAM server to return the inference result before starting to calculate the next batch of input sequences, but can directly calculate the next batch of input sequences, thereby improving the inference efficiency.

[0046] In a possible implementation of the first aspect, during the process of writing the to-be-processed task into the shared buffer of the communication control module, the GPU client switches the state of the shared buffer to a write state;

[0047] After completing the writing of the to-be-processed task, the GPU client switches the state of the shared buffer to a ready state.

[0048] In the embodiment of the application, the GPU client and the RRAM server in the embodiment of the application perform data reading and processing on the read data through the state identifier of the shared buffer, thereby realizing asynchronous pipeline processing and improving the inference efficiency.

[0049] In a possible implementation of the first aspect, the controller reads the to-be-processed task from the shared buffer and switches the state of the shared buffer to a calculation state when monitoring that the state of the shared buffer is switched to a ready state;

[0050] After the controller writes the inference result into the shared buffer, the state of the shared buffer is switched to a completion state.

[0051] In the embodiment of the application, after the controller writes the inference result into the shared buffer, the state of the shared buffer is switched to a completion state, which facilitates the CPU client to extract the inference result from the shared buffer in a timely manner.

[0052] In a possible implementation of the first aspect, after the controller reads the to-be-processed task from the shared buffer, the method further includes:

[0053] The controller merges the activation vectors corresponding to the plurality of to-be-processed tasks according to the target expert index to generate a merged activation vector;

[0054] The RRAM in-memory computing array in the RRAM server is controlled to perform operation on the activation vector, including:

[0055] The RRAM in-memory computing array is controlled to perform operation on the merged activation vector.

[0056] This application embodiment merges multiple tasks to be processed based on the target expert index, so that the same expert module can process multiple tasks at once. This allows more array units to work simultaneously, improves the utilization rate of the RRAM in-memory array, and reduces the energy consumption of frequent weight loading.

[0057] In one possible implementation of the first aspect, the method further includes, before obtaining the reasoning result:

[0058] If the controller detects an anomaly in the RRAM in-memory computing array, a rollback command is sent to the GPU client.

[0059] After receiving the rollback instruction, the GPU client processes the task to be processed using numerical calculations.

[0060] This application embodiment monitors the RRAM in-memory computing array. When the RRAM in-memory computing array malfunctions, a rollback command is sent to the GPU client, allowing the GPU client to handle the pending tasks, thereby ensuring the accuracy of inference.

[0061] In one possible implementation of the first aspect, the method further includes:

[0062] The controller performs bit-slicing on the weight matrix corresponding to each expert module and programs the segmented weight matrix into the RRAM memory computing array according to the conductance value.

[0063] The RRAM in-memory computing array is calibrated using a write-verify mechanism to obtain an RRAM in-memory computing array that meets the accuracy requirements.

[0064] This application embodiment improves the computational accuracy of the RRAM in-memory computing array by combining bit-slicing and write-verify mechanisms.

[0065] Thirdly, embodiments of this application provide a non-transitory computer-readable storage medium, comprising:

[0066] A non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform methods in various possible implementations of the second aspect.

[0067] Fourthly, embodiments of this application provide a computer program product, including computer program instructions, which are read and executed by a processor to perform the methods in various possible implementations of the second aspect.

[0068] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0069] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 A schematic diagram of an expert model inference system architecture based on RRAM in-memory computation is provided for an embodiment of this application;

[0071] Figure 2 This is a schematic diagram of another inference system architecture provided in an embodiment of this application;

[0072] Figure 3 An interaction diagram of an expert model-based reasoning method provided in an embodiment of this application. Detailed Implementation

[0073] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0075] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0076] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0077] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0078] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0079] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0080] In recent years, Large Language Models (LLM) and Sparse Expert Models (MoE) have become important directions in the development of artificial intelligence. MoE is a neural network architecture that includes multiple expert modules, each specializing in processing a specific type or pattern of data. During inference, a gate controller or router dynamically selects and activates a small number of relevant expert modules for each input.

[0081] With the rapid development of AI large language model inference devices and the rapid growth in societal demand, the deployment of edge AI large language model inference devices faces numerous difficulties and challenges. For example, as edge large models develop rapidly, the number of parameters increases with iteration, leading to a corresponding increase in the computational load of expert matrix multiplication and a continuous increase in bandwidth requirements. Therefore, edge AI large language models currently suffer from low inference efficiency.

[0082] To address the aforementioned technical problems, this application provides an expert model inference system based on RRAM in-memory computation. This system implements matrix multiplication operations of the expert model on the RRAM server, while attention mechanism computation and expert routing computation are performed on the GPU client. Data transmission between the GPU client and the RRAM server is achieved through a shared buffer, enabling efficient inference of large model expert modules. It should be noted that the expert model inference system based on RRAM in-memory computation in this application embodiment can be simply referred to as an inference system or a system.

[0083] For ease of understanding, the relevant terms used in the embodiments of this application are explained below:

[0084] Resistive Random Access Memory (RRAM) utilizes the property that certain special materials (such as metal oxides) undergo reversible resistance changes under an applied voltage. Its core is a Metal-Insulator-Metal (M-IM) sandwich structure. By applying a voltage pulse to the insulating layer, conductive filaments can be formed or broken, thus reversibly switching between a high-resistivity state (HRS, representing 0) and a low-resistivity state (LRS, representing 1).

[0085] In-memory computing: This involves embedding computing functions directly into storage units or storage arrays. Simply put, it means performing calculations where the data is stored, thereby reducing the need for data movement.

[0086] In the RRAM in-memory computation matrix, word lines and bit lines intersect perpendicularly, with an RRAM device placed at each intersection. In this embodiment, the matrix multiplication algorithm for the expert modules can be calculated by mapping the RRAM in-memory computation matrix to the expert module weights. Specifically, the expert module weights are stored in the conductance values ​​of the RRAM in-memory computation matrix. The input sequence is applied to the word lines as voltages. According to Kirchhoff's laws, the current flowing through each bit line is equal to the sum of the products of the voltages on all connected word lines and the corresponding RRAM conductances. The current values ​​measured on the bit lines are the result of matrix-vector multiplication.

[0087] Bit-slicing is a data-level parallel programming technique. Its basic idea is to decompose a high-precision numerical operation into multiple low-precision sub-operations, and then recover the final result through parallel processing and combination. In this embodiment, bit-slicing is used to segment the weights of the expert module.

[0088] BlockAMC breaks down the computational tasks of large models into smaller, more manageable blocks, then processes these blocks independently or in parallel, and finally merges the results.

[0089] Therefore, bit-slicing is a decomposition of the precision dimension, while BlockAMC is a decomposition of the scale dimension. In this embodiment, bit-slicing and BlockAMC are combined to solve the problems of numerical precision and computational scale at the same time.

[0090] GPUDirect Async protocol: It implements a true direct communication path: GPU memory - direct DMA - external device, which not only eliminates copying, but also achieves fully asynchronous operation, with the CPU hardly participating in the data transfer process.

[0091] Expert-as-a-Service (EaaS) encapsulates each expert module in the MoE model into an independent service that can be remotely invoked over a network. In this embodiment, for the expert model under the EaaS model, each expert module is deployed as an independent service instance, which can be distributed across different servers. The router acts as a client, using the expert service via remote network invocation.

[0092] In this embodiment, the router is located on the GPU client, and the expert module is located on the RRAM server.

[0093] Figure 1 A schematic diagram of an expert model inference system architecture based on RRAM in-memory computation is provided for embodiments of this application, as shown below. Figure 1 As shown, the system includes a GPU client 101, a communication control module 102, and an RRAM server 103. The GPU client 101 communicates with the RRAM server 103 through the communication control module 102.

[0094] The GPU client 101 is used to perform attention mechanism calculations on the input sequence to obtain the activation vector of the input sequence; and to perform expert route calculations to obtain the target expert index; and to encapsulate the activation vector and the target expert index into a task to be processed. In addition, the GPU client 101 can also preprocess the user input, for example, by processing the user input through an embedding layer, so that the obtained input sequence can be recognized and processed by the subsequent attention mechanism module. The attention mechanism layer performs attention calculations on the input sequence to obtain the context representation of the input sequence, i.e., the activation vector.

[0095] Each expert module has a unique expert index. The target expert index refers to the expert index of the expert module selected from multiple expert modules after expert routing calculation, which is used to process each token in the input sequence. One or more expert modules can be selected, depending on pre-configuration requirements.

[0096] After obtaining the activation vector and the target expert index, they can be packaged into a task to be processed. Understandably, parameters such as a task identifier can also be added to the task to be processed.

[0097] The communication control module 102 includes a shared buffer used to store tasks to be processed. The buffer typically resides in a high-bandwidth memory space shared by the GPU and the RRAM server, mapping the same physical address region between the two nodes via GPUDirect RDMA or NVLink technology. Physically, the shared buffer can be housed in GPU-side video memory (HBM or GDDR), created by the GPU driver, while the FPGA or CPU of the RRAM server directly accesses this video memory region via a bus (such as InfiniBand). This design avoids traditional host memory intermediaries, enabling the GPU and RRAM hardware to directly read and write to the same buffer, achieving high-speed asynchronous communication.

[0098] RRAM server 103 includes at least one RRAM in-memory compute card, which comprises an RRAM in-memory compute array and a controller. The RRAM in-memory compute array is mapped to multiple expert modules. Specifically, the conductance values ​​at each intersection of word lines and bit lines in the RRAM in-memory compute array are programmed as weights for the expert modules. The controller can be a GPU or an FPGA. The controller reads tasks to be processed from a shared buffer and controls the RRAM in-memory compute array to perform operations on activation vectors to obtain inference results. The controller writes the inference results to the shared buffer. Therefore, the shared buffer is also used to store the inference results for the GPU client to read.

[0099] It should be noted that in practical applications, the number of GPU clients communicating with the RRAM server can be one or more.

[0100] In this embodiment, attention calculation and expert routing calculation are performed by a GPU client, and matrix multiplication calculation of the expert module is performed by an RRAM server. The GPU client and RRAM server achieve asynchronous processing of tasks through a shared buffer. Therefore, the system can improve inference efficiency.

[0101] Based on the above embodiments, the shared buffer may include multiple slots, each slot including a status area, a header information area, and a payload area. The status area can represent the status of the corresponding slot through a status identifier, which includes idle, write, ready, compute, and complete states. When there is no data in the shared buffer for the GPU client and RRAM server to read, its status is idle. When the GPU client writes a task to be processed into the shared buffer, the status of the corresponding slot in the shared buffer switches from idle to write. After the GPU client completes writing the task, the status of the slot switches from write to ready. The controller in the RRAM server periodically polls the status of the shared buffer. When it detects a slot in the shared buffer that is ready, it reads the task to be processed from the shared buffer and switches the status of the shared buffer to compute. After the RRAM server completes the computation and obtains the inference result, the controller writes the inference result into the shared buffer and switches the status of the slot from compute to complete. When the GPU client detects a slot in the shared buffer that is complete, it reads the inference result from the shared buffer and switches its status back to idle. It should be noted that the controller can also monitor the status of the shared buffer in real time. When the status of the shared buffer is detected as ready, the controller reads the tasks to be processed from the shared buffer and switches the status of the shared buffer to the computing state.

[0102] The GPU client and RRAM server implement a task pipeline through a state machine mechanism with a shared buffer. While the GPU client performs attention calculations and expert routing calculations, the RRAM server performs in-memory matrix multiplication calculations.

[0103] In this embodiment, the GPU client and RRAM server read and process data through a shared buffer status identifier, achieving asynchronous pipelined processing and improving inference efficiency.

[0104] Based on the above embodiments, there are multiple tasks to be processed; the controller is specifically used to merge multiple tasks to be processed based on the target expert index.

[0105] In the specific implementation process, when the controller reads multiple tasks to be processed from the shared buffer, the corresponding target expert index can be extracted from each of the multiple tasks to be processed. Then, the tasks to be processed that belong to the same target expert index can be merged. Specifically, their activation vectors can be merged and combined into a matrix. The corresponding expert module can then perform operations on the matrix to obtain the inference result.

[0106] For example, the controller reads task A, task B, and task C from the shared buffer. Task A has target expert indices of 001 and 010, task B has 010 and 011, and task C has 111. Since both task A and task B have target expert index 010, the activation vectors of task A and task B can be combined to form a matrix, which is then processed by the expert module with target expert index 010. When combining two or more activation vectors, they can be concatenated to form a matrix. The expert module with target expert index 001 processes the activation vector of task A, the expert module with target expert index 011 processes the activation vector of task B, and the expert module with target expert index 111 processes the activation vector of task C.

[0107] In actual inference, some expert modules may receive a large number of pending tasks. In this case, they can be processed by activating the array in multiple rounds. Of course, some expert modules may also receive fewer pending tasks. In this case, depending on the business latency requirements, the module can choose to compute immediately or wait for more tasks to start computing, thus achieving a balance between latency and throughput.

[0108] This application embodiment merges multiple tasks to be processed based on the target expert index, so that the same expert module can process multiple tasks at once. This allows more array units to work simultaneously, improves the utilization rate of the RRAM in-memory computing array, and reduces the energy consumption of frequent weight loading.

[0109] Based on the above embodiments, the controller is also used to program the weights of the expert module, perform shift-add accumulation on the results output by the RRAM in-memory computing array, and compensate for conductance drift using calibration parameters.

[0110] In the specific implementation process, during the system power-on and initialization phase, the RRAM server can map the expert module weights to the RRAM in-memory computing arrays. Specifically, the controller can perform bit-slicing to divide the weight matrix corresponding to each expert module into blocks, that is, map the expert module weight matrix to a bit plane. Assuming that the data type of the weight matrix in the expert module is an 8-bit integer with a shape of (4096, 4096), the 8-bit integer weight matrix is ​​decomposed into 8 binary (1-bit) matrices, each matrix being called a bit plane. The 8 decomposed binary matrices are then mapped to 8 independent RRAM in-memory computing arrays. The mapping rule is: if the value at a certain position in the binary matrix is ​​1, then the corresponding RRAM in-memory computing array becomes a high-conductivity state; if the value is 0, then it becomes a low-conductivity state.

[0111] After programming the expert module weights, an array mapping table is generated. This array mapping table includes: the physical location (row and column address) of each weight in the array, the bit-slice level to which the weight belongs, the target conductivity value and the actual written conductivity value, the cell health status (such as whether it is a bad cell or a drift anomaly), the required digital compensation parameters (such as scaling factor α, offset β) and the corresponding temperature calibration factor.

[0112] The system also inputs a set of standard test vectors after writing, and automatically derives calibration coefficients (such as scaling, offset, temperature compensation factor, etc.) by comparing the calculation results of the RRAM in-memory computing array with the ideal values. This information ultimately forms a complete array mapping table and calibration parameter table for accurate compensation in subsequent inference processes. It should be noted that the system performs a lightweight digital calibration after each array calculation to compensate for small drifts. When the conductance change or output error exceeds a preset threshold, a recalibration mechanism such as reprogramming or updating the calibration table is triggered to ensure long-term accuracy stability while maintaining throughput.

[0113] During the inference phase, after the RRAM in-memory computing array performs operations on the activation vector, it can perform shift-add accumulation on the output results to restore the full data precision.

[0114] The controller can also use correction parameters to compensate for conductance drift in the output of the RRAM in-memory computing array, thereby improving the accuracy of the inference results.

[0115] This application embodiment establishes a mapping relationship between expert modules and RRAM in-memory computing array by programming the weights of expert modules, and improves the accuracy of inference by using calibration parameters to compensate for conductance drift.

[0116] Based on the above embodiments, the controller can also be used to monitor the status of the RRAM in-memory computing array. If an abnormality is detected in the RRAM in-memory computing array, a rollback command is sent to the GPU client. The rollback command is used to enable the GPU client to process the task to be processed using digital computing.

[0117] In the specific implementation process, the controller continuously monitors the health status of the RRAM in-memory computing array. If the health status is abnormal, such as conductance drift, noise, or computational errors exceeding a threshold, a rollback mechanism is triggered. Specifically, a rollback command can be sent to the GPU client, or a rollback function can be called, allowing the GPU client to process the task using digital computation methods and complete the inference of the task. Therefore, this inference system has two task inference channels. The RRAM server is responsible for efficient task inference computation. When the RRAM server malfunctions, the GPU client can act as a fallback, thereby ensuring the accuracy of the computation and realizing an adaptive inference mechanism where the computation result is recoverable and the service is uninterrupted.

[0118] The controller monitors the health status of the RRAM in-memory computing array by querying the array mapping table and calibration parameters recorded by the controller, and by receiving the real-time calculation error reported by the controller.

[0119] For example, its specific workflow includes: when performing online calibration or specific diagnostic tasks, the controller compares the deviation between the monitored conductance drift value or calculated result and the ideal value with a preset threshold; when the deviation exceeds the limit, a decision is made according to a predefined strategy (such as the ID of the anomaly expert module, the urgency of the task), and then a specific "rollback status flag" is written into the status identifier area of ​​the shared buffer, or a rollback command is sent directly to the GPU client through the GPUDirect Async communication path. After the GPU client polls for the "rollback status flag" or receives the rollback command, it triggers the local digital computation rollback path.

[0120] This application embodiment monitors the RRAM in-memory computing array. When the RRAM in-memory computing array malfunctions, a rollback mechanism is triggered, allowing the GPU client to handle pending tasks, thereby ensuring the accuracy of inference and the continuity of service.

[0121] Based on the above embodiments, the controller is also used to record the access frequency of each expert module and copy the expert modules with access frequencies greater than a first preset frequency to multiple RRAM in-memory computing array copies.

[0122] In practical implementation, the access frequency of an expert module refers to the number of tasks processed by the expert module within a preset time period. If the access frequency exceeds a first preset frequency, the expert module is considered to have a high access frequency. To achieve load balancing, the expert module can be replicated to multiple RRAM memory compute array replicas. Multiple expert modules can execute the inference operations of the tasks to be processed in parallel.

[0123] If the access frequency is less than the second preset frequency, the access frequency of the expert module is considered low. In order to reduce energy consumption, these expert modules can be temporarily put into a dormant state. When a task requires these expert modules to perform inference calculations, they can be activated again.

[0124] It should be noted that the second preset frequency is less than the first preset frequency, and the specific values ​​of the first preset frequency and the second preset frequency can be set according to the actual situation.

[0125] This application embodiment achieves load balancing and hot migration by copying expert modules with access frequencies higher than a first preset frequency to multiple RRAM in-memory computing array replicas, and puts expert modules with low access frequencies into hibernation, thereby reducing power consumption.

[0126] Based on the above embodiments, the RRAM server further includes a digital-to-analog conversion module and an analog-to-digital conversion module; wherein the digital-to-analog conversion module is used to convert the activation vector in the task to be processed into a voltage signal; and the analog-to-digital conversion module is used to convert the inference result in the form of an analog voltage signal output by the RRAM in-memory computing array into the inference result in the form of a digital signal.

[0127] In the specific implementation process, since the data format processed by the GPU client is in digital signal form, while the data format processed by the RRAM in-memory computing array is in analog voltage signal form, the RRAM server uses a digital-to-analog converter (DAC) module and an analog-to-digital converter (ADC) module to convert the data format. Specifically, after the controller reads the task to be processed from the shared buffer, the DAC module converts the activation vector in the task to an analog voltage signal and loads it into the RRAM in-memory computing array, initiating the memory matrix multiplication calculation. After the RRAM in-memory computing array obtains the inference result, the ADC module converts the inference result from analog voltage signal form to digital signal form.

[0128] Based on the above embodiments, the communication control module adopts the GPUDirect Async asynchronous communication protocol, and the GPU client and RRAM server perform copyless data transmission through the GPUDirect Async asynchronous communication protocol.

[0129] In practical implementation, based on the GPUDirect Async asynchronous communication protocol, the RRAM server can directly read input data from the GPU memory and write inference results back to the GPU memory, without needing to transfer data through the host CPU or system memory. This significantly reduces communication latency and makes asynchronous parallel computing more efficient.

[0130] Based on the above embodiments, the RRAM server uses the Bit-Slicing and BlockAMC mapping method to perform matrix multiplication operations on the activation vector within the RRAM memory computing array to obtain the inference result.

[0131] In the specific implementation, Bit-Slicing decomposes the expert module weights along the accuracy dimension, while BlockAMC decomposes them along the scale dimension. During the initialization phase, the trained expert module weight matrix can be simultaneously decomposed using both Bit-Slicing and BlockAMC, and the resulting binary block weights are programmed onto the corresponding RRAM in-memory computation array. During the inference computation phase, after receiving the task, the RRAM server divides the activation vector into blocks according to Bit-Slicing and BlockAMC, and broadcasts these blocks to the RRAM in-memory computation array. Each block on the RRAM in-memory computation array computes its own bit-plane block results in parallel, the local digital logic reconstructs the bit-plane results, and the results from all blocks are globally aggregated to obtain the inference result.

[0132] This application embodiment uses the Bit-Slicing and BlockAMC mapping method to enable the expert module to perform matrix multiplication operations using the RRAM in-memory computing array, thereby obtaining high-precision matrix calculations.

[0133] Figure 2 Another inference system architecture diagram is provided for embodiments of this application, such as... Figure 2As shown, the inference system architecture includes a GPU-side attention computation module, an EaaS asynchronous communication layer, and an RRAM in-memory computation expert module. The GPU is responsible for executing the embedding layer, self-attention layer, and expert routing computation layer (Router) in the Transformer, used to generate the activation vector of the token and expert selection weights, and to determine the target expert index based on the expert selection weights. The EaaS asynchronous communication layer achieves high-speed transmission of activation vectors and inference results through a shared buffer and the GPUDirect asynchronous communication mechanism. The RRAM hardware module undertakes the feedforward network computation tasks of the expert module, using in-memory matrix multiplication (MVM) to achieve energy-efficient linear transformation. After receiving the Router allocation, the GPU client sends the activation data and the target expert index to the RRAM server. The RRAM server completes the matrix multiplication and nonlinear activation and returns the inference result, which the GPU then performs fusion and subsequent layer computations.

[0134] In this architecture, the GPU client and RRAM server work asynchronously. The RRAM server provides massively parallel and energy-efficient expert computing capabilities, while the GPU client is responsible for dynamic control and high-precision calculations, achieving hardware-software complementarity and computational power stratification. This system significantly reduces computational redundancy and energy consumption in large model inference and features EaaS service characteristics that enable scalable experts and migrated nodes, making it suitable for efficient deployment of distributed MoE inference.

[0135] This application also provides a reasoning method based on the reasoning system of any of the above embodiments, such as... Figure 3 As shown, the asynchronous communication and computation process of the inference system mainly includes a GPU client, a communication control module, and an RRAM server. The RRAM server includes an RRAM expert server and an RRAM in-memory computing card, and the RRAM in-memory computing card includes an RRAM in-memory computing matrix.

[0136] The complete process of the inference method executed by the inference system includes: system deployment, initialization, activation generation, asynchronous task transmission, RRAM in-memory computation, result return, and asynchronous recycling. Each step is described below. Before executing step 301, the inference system can be pre-deployed and initialized. Specifically, the RRAM server maps the expert module weights to the RRAM in-memory computation array. Specifically, the controller performs bit-slicing on the weight matrices W1W_1W1 and W2W_2W2 corresponding to each expert module, programs them into the RRAM array cells according to their conductance values, and repeatedly calibrates them using a write-verify mechanism until the expected accuracy is achieved. The controller establishes an array mapping table and calibration parameters for compensating for drift errors in subsequent calculations. After array initialization, the RRAM server starts temperature monitoring and online calibration processes on the controller to ensure the stability and long-term reliability of in-memory matrix multiplication (MVM) calculations. After completing the above deployment and initialization work, the inference task can begin execution, as follows:

[0137] Step 301: Write activation data; Upon entering the inference phase, when the Transformer model execution value includes layers with a MoE structure, the GPU client performs attention mechanism calculations on the input sequence to obtain the activation vector; and performs expert routing calculations to obtain the expert weight corresponding to each token, determining the target expert index based on these expert weights. The GPU client encapsulates the obtained activation vector and target expert index into a task to be processed, and writes the task to be processed as activation data into the shared buffer. It is understood that the task to be processed may also include a task identifier.

[0138] A shared buffer consists of multiple slots, each comprising a state area, a header area, and a payload area. The state area characterizes the current state of the shared buffer; it does not store actual data but rather metadata used to coordinate and manage the shared buffer's state. The shared buffer's state can include: idle, write, ready, computation, and completed. The header area describes the metadata of the data stored in the payload area. The payload area stores the actual data; tasks to be processed are stored in the payload area.

[0139] Step 302: Update the state to ready; During the process of writing the task to be processed into the shared buffer, the GPU client switches the state of the corresponding slot from idle to write. After completing the writing of the task, the state of the slot is switched from write to ready. The GPU client then processes the attention mechanism calculation and expert routing calculation of the subsequent input sequence, thus forming an asynchronous pipeline mechanism between the GPU client and the RRAM server.

[0140] Step 303: Poll the status of the shared buffer; The controller in the RRAM server periodically polls the status of the shared buffer. When a ready state is detected, it reads the task to be processed from the shared buffer and switches the status of the shared buffer from the ready state to the compute state to prevent the task to be processed from being used repeatedly.

[0141] Step 304: Submit batch tasks: After the RRAM server reads the tasks to be processed, if there are multiple tasks to be processed, it will aggregate the tasks to be processed by the same expert module according to the target expert index, extract the activation vectors of the aggregated tasks to be processed, merge them into a matrix, control the FPGA to convert the task input into an analog voltage signal through the digital-to-analog converter module and load it into the RRAM array, and start the in-memory matrix multiplication calculation process.

[0142] Step 305: Perform matrix multiplication calculation; within the RRAM memory, the input voltage acts on the array's word lines, and the array cell conductance and input voltage together determine the accumulated current value on the bit lines. The analog current signal output by the array is converted into a digital quantity by the ADC module, and then passed to the FPGA or CPU for shift-add accumulation to restore the complete numerical accuracy. This process is performed in parallel across multiple bit-slicing slices to ensure high-precision and scalable calculation. The intermediate result h=W1xh=W_1xh=W1x after the first layer of the array is processed by nonlinear activation (such as ReLU or GELU) and then enters the second layer of the array to execute y=W2hy=W_2hy=W2h, obtaining the final output result of the expert module.

[0143] Step 306: Output Results; The RRAM memory calculation card outputs the calculation results.

[0144] Step 307: The RRAM expert server writes back the results; the RRAM expert server performs analog-to-digital conversion and encapsulation on the results output by the RRAM in-memory computing card and writes them back to the payload area of ​​the shared buffer. At this time, the task status is updated to complete.

[0145] Step 308: The GPU client polls until the task is completed; the CPU client's status monitoring thread or streaming polling module continuously monitors the task status in the shared buffer. When a task is completed, the reading process is triggered.

[0146] Step 309: Read the results and continue inference: Read the inference results returned by the RRAM server into the GPU memory without intermediate copying. Then, according to the expert module weights, the results of each expert are weighted and fused to restore the complete MoE layer output. After fusion, the GPU client updates the status of the shared buffer to idle, indicating that the task slot has been reclaimed and can be used for new task allocation.

[0147] The entire process forms a lock-free asynchronous state loop, with the state sequentially going through the cycle of idle state - write state - ready state - compute state - complete state - idle state. Since the GPU and RRAM server each run independent state machine control logic, the execution of both is completely asynchronous and parallel, avoiding global synchronization blocking.

[0148] This asynchronous communication and computation mechanism brings significant system-level optimization effects. On the one hand, the GPU no longer waits for expert output returns when performing attention and routing calculations, while the RRAM continuously performs in-memory matrix multiplications in parallel in the background, thus achieving complete overlap between the front-end and back-end. On the other hand, the shared buffer architecture of the communication layer and GPUDirect Async technology ensure ultra-low latency and high bandwidth for data transmission, eliminating bottlenecks from traditional PCIe intermediaries or CPU memory copying. During system operation, the FPGA automatically monitors the health status and conductance drift of the RRAM array; if an abnormal conductance change is detected during computation, causing the result deviation to exceed the threshold, the control module immediately reports the status and triggers a rollback mechanism, and the GPU automatically executes matrix multiplication operations for the same batch of tasks to ensure the correctness of the calculation results. Through this asynchronous scheduling and redundancy mechanism, the system achieves an optimal balance between energy efficiency and accuracy.

[0149] Throughout the process, the FPGA or CPU on the RRAM expert server side simultaneously handles computation scheduling and array health management. The system periodically measures array conductance changes; when conductance drift or noise exceeds a threshold, it automatically initiates write-back calibration or backup cell switching to ensure long-term accuracy stability. When the error exceeds the tolerable range, the system triggers a GPU rollback computation path via the controller. Specifically, the controller sends a rollback command to the GPU client, which, upon receiving the command, takes over the computation of the pending task and recompiles this part of the inference using digital computation methods. This ensures the correctness of the inference results and service continuity.

[0150] In addition, the inference system also implements a copying and migration mechanism for expert modules. Popular experts with high access frequency are automatically copied to multiple RRAM cards to achieve load balancing and hot migration, while low-frequency experts can temporarily hibernate to reduce power consumption.

[0151] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the methods provided in the above-described method embodiments, such as: a GPU client performing attention mechanism calculation on an input sequence to obtain an activation vector of the input sequence; and performing expert routing calculation to obtain a target expert index; the GPU client encapsulating the activation vector and the target expert index into a task to be processed, and writing the task to be processed into a shared buffer of a communication control module; a controller in an RRAM server reading the task to be processed from the shared buffer, and controlling the RRAM in-memory computing array in the RRAM server to perform calculations on the activation vector to obtain an inference result; wherein the RRAM in-memory computing array has a mapping relationship with multiple expert modules; the controller writing the inference result into the shared buffer; and the GPU client reading the inference result from the shared buffer.

[0152] This embodiment provides a non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute the methods provided in the above-described method embodiments. These instructions include, for example: a GPU client performing attention mechanism calculations on an input sequence to obtain an activation vector; and performing expert routing calculations to obtain a target expert index; the GPU client encapsulating the activation vector and the target expert index into a task to be processed, and writing the task to be processed into a shared buffer of a communication control module; a controller in an RRAM server reading the task to be processed from the shared buffer and controlling the RRAM in-memory computing array in the RRAM server to perform calculations on the activation vector to obtain an inference result; wherein the RRAM in-memory computing array has a mapping relationship with multiple expert modules; the controller writing the inference result into the shared buffer; and the GPU client reading the inference result from the shared buffer.

[0153] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0154] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0156] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0157] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An expert model inference system based on RRAM in-memory computation, characterized in that, It includes a GPU client, a communication control module, and an RRAM server, wherein the GPU client communicates with the RRAM server through the communication control module; The GPU client is used to perform an attention mechanism on the input sequence to calculate the activation vector of the input sequence; And perform expert route calculation to obtain the target expert index; encapsulate the activation vector and the target expert index into a task to be processed; The communication control module includes a shared buffer, which is used to store the tasks to be processed. The RRAM server includes at least one RRAM in-memory computing card, which includes an RRAM in-memory computing array and a controller. The RRAM in-memory computing array is mapped to multiple expert modules. The controller is used to read the task to be processed from the shared buffer and control the RRAM in-memory computing array to perform operations on the activation vector to obtain inference results. The shared buffer is also used to store the inference results for the GPU client to read.

2. The system according to claim 1, characterized in that, The shared buffer includes a status identifier, and the status identifier corresponds to an idle state, a write state, a ready state, a computation state, and a completed state; wherein: The idle state is used to indicate that there are no pending tasks or inference results in the shared buffer; The write status is used to indicate that the GPU client is writing pending tasks to the shared buffer; The ready state indicates that the GPU client has completed writing the pending task; The computation status is used to indicate that the RRAM server has read the task to be processed from the shared buffer and is performing inference computation on the task to be processed; The completion status indicates that the RRAM server has written the inference results of the inference computation to the shared buffer.

3. The system according to claim 1, characterized in that, The number of tasks to be processed is multiple; the controller is specifically used to merge multiple tasks to be processed based on the target expert index.

4. The system according to claim 1, characterized in that, The controller is also used to program the weights of the expert module, perform shift-add accumulation of the results output by the RRAM in-memory computing array, and compensate for conductance drift using calibration parameters.

5. The system according to claim 1, characterized in that, The controller is also used to trigger a rollback mechanism when an anomaly is detected in the RRAM in-memory computing array, so that the GPU client can use digital computing to process the task to be processed.

6. The system according to claim 1, characterized in that, The controller is also used to record the access frequency of each expert module and copy the expert modules with access frequencies greater than a first preset frequency to multiple RRAM in-memory computing array replicas.

7. The system according to claim 1, characterized in that, The controller is also used to record the access frequency of each expert module and set the expert module with an access frequency less than a second preset frequency to sleep mode; the second preset frequency is less than the first preset frequency.

8. The system according to claim 1, characterized in that, The RRAM in-memory computing card further includes a digital-to-analog converter module and an analog-to-digital converter module; the digital-to-analog converter module is used to convert the activation vector into a voltage signal; the analog-to-digital converter module is used to convert the inference result in analog voltage signal form output by the RRAM in-memory computing array into the inference result in digital signal form.

9. The system according to claim 1, characterized in that, The GPU client and RRAM server implement a task pipeline through the state machine mechanism of the shared buffer. While the GPU client performs attention calculation and expert routing calculation, the RRAM server performs in-memory matrix multiplication calculation.

10. The system according to claim 1, characterized in that, The communication control module adopts the GPUDirectAsync asynchronous communication protocol, and the GPU client and the RRAM server perform copyless data transmission through the GPUDirectAsync asynchronous communication protocol.

11. The system according to any one of claims 1-10, characterized in that, The RRAM server uses the Bit-Slicing and BlockAMC mapping method to perform matrix multiplication on the activation vector within the RRAM memory computing array to obtain the inference result.

12. An expert model inference method based on RRAM in-memory computation, characterized in that, include: The GPU client uses an attention mechanism to calculate the activation vector of the input sequence. And perform expert routing calculations to obtain the target expert index; The GPU client encapsulates the activation vector and the target expert index into a task to be processed, and writes the task to be processed into the shared buffer of the communication control module; The controller in the RRAM server reads the task to be processed from the shared buffer and controls the RRAM in-memory computing array in the RRAM server to perform operations on the activation vector to obtain the inference result; wherein, the RRAM in-memory computing array has a mapping relationship with multiple expert modules; The controller writes the inference result into the shared buffer; The GPU client reads the inference results from the shared buffer.

13. The method according to claim 12, characterized in that, After writing the task to be processed into the shared buffer of the communication control module, the method further includes: The GPU client performs attention mechanism and expert routing calculations on the next batch of input sequences.

14. The method according to claim 12, characterized in that, During the process of writing the task to be processed into the shared buffer of the communication control module, the GPU client switches the state of the shared buffer to the write state; After completing the writing of the task to be processed, the GPU client switches the state of the shared buffer to the ready state.

15. The method according to claim 14, characterized in that, When the controller detects that the state of the shared buffer has switched to the ready state, it reads the task to be processed from the shared buffer and switches the state of the shared buffer to the computing state. After the controller writes the inference result into the shared buffer, it switches the state of the shared buffer to the completed state.

16. The method according to claim 12, characterized in that, After the controller reads the task to be processed from the shared buffer, the method further includes: The controller merges the activation vectors corresponding to multiple tasks to be processed according to the target expert index to generate a merged activation vector; The process of controlling the RRAM in-memory computing array in the RRAM server to perform calculations on the activation vector includes: The system controls the RRAM in-memory computing array to perform calculations on the merged activation vector.

17. The method according to claim 12, characterized in that, Before obtaining the reasoning result, the method further includes: If the controller detects an anomaly in the RRAM in-memory computing array, it sends a rollback command to the GPU client. After receiving the rollback instruction, the GPU client uses numerical calculation to process the task to be processed.

18. The method according to any one of claims 12-17, characterized in that, The method further includes: The controller performs bit-slicing on the weight matrix corresponding to each expert module and programs the segmented weight matrix into the RRAM memory computing array according to the conductance value. The RRAM in-memory computing array is calibrated using a write-verify mechanism to obtain an RRAM in-memory computing array that meets the accuracy requirements.

19. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 12-18.

20. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 12-18.

Images