A model processing method, apparatus, system, and electronic device

Abstract

This disclosure relates to a model processing method, apparatus, system, and electronic device. The method includes: acquiring a graph structure describing the computation process of a model to be processed, wherein the graph structure uses a specified data structure to represent at least one parameter in the model to be processed, the specified data structure stores the metadata of at least one parameter but does not store the parameter value of at least one parameter, and the parameter value of at least one parameter is stored in an external storage device; during the processing of the graph structure, in response to the need to use the parameter value of a target parameter among the at least one parameters, loading the parameter value of the target parameter from the external storage device to a main storage device according to the metadata of the target parameter; and continuing to process the graph structure using the parameter value of the target parameter in the main storage device. Through this disclosure, the graph structure and parameter values ​​are separated, enabling the loading of model parameter values ​​to the main storage device in batches during the graph structure processing, reducing the probability of parameter loading failure and inability to complete model processing.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a model processing method, apparatus, system, and electronic device. Background Technology

[0002] With the continuous development of generative large language models, they have achieved remarkable performance in dialogue and multimodal domains. However, in pursuit of even better performance, new large language models are becoming increasingly massive in scale, resulting in a significant increase in the number of parameters. For example, the Grok-1 open-source model has 314 billion parameters. Moreover, as large language models continue to evolve, the number of parameters will continue to grow. Therefore, how to handle such ultra-large-scale language models has become one of the most pressing issues to address.

[0003] In related technologies, all parameters of a large language model need to be loaded from the disk into memory and / or video memory at once, and then the processor processes the large language model. However, since the number of parameters of a large language model is usually large, while memory and / or video memory resources are limited, the loading of parameters of the large language model may fail, and the processing of the large language model cannot be completed. Summary of the Invention

[0004] Some existing model processing methods suffer from insufficient memory and / or video memory, leading to parameter loading failures for large language models and preventing the processing of such models. In view of this, this disclosure proposes a model processing method, apparatus, system, electronic device, storage medium, and computer program product.

[0005] According to one aspect of this disclosure, a model processing method is provided, the method comprising:

[0006] Obtain a graph structure describing the computation process of the model to be processed, wherein the graph structure uses a specified data structure to represent at least one parameter in the model to be processed, the specified data structure stores the meta-information of the at least one parameter but does not store the parameter value of the at least one parameter, and the parameter value of the at least one parameter is stored in an external storage device.

[0007] During the processing of the graph structure, in response to the need to use the parameter value of the target parameter among the at least one parameters, the parameter value of the target parameter is loaded from the external storage device to the main storage device according to the meta-information of the target parameter;

[0008] The graph structure is further processed using the parameter values ​​of the target parameters in the main storage device.

[0009] In some embodiments, the method further includes:

[0010] After the target parameter value has been used, the target parameter value is deleted from the main storage device; and / or,

[0011] Based on the results obtained using the parameter values ​​of the target parameters, perform one or more of the following:

[0012] Update the parameter value of the target parameter in the external storage device.

[0013] At least a portion of the results obtained using the parameter values ​​of the target parameter are stored as parameter values ​​for new parameters other than the at least one parameter in the external storage device.

[0014] In some embodiments, obtaining the graph structure of the model to be processed includes:

[0015] Obtain the model structure data of the model to be processed, wherein the model structure data includes meta-information of the at least one parameter and calculation operations for the at least one parameter;

[0016] Based on the metadata of the at least one parameter, the specified data structure is used to represent the at least one parameter;

[0017] The graph structure is constructed based on the at least one parameter represented by the specified data structure and the calculation operation of the at least one parameter.

[0018] In some embodiments, the method further includes:

[0019] Before processing the graph structure, model structure data of the model to be processed is obtained, wherein the model structure data includes the parameter value of the at least one parameter;

[0020] The parameter value of at least one parameter is stored in the external storage device.

[0021] In some embodiments, the metadata for each parameter includes: the name of each parameter;

[0022] And / or,

[0023] The metadata for each parameter includes at least one of the following: the occupancy of the main storage device corresponding to each parameter, the type of the main storage device, and the data type of each parameter, as well as the name of each parameter.

[0024] According to another aspect of this disclosure, a model processing apparatus is provided, the apparatus comprising:

[0025] The acquisition module is configured to acquire a graph structure describing the computation process of the model to be processed, wherein the graph structure uses a specified data structure to represent at least one parameter in the model to be processed, the specified data structure stores the meta-information of the at least one parameter and does not store the parameter value of the at least one parameter, and the parameter value of the at least one parameter is stored in an external storage device.

[0026] A loading module is configured to, during the processing of the graph structure, in response to a request to use the parameter value of a target parameter among the at least one parameters, load the parameter value of the target parameter from the external storage device to the main storage device based on the metadata of the target parameter;

[0027] The processing module is configured to continue processing the graph structure using the parameter values ​​of the target parameters in the main storage device.

[0028] According to another aspect of this disclosure, a model processing system is provided, the system comprising: a processor for performing the steps of the above-described method.

[0029] In some embodiments, the system further includes:

[0030] An external storage device is configured to store the parameter values ​​of at least one parameter in the model to be processed; and,

[0031] The primary storage device is configured to store the parameter value of the target parameter among the at least one parameters loaded from the external storage device.

[0032] According to another aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described above.

[0033] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method.

[0034] According to another aspect of this disclosure, a computer program product is provided, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0035] Through various aspects of this disclosure, a graph structure describing the computational process of a model to be processed is obtained, wherein at least one parameter in the model to be processed is represented by a specified data structure in the graph structure, the specified data structure stores the meta-information of the at least one parameter but does not store the parameter value of the at least one parameter, and the parameter value of the at least one parameter is stored in an external storage device; during the processing of the graph structure, in response to the need to use the parameter value of a target parameter among the at least one parameters, the parameter value of the target parameter is loaded from the external storage device to the main storage device according to the meta-information of the target parameter; the graph structure is further processed using the parameter value of the target parameter in the main storage device. In this way, by separating the graph structure of the model to be processed from the parameter values ​​of at least one parameter in the model, a specified data structure is used in the graph structure to represent the at least one parameter in the model to be processed, storing the meta-information of the at least one parameter but not the parameter value. The parameter value of the at least one parameter is stored in an external storage device, thereby reducing the space occupied by the graph structure in the main storage device. Furthermore, during the processing of the graph structure, when the parameter value of the target parameter is needed, the parameter value of the target parameter is loaded into the main storage device in a timely / real-time manner according to the meta-information of the target parameter. In this way, the timely / real-time parameter value loading method supports the actual access to the parameter values, realizing the batch loading of the parameter values ​​of the model to be processed to the main storage device during the graph structure processing, reducing the probability of loading failure of the model to be processed and the probability of not being able to complete the processing of the model. In particular, it can reduce the probability of loading failure and the inability to complete the processing of the model when the number of parameters of the model to be processed is too large, which may lead to insufficient main storage device.

[0036] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0037] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0038] Figure 1 A schematic diagram of the structure of a model processing system according to some embodiments of the present disclosure is shown.

[0039] Figure 2 A flowchart illustrating a model processing method according to some embodiments of the present disclosure is shown.

[0040] Figure 3 A structural diagram of a model processing apparatus according to some embodiments of the present disclosure is shown.

[0041] Figure 4A block diagram of an electronic device according to some embodiments of the present disclosure is shown. Detailed Implementation

[0042] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0043] As used herein, the terms “comprising,” “including,” “having,” or variations thereof are open-ended and include one or more of the stated features, integrals, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integrals, elements, steps, components, functions, or groups thereof.

[0044] When an element is referred to as “connected,” “coupled,” “responding,” or a variation thereof relative to another element, it may be directly connected, coupled, or responding to another element, or there may be an intermediate element present.

[0045] Although the terms first, second, third, etc., may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another. Therefore, without departing from the teachings of the inventive concept, a first element / operation in some embodiments may be referred to as a second element / operation in other embodiments.

[0046] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0047] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0048] In related technologies, the actual deployment of large language models generally requires the use of deep learning compilers. For some common deep learning framework systems, the compiler's processing of large language models is as follows: First, the parameters of the large language model are loaded from the disk into the device's memory and / or video memory at once; second, if the large language model needs to be quantized, the quantization algorithm is called on the entire large language model to obtain the quantized large language model; third, certain optimization processing is performed on the large language model, such as optimization shown in the figure.

[0049] The aforementioned processing involves loading the parameters of a large language model from disk to memory and / or video memory all at once, quantization processing of the large language model, and optimization processing of the large language model. On the one hand, when the number of parameters in the large language model is large and the device's memory and / or video memory are insufficient, the requirement for one-time loading cannot be met. Especially for ultra-large-scale language models, most devices cannot meet the requirement of loading the parameters of the large language model from disk to the device's memory and / or video memory all at once. On the other hand, even if the device can complete the one-time loading of the parameters of the large language model, the quantization or optimization processing of the large language model may involve changes to the ultra-large-scale parameters of the large language model, which will generate more memory and / or video memory overhead. Taking quantization processing as an example, after quantizing each weight of the large language model, a new quantization parameter is generated, which will lead to a significant increase in the required memory and / or video memory. As a result, the quantization processing of the large language model may not be completed due to insufficient memory and / or video memory of the device. For example, for a large language model like Grok-1, due to the limited memory and / or video memory of a single device, especially laptops with smaller memory and / or video memory, it is usually impossible to meet the requirement of loading and processing the parameters of the large language model all at once.

[0050] It is evident that the relevant technologies suffer from insufficient device memory and / or video memory, leading to the failure to load parameters for large language models and the inability to process large language models.

[0051] To address the aforementioned technical issues, this disclosure provides a model processing method (details below). By separating the graph structure of the model from the parameter values ​​of the model parameters and combining this with a timely / real-time parameter value loading method to support real access to the parameter values, this method enables the batch loading of model parameter values ​​to the main storage device during graph structure processing. This reduces the probability of model parameter loading failure and the probability of model processing failure, especially in cases where the large number of model parameters can lead to insufficient main storage. Furthermore, it significantly reduces the requirements for device memory and / or video memory during model processing, enabling the processing of large-scale or ultra-large-scale models even with limited memory and / or video memory resources. In some examples, even on single devices with limited memory and / or video memory, large-scale or ultra-large-scale model processing can be completed on a single device, even if the single device cannot load the entire model at once, without the need for additional video memory and / or memory resources. For example, it eliminates the need to increase the number of devices to expand memory and / or video memory, thus significantly saving on model processing costs.

[0052] For example, the model processing method provided in this disclosure can be executed by electronic devices such as terminal devices and servers, or by a part of an electronic device (such as a processor). The terminal device can be a desktop terminal or a mobile terminal, such as a laptop, tablet, desktop computer, smartphone, smart speaker, smartwatch, smart TV, in-vehicle terminal, or other types of electronic devices. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0053] Figure 1 A schematic diagram of the structure of a model processing system according to some embodiments of the present disclosure is shown. Figure 1 As shown, the system may include: a processor; the processor may be used to execute the model processing method provided in this disclosure.

[0054] For example, processors include, but are not limited to, CPUs and GPUs. The number of processors can be one or more, and the types of different processors can be the same or different; there is no limitation on this.

[0055] In some embodiments, the system further includes: an external storage device and a main storage device; wherein, the main storage device is a storage device directly accessed by the processor, including but not limited to memory, video memory, etc.; the external storage device is a storage device indirectly accessed by the processor through a medium such as an I / O controller, including but not limited to a disk. The number of main storage devices or external storage devices can be one or more, and the types of different main storage devices or external storage devices can be the same or different, without limitation.

[0056] In some embodiments, an external storage device is configured to store parameter values ​​of at least one parameter in the model to be processed; a main storage device is configured to store parameter values ​​of a target parameter among the at least one parameter loaded from the external storage device.

[0057] For example, the model to be processed can be any type of neural network model, such as deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), etc.; furthermore, there are no restrictions on the structure and size (i.e., the number of parameters) of the model to be processed. As an example, the model to be processed can be a large language model; the parameters of the model to be processed may include weights, biases, etc.

[0058] In some scenarios, the storage space required for the parameters in the model to be processed exceeds the available storage space of the main storage device. In other words, it is impossible to load all the parameters of the model to be processed into the main storage device at once. For example, the model to be processed may be a very large language model, and the available storage space of the main storage device is insufficient to store all the parameters of the large language model.

[0059] The following is combined with Figure 1 The model processing system shown herein provides a detailed description of the model processing method provided in this disclosure.

[0060] Figure 2 This diagram illustrates a flowchart of a model processing method according to some embodiments of the present disclosure, which can be derived from the above-described... Figure 1 The processor in the system shown executes, such as Figure 2 As shown, the method may include the following steps:

[0061] Step 201: Obtain a graph structure describing the computation process of the model to be processed, wherein the graph structure uses a specified data structure to represent at least one parameter in the model to be processed, the specified data structure stores the meta-information of the at least one parameter and does not store the parameter value of the at least one parameter, and the parameter value of the at least one parameter is stored in an external storage device.

[0062] In this step, a designated data structure is used to represent at least one parameter in the graph structure describing the computation process of the model to be processed. This designated data structure stores the metadata of at least one parameter but does not store its parameter value. In other words, the designated data structure is limited in its storage structure to retain only the metadata of the at least one parameter, without storing or carrying its parameter value. In contrast, the storage space required for the parameter's metadata is usually much smaller than the storage space required for its parameter value. Thus, by using a designated data structure, the graph structure of the model to be processed is separated from the parameter value of at least one parameter, effectively reducing the storage space occupied by the graph structure.

[0063] For example, the obtained graph structure describing the computation process of the model to be processed is stored in the main storage device. Since the parameter values ​​of at least one parameter in the model to be processed, represented by the specified data structure, are stored in the external storage device, it is not necessary to apply for actual storage space in the main storage device for the parameter values ​​of the at least one parameter. This effectively reduces the storage space occupied by the at least one parameter in the main storage device, and correspondingly reduces the storage space occupied by the graph structure in the main storage device. The number of at least one parameter can be configured according to needs and is not limited here. For example, it can be all the parameters in the model to be processed, or it can be some of the parameters in the model to be processed. As an example, the parameter values ​​of all parameters in the model to be processed can be stored in the external storage device, wherein the parameter value of each parameter can be stored in the external storage device as an independent file.

[0064] The graph structure, also known as the model representation, model structure, or computation graph, describes the computational process of the model under test. It represents the computational process of the model, specifically its inference computation method. This computational process can include various computational operations performed by the model during inference, such as multiplication, convolution, pooling, dot addition, deconvolution, rearrangement, nonlinearity, and normalization. For example, based on whether the computational operation involves the parameters of the model under test, operations can be categorized into those involving the model's parameters and those not. For instance, performing a convolution operation between the feature matrix and a convolution kernel involves the weights in the kernel and is therefore a computational operation involving the model's parameters. Conversely, performing a pooling operation on the feature matrix does not involve the model's weights and is therefore a computational operation not involving the model's parameters.

[0065] For example, a graph structure can be a structure represented by sampling nodes and edges, where related nodes can be connected by edges. The form of the graph structure can be configured according to requirements and is not limited thereto. As an example, in a graph structure, nodes represent computational operations, and the edges between two nodes can represent the dependencies between the corresponding computational operations. The direction of the edges can represent the flow of data between the two computational operations. For example, if a graph structure includes node A1 representing computational operation a1 and node A2 representing computational operation a2, and nodes A1 and A2 are connected by an edge with the direction of the edge pointing from node A1 to node A2, then the output data of computational operation a1 is the input data of computational operation a2. As another example, in a graph structure, edges represent computational operations, and nodes represent the input or output data of the computational operations. For example, if a graph structure includes node B1 representing data b1 and node B2 representing data b2, and nodes B1 and B2 are connected by an edge representing computational operation C with the direction of the edge pointing from node B1 to node B2, then data b1 is processed by computational operation C to obtain data b2.

[0066] Meta-information, also known as attribute information, serves as essential information for identifying parameters or reflecting their properties. It allows for the differentiation of different parameters and reveals their properties. By storing the meta-information of at least one parameter in a specified data structure, the validity of that parameter within the graph structure is guaranteed. This ensures the processor can correctly identify the at least one parameter and process or store its value during graph processing. The specific content of the meta-information can be designed according to requirements, while ensuring the smooth operation of the processor in processing the graph structure; no limitations are imposed on this design.

[0067] In some embodiments, the metadata for each parameter includes the name of each parameter. For example, the name of a parameter can serve as a unique identifier, with different identifiers for different parameters; thus, the name of each parameter serves as a unique identifier to distinguish it from other parameters; the processor can differentiate different parameters in the model to be processed based on the parameter name during the processing of the graph structure.

[0068] In other embodiments, the metadata for each parameter includes at least one of the following: the occupancy of the main storage device corresponding to each parameter, the type of the main storage device, and the data type of each parameter, as well as the name of each parameter. Thus, in addition to the parameter name, the metadata for each parameter can also be configured as needed with information reflecting different aspects of the parameter's properties, such as the occupancy of the main storage device corresponding to the parameter, and / or the type of the main storage device corresponding to the parameter, and / or the data type of the parameter, thereby meeting the processor's needs for processing or storing parameter values ​​during the processing of the graph structure.

[0069] For example, the occupancy of the main storage device corresponding to the parameter represents the amount of storage space on the main storage device that the parameter value needs to occupy. It can be understood that, for parameters represented by a specified data structure, since the parameter value is stored on an external storage device, the occupancy of the main storage device corresponding to the parameter stored in the specified data structure represents the amount of storage space on the main storage device that would be required if the parameter value were stored on the main storage device, rather than the amount of storage space on the main storage device that has actually been occupied.

[0070] For example, the type of primary storage device corresponding to a parameter can include main memory, video memory, etc. For instance, if the type of primary storage device corresponding to a parameter is main memory, it means that the parameter value needs to be stored in main memory, and the processor can read the parameter value by directly accessing main memory. It is understood that for parameters represented using a specified data structure, the type of primary storage device corresponding to the parameter indicates that if the parameter value is to be stored in primary storage, it needs to be stored in a primary storage device of that type, rather than actually being stored in a primary storage device of that type.

[0071] For example, the data type of the parameter can include floating-point numbers, integers, etc., such as FP32 (Single-precision floating-point), FP16 (Half-precision Floating Point Number), INT8 (8-bit Integer), INT4 (4-bit Integer), etc.

[0072] The specified data structure can be designed according to requirements, ensuring that the metadata of at least one parameter can be retained in the storage structure. Since the specified data structure of this disclosure does not store the specific parameter value but stores the parameter's metadata, the processor can use a tensor represented by the specified data structure when processing the graph structure. In this way, using a tensor represented by the specified data structure guarantees the validity of the parameter in the graph structure while avoiding the storage space occupation of the main storage device caused by storing the actual parameter value. As an example, unlike using a tensor carrying the parameter value to represent a parameter in the model to be processed, if a tensor represented by the specified data structure is used to represent the parameter, then the tensor represented by the specified data structure can store all information in the tensor carrying the parameter value except for the parameter's value.

[0073] In some embodiments, obtaining the graph structure of the model to be processed includes: obtaining model structure data of the model to be processed, wherein the model structure data includes metadata of the at least one parameter and calculation operations for the at least one parameter; representing the at least one parameter using the specified data structure based on the metadata of the at least one parameter; and constructing the graph structure based on the at least one parameter represented by the specified data structure and the calculation operations for the at least one parameter.

[0074] For example, the model structure data of the model to be processed can be obtained from an open-source database or locally. In some examples, the obtained model structure data of the model to be processed may include metadata of each parameter of the model to be processed and various computational operations performed by the model to be processed during inference; wherein, the metadata of each parameter of the model to be processed may include metadata of at least one parameter and metadata of other parameters (i.e., parameters in the model to be processed that are not represented by a specified data structure), and the various computational operations performed by the model to be processed during inference may include computational operations for the at least one parameter, computational operations for the other parameters, and computational operations not for parameters of the model to be processed.

[0075] For example, a specified data structure can be configured to store the metadata of the at least one parameter but not the parameter value of the at least one parameter, thereby representing the at least one parameter using the specified data structure.

[0076] For example, different forms of graph structures can be flexibly constructed using different methods according to requirements, without limitation. For instance, a graph structure can be constructed with computational operations as nodes and data dependencies between different computational operations as edges; another example is a graph structure with computational operations as edges and input or output data of the computational operations as nodes. In some examples, a graph structure describing the computational process of the model to be processed can be constructed based on the parameters of the model to be processed (including at least one parameter represented using a specified data structure) and the computational operations performed during inference on the model. The constructed graph structure does not contain parameter values ​​of the parameters in the model to be processed represented using the specified data structure, thus occupying less storage space.

[0077] In some embodiments, the method further includes: acquiring model structure data of the model to be processed before processing the graph structure, wherein the model structure data includes parameter values ​​of the at least one parameter; and storing the parameter values ​​of the at least one parameter in the external storage device. Thus, before processing the graph structure, the parameter values ​​of the parameters in the model to be processed, represented by a specified data structure, are stored in the external storage device so that during subsequent graph structure processing, the parameter values ​​of relevant parameters can be loaded from the external storage device to the main storage device based on requirements, thereby ensuring the smooth progress of graph structure processing.

[0078] In some examples, the model structure data of the model to be processed also includes parameter values ​​of other parameters (i.e., parameters in the model to be processed that are not represented by a specified data structure). These other parameter values ​​can be stored in the main storage device so that they can be used as needed during subsequent processing of the graph structure.

[0079] Step 202: During the processing of the graph structure, in response to the need to use the parameter value of the target parameter among the at least one parameters, the parameter value of the target parameter is loaded from the external storage device to the main storage device according to the metadata of the target parameter.

[0080] In some scenarios, after obtaining the graph structure describing the computational process of the model to be processed, the processor can process the graph structure. During this processing, parameter values ​​from the model may be used. If a parameter value represented by a specified data structure is needed, the processor loads the target parameter value from the external storage device to the main storage device based on the metadata of the target parameter stored in the specified data structure, allowing the processor to directly access the main storage device to read the target parameter value. If other parameters (i.e., parameters in the model not represented by a specified data structure) are needed, the processor directly accesses the main storage device to read the corresponding parameter value. Thus, based on the idea of ​​batch parameter loading, when the processor needs to use the parameter value of a parameter represented by a specified data structure, the processor loads the parameter value from the external storage device to the main storage device in a timely manner / in real-time, based on the parameter's metadata, ensuring the smooth processing of the graph structure.

[0081] For example, in response to a request to use the parameter value of a target parameter among the at least one parameter, the processor loads the parameter value of the target parameter from an external storage device to the main storage device via a medium such as an I / O controller, based on the metadata of the target parameter stored in a specified data structure. As an example, the parameter value of the target parameter may be stored in the external storage device as a separate file; in this case, the parameter value in the separate file corresponding to the target parameter in the external storage device can be loaded into the main storage device. The target parameter can be understood as any parameter among the at least one parameter.

[0082] For example, processing the graph structure may include at least one of compression and graph optimization.

[0083] Compression refers to reducing the size of model parameters, such as parameter quantization and parameter sparsity. For example, one or more weights in the model can be quantized to a low bit width. If the weight data type is FP16 and the weight parameter value is 16 bits wide, INT8 quantization can be performed, resulting in an 8-bit parameter value. Another example is setting the weight values ​​of some weights in a network layer to zero, thereby reducing the effective weights in that layer and achieving weight sparsity. Graph optimization refers to optimizing the graph structure without changing its form, such as computational operation pruning, computational operation merging, and parameter merging. For example, removing unnecessary or ineffective computational operations is called computational operation pruning; merging multiple computational operations into one is called computational operation merging; and merging weights from multiple network layers into one weight is called weight merging.

[0084] Understandably, when processing graph structures, some processes do not involve the parameter values ​​of the target parameters. For example, when calculating pruning operations, it is not necessary to use the parameter values ​​of the parameters in the model to be processed; correspondingly, the parameter values ​​can be ignored during the processing. However, some processes require the parameter values ​​of the parameters in the model to be processed. For example, when performing parameter quantization, it is necessary to use the parameter values ​​of the parameters in the model to be processed, and correspondingly, the processor needs to read the corresponding parameter values ​​from the main storage device. For example, if the parameter values ​​of the required parameters are stored in the main storage device, the processor can directly read the parameter values ​​from the main storage device. If the parameter values ​​of the required parameters are stored in the external storage device, the processor can load the parameter values ​​from the external storage device to the main storage device based on the metadata of the required parameters (i.e., the target parameters), and then the processor can directly read the parameter values ​​from the main storage device.

[0085] In some embodiments, the metadata for each parameter includes: the name of each parameter. Since the name of each parameter serves as a unique identifier to distinguish it from other parameters, during the processing of the graph structure, in response to the need to use the parameter value of the target parameter, a retrieval can be performed on the external storage device based on the name of the target parameter, and the retrieved parameter value can be loaded into the main storage device so that the processor can use the parameter value.

[0086] In some embodiments, the metadata of each parameter includes at least one of the following: the occupancy of the main storage device corresponding to each parameter, the type of the main storage device, and the data type of each parameter, as well as the name of each parameter. Thus, during graph structure processing, in response to the need to use the parameter value of a target parameter, a search can be performed on the external storage device based on the name of the target parameter. Suitable storage space on the main storage device can be requested by combining the occupancy of the main storage device corresponding to the target parameter, and / or the type of the main storage device, and / or the data type of the target parameter, etc., thereby loading the retrieved parameter value into the main storage device so that the processor can use the parameter value; for example, if the type of the main storage device corresponding to the target parameter is memory, then the retrieved parameter value of the target parameter can be loaded into memory.

[0087] Step 203: Using the parameter values ​​of the target parameters in the main storage device, continue to process the graph structure.

[0088] For example, during the processor's processing of the graph structure, in response to the need to use the parameter value of the target parameter among the at least one parameters, after loading the parameter value of the target parameter from the external storage device to the main storage device, the processor can directly access the parameter value in the main storage device, thereby using the parameter value to continue processing the graph structure.

[0089] It should be noted that, since the processor may need to use the target parameter value multiple times during the graph structure processing, steps 202-203 above may need to be executed multiple times. That is, during the graph structure processing, in response to the need to use the target parameter value each time, the target parameter value is loaded from the external storage device to the main storage device based on the target parameter's metadata. The processor then uses the target parameter value in the main storage device to continue processing the graph structure until the graph structure processing is completed. The target parameter used each time can be the same parameter or a different parameter.

[0090] In some embodiments, the method further includes: deleting the target parameter value from the main storage device after using the target parameter value. For example, during the processor's graph structure processing, if the target parameter value is loaded from the external storage device to the main storage device multiple times, the target parameter value used in this instance is deleted from the main storage device after each use of the target parameter value. Considering that during the processor's processing of the graph structure, after processing the graph structure using the target parameter value from the main storage device, the processor no longer needs to use the target parameter value or will not need to use it again for a certain period, the target parameter value can be deleted from the main storage device, thereby saving storage space on the main storage device. Simultaneously, the saved storage space on the main storage device can also be used to provide the necessary storage space for the processor to continue subsequent graph structure processing, thereby ensuring the smooth progress of graph structure processing, improving the utilization rate of the main storage device, further reducing the probability of loading the model to be processed failing, and reducing the probability of not being able to complete the processing of the model to be processed.

[0091] In some embodiments, the method further includes: based on the result obtained using the parameter value of the target parameter, performing one or more of the following: updating the parameter value of the target parameter in the external storage device; storing at least a portion of the result obtained using the parameter value of the target parameter as the parameter value of a new parameter other than the at least one parameter in the external storage device. For example, during the processor's graph structure processing, if the parameter value of the target parameter is loaded from the external storage device to the main storage device multiple times, then after each use of the parameter value of the target parameter in the main storage device, the parameter value of the target parameter in the external storage device can be updated based on the result obtained by the processor using the parameter value of the target parameter in this instance; or, at least a portion of the result obtained by the processor using the parameter value of the target parameter in this instance can be stored in the external storage device as the parameter value of a new parameter other than the at least one parameter.

[0092] As an example, the parameter value of the target parameter in the external storage device can be updated based on the result obtained using the parameter value of the target parameter. For instance, if the weights in a network layer of a graph structure are represented using a specified data structure, and when performing weight sparsification on that network layer, it is necessary to set the weight value of a certain weight to zero, the processor can load the weight value of that weight from the external storage device to the main storage device based on the metadata of that weight. Then, the processor reads the weight value from the main storage device and modifies it to zero, accordingly updating the weight value of that weight in the external storage device to zero.

[0093] As another example, the parameter value of the target parameter in the external storage device can be updated based on the result obtained using the parameter value of the target parameter, and at least a portion of the result obtained using the parameter value of the target parameter can be stored in the external storage device as the parameter value of a new parameter other than the at least one parameter. For example, if a certain weight A in the graph structure is represented by a specified data structure, when quantizing the weight A, the processor can load the weight value a of the weight A from the external storage device to the main storage device according to the metadata of the weight A. Then, the processor reads the weight value a from the main storage device and performs quantization using a quantization algorithm to generate the quantized weight value a* and quantization parameter b (such as scaling factor, offset, etc.). Then, the weight value of the weight A in the external storage device is updated to a*, and the quantization parameter b is used as a new parameter. Correspondingly, the parameter value of the quantization parameter b (such as the value of the scaling factor, the value of the offset, etc.) is stored in the external storage device as the parameter value of the new parameter.

[0094] Furthermore, in some examples, after the processor completes the processing of the graph structure, instruction code can be generated for the processed graph structure, thereby generating instruction code, such as binary instruction code, which can be used to deploy the model to be processed on the hardware platform for inference and other tasks.

[0095] In this embodiment of the disclosure, a graph structure describing the computation process of a model to be processed is obtained. The graph structure uses a specified data structure to represent at least one parameter in the model to be processed. The specified data structure stores metadata of the at least one parameter but does not store the parameter value of the at least one parameter. The parameter value of the at least one parameter is stored in an external storage device. During the processing of the graph structure, in response to the need to use the parameter value of a target parameter among the at least one parameters, the parameter value of the target parameter is loaded from the external storage device to the main storage device based on the metadata of the target parameter. The graph structure is then further processed using the parameter value of the target parameter in the main storage device. In this way, by separating the graph structure of the model to be processed from the parameter values ​​of at least one parameter of the model to be processed, a specified data structure is used in the graph structure to represent the at least one parameter in the model to be processed, which stores the meta-information of at least one parameter but does not store the parameter value of at least one parameter. The parameter value of at least one parameter is stored in an external storage device, thereby reducing the space occupied by the graph structure in the main storage device. Furthermore, during the processing of the graph structure, when the parameter value of the target parameter is needed, the parameter value of the target parameter is loaded into the main storage device in a timely / real-time manner according to the meta-information of the target parameter. In this way, the timely / real-time parameter value loading method supports the actual access to the parameter value, and realizes the batch loading of the parameter values ​​of the model to be processed to the main storage device during the graph structure processing, reducing the probability of loading failure of the model to be processed and the probability of not being able to complete the processing of the model to be processed. In particular, it can reduce the probability of loading failure and the inability to complete the processing of the model to be processed when the number of parameters of the model to be processed is too large, which may lead to insufficient main storage device.

[0096] Furthermore, it significantly reduces the requirements for device memory and / or video memory during model processing, thereby enabling the processing of large-scale or ultra-large-scale models even with limited memory and / or video memory resources. In some examples, even if a single device cannot load a large-scale or ultra-large-scale model at once, it can still process large-scale or ultra-large-scale models on a single device without the need for additional video memory and / or memory resources. For example, it eliminates the need to expand memory and / or video memory by increasing the number of devices, thus greatly saving and reducing the cost of model processing.

[0097] The following example, using a large language model as the model to be processed, illustrates the above model processing method.

[0098] First, a graph structure describing the computational process of the large language model is obtained. Specifically, the processor can obtain the model structure data of the large language model, which may include the meta-information of each weight in the large language model, the weight values ​​of each weight, and the computational operations performed by the large language model during inference. For some weights in the large language model (such as weights a, b, c, d, etc.), based on the meta-information of these weights, tensors with specified data structures can be used to represent these weights, and the weight values ​​of these weights can be stored on disk. For other weights in the large language model (such as weights e, f, etc.), based on the meta-information of these weights and their parameter values, tensors carrying parameter values ​​can be used to represent these weights. Then, based on the partial weights in the large language model represented by tensors with specified data structures, the other weights in the large language model represented by tensors carrying parameter values, and the computational operations performed by the large language model during inference, a graph structure describing the computational process of the large language model is constructed; the graph structure can be stored in memory and / or video memory.

[0099] Then, the processor can process the graph structure describing the computation process of the large language model. During the processing of the graph structure, each time in response to the demand for the weight value of the target weight represented by a tensor using a specified data structure, the processor loads the weight value of the target weight from the disk into memory and / or video memory based on the meta-information of the target weight. The processor then uses the weight value of the target weight loaded into memory and / or video memory to continue processing the graph structure until the processing of the graph structure is completed. After each use of the weight value of the target weight, the processor deletes the weight value of the target weight from memory and / or video memory. Taking the sequential quantization of weights a, b, e, c, d, and f in a large language model as an example, since the weight value of weight a is stored on disk, the processor loads the weight value of weight a into memory based on the metadata of weight a. The processor reads the weight value of weight a from memory, quantizes it, stores the quantized weight value of weight a and the quantization parameters on disk, and deletes the original weight value of weight a stored in memory. This completes the quantization of weight a. The processor continues based on the metadata of weight b, loading the weight value of weight b into memory. The processor reads the weight value of weight b from memory, quantizes it, stores the quantized weight value of weight b and the quantization parameters on disk, and deletes the original weight value of weight b stored in memory. This completes the quantization of weight b. Since the weight value of weight e is stored in memory, the processor can continue to directly read the weight value of weight e from memory, quantize it, and store the quantized weight value of weight e and the quantization parameters in memory. This completes the quantization of weight e. And so on, traversing all the weights that need to be quantized, completing the quantization of these weights.

[0100] In the aforementioned processing of large language models, it is unnecessary to load all weights from the tensor representation of the large language model (represented by a specified data structure) along with the weights of other weights in the tensor representation carrying parameter values ​​into memory and / or video memory at once. Instead, the weights of that portion of the large language model are stored on disk. During graph structure processing, each time a weight value representing a weight in a tensor representation of a specified data structure is needed, the corresponding parameter value is loaded from disk into memory and / or video memory. After the processor has finished using the weight value, it is deleted from memory and / or video memory. By loading the weight values ​​of the large language model into memory and / or video memory in batches, the risk of weight loading failure and inability to complete the processing of the large language model, which would be caused by loading all weight values ​​of the large language model into memory and / or video memory at once, is reduced.

[0101] Based on the same inventive concept of the above method embodiments, the present disclosure also provides a model processing apparatus that can be used to execute the technical solutions described in the above method embodiments.

[0102] Figure 3 This diagram illustrates a structural diagram of a model processing apparatus according to some embodiments of the present disclosure, such as... Figure 3 As shown, the device may include: an acquisition module 301, configured to acquire a graph structure describing the computation process of a model to be processed, wherein the graph structure uses a specified data structure to represent at least one parameter in the model to be processed, the specified data structure stores the metadata of the at least one parameter but does not store the parameter value of the at least one parameter, and the parameter value of the at least one parameter is stored in an external storage device; a loading module 302, configured to, during the processing of the graph structure, in response to the need to use the parameter value of a target parameter among the at least one parameter, load the parameter value of the target parameter from the external storage device to a main storage device based on the metadata of the target parameter; and a processing module 303, configured to continue processing the graph structure using the parameter value of the target parameter in the main storage device.

[0103] In this embodiment of the disclosure, a graph structure describing the computation process of a model to be processed is obtained. The graph structure uses a specified data structure to represent at least one parameter in the model to be processed. The specified data structure stores metadata of the at least one parameter but does not store the parameter value of the at least one parameter. The parameter value of the at least one parameter is stored in an external storage device. During the processing of the graph structure, in response to the need to use the parameter value of a target parameter among the at least one parameters, the parameter value of the target parameter is loaded from the external storage device to the main storage device based on the metadata of the target parameter. The graph structure is then further processed using the parameter value of the target parameter in the main storage device. In this way, by separating the graph structure of the model to be processed from the parameter values ​​of at least one parameter of the model to be processed, a specified data structure is used in the graph structure to represent the at least one parameter in the model to be processed, which stores the meta-information of at least one parameter but does not store the parameter value of at least one parameter. The parameter value of at least one parameter is stored in an external storage device, thereby reducing the space occupied by the graph structure in the main storage device. Furthermore, during the processing of the graph structure, when the parameter value of the target parameter is needed, the parameter value of the target parameter is loaded into the main storage device in a timely / real-time manner according to the meta-information of the target parameter. In this way, the timely / real-time parameter value loading method supports the actual access to the parameter value, and realizes the batch loading of the parameter values ​​of the model to be processed to the main storage device during the graph structure processing, reducing the probability of loading failure of the model to be processed and the probability of not being able to complete the processing of the model to be processed. In particular, it can reduce the probability of loading failure and the inability to complete the processing of the model to be processed when the number of parameters of the model to be processed is too large, which may lead to insufficient main storage device.

[0104] In some embodiments, the processing module 303 is further configured to: delete the parameter value of the target parameter in the main storage device after the use of the parameter value of the target parameter is completed; and / or, based on the result obtained using the parameter value of the target parameter, perform one or more of the following: update the parameter value of the target parameter in the external storage device, and store at least a portion of the result obtained using the parameter value of the target parameter as the parameter value of a new parameter other than the at least one parameter in the external storage device.

[0105] In some embodiments, the acquisition module 301 is further configured to: acquire model structure data of the model to be processed, wherein the model structure data includes metadata of the at least one parameter and calculation operations for the at least one parameter; represent the at least one parameter using the specified data structure based on the metadata of the at least one parameter; and construct the graph structure based on the at least one parameter represented by the specified data structure and the calculation operations for the at least one parameter.

[0106] In some embodiments, the acquisition module 301 is further configured to: acquire model structure data of the model to be processed before processing the graph structure, wherein the model structure data includes parameter values ​​of the at least one parameter; and store the parameter values ​​of the at least one parameter in the external storage device.

[0107] In some embodiments, the metadata of each parameter includes: the name of each parameter; and / or, the metadata of each parameter includes: at least one of the following: the occupancy of the main storage device corresponding to each parameter, the type of the main storage device, and the data type of each parameter, as well as the name of each parameter.

[0108] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0109] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0110] This disclosure also provides a non-volatile computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0111] This disclosure also provides a computer program product, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above method.

[0112] Figure 4 A block diagram of an electronic device 1900 according to some embodiments of the present disclosure is shown. For example, the electronic device 1900 may be provided as a server or a terminal device. (Refer to...) Figure 4 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0113] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). Electronic device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM Mac OS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0114] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0115] Computer-readable storage media can be tangible devices capable of holding and storing programs / instructions used by instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0116] The computer program (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage medium in the respective computing / processing device.

[0117] The computer program (or computer program instructions) used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions to implement various aspects of this disclosure.

[0118] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0119] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0120] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0121] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0122] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A model processing method, characterized in that, The method includes: Obtain a graph structure describing the computation process of the model to be processed, wherein the graph structure uses a specified data structure to represent at least one parameter in the model to be processed, the specified data structure stores the meta-information of the at least one parameter but does not store the parameter value of the at least one parameter, and the parameter value of the at least one parameter is stored in an external storage device. During the processing of the graph structure, in response to the need to use the parameter value of the target parameter among the at least one parameters, the parameter value of the target parameter is loaded from the external storage device to the main storage device according to the meta-information of the target parameter; The graph structure is further processed using the parameter value of the target parameter in the main storage device; after the use of the parameter value of the target parameter is completed, the parameter value of the target parameter in the main storage device is deleted.

2. The method according to claim 1, characterized in that, The method further includes: Based on the results obtained using the parameter values ​​of the target parameters, perform one or more of the following: Update the parameter value of the target parameter in the external storage device. At least a portion of the results obtained using the parameter values ​​of the target parameter are stored as parameter values ​​for new parameters other than the at least one parameter in the external storage device.

3. The method according to claim 1 or 2, characterized in that, The graph structure for obtaining the model to be processed includes: Obtain the model structure data of the model to be processed, wherein the model structure data includes meta-information of the at least one parameter and calculation operations for the at least one parameter; Based on the metadata of the at least one parameter, the specified data structure is used to represent the at least one parameter; The graph structure is constructed based on the at least one parameter represented by the specified data structure and the calculation operation of the at least one parameter.

4. The method according to claim 1 or 2, characterized in that, The method further includes: Before processing the graph structure, model structure data of the model to be processed is obtained, wherein the model structure data includes the parameter value of the at least one parameter; The parameter value of at least one parameter is stored in the external storage device.

5. The method according to claim 1 or 2, characterized in that, The metadata for each parameter includes: the name of each parameter; And / or, The metadata for each parameter includes at least one of the following: the occupancy of the main storage device corresponding to each parameter, the type of the main storage device, and the data type of each parameter, as well as the name of each parameter.

6. A model processing device, characterized in that, The device includes: The acquisition module is configured to acquire a graph structure describing the computation process of the model to be processed, wherein the graph structure uses a specified data structure to represent at least one parameter in the model to be processed, the specified data structure stores the meta-information of the at least one parameter and does not store the parameter value of the at least one parameter, and the parameter value of the at least one parameter is stored in an external storage device. A loading module is configured to, during the processing of the graph structure, in response to a request to use the parameter value of a target parameter among the at least one parameters, load the parameter value of the target parameter from the external storage device to the main storage device based on the metadata of the target parameter; The processing module is configured to continue processing the graph structure using the parameter value of the target parameter in the main storage device; and after the use of the parameter value of the target parameter is completed, to delete the parameter value of the target parameter in the main storage device.

7. A model processing system, characterized in that, The system includes a processor for performing the method according to any one of claims 1 to 5.

8. The system according to claim 7, characterized in that, The system also includes: An external storage device is configured to store the parameter values ​​of at least one parameter in the model to be processed; and, The primary storage device is configured to store the parameter value of the target parameter among the at least one parameters loaded from the external storage device.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

10. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

11. A computer program product comprising a computer program, or a non-volatile computer-readable storage medium carrying a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images