Distributed large model

Abstract

The invention provides a distributed large model, which belongs to the technical field of cloud computing and comprises a plurality of intelligent computing units (AIUs) used for data training and reasoning; the AIUs are distributed in a base station and a central machine room of a computing power network; the plurality of intelligent controllers AICs are used for controlling integration of model parameter information; the AICs are distributed in a control machine room of the computing power network; after data input by a user is received, parameters of the AIUs and the AICs are updated step by step on the basis of the difference between the reasoning result of the distributed large model and source data information; the source data information is original output data corresponding to the input data. According to the method, a large model basic structure is decoupled and fused with computing network resources, a distributed large model is realized, and large model parameter scale flexible expansion is realized. Through comparing the source data information with the output data of the distributed large model, the unsupervised training of the distributed large model in the operation process is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing, and in particular to a distributed large model. BACKGROUND

[0002] A large model refers to a machine learning model with a large number of parameters, especially a deep learning model. A large model has strong learning ability due to the large number of parameters, can capture and learn complex relationships and patterns in data, and has stronger generalization ability, and can have the ability of logical reasoning and innovation generation to a certain extent. As a generation model, the size of the parameter quantity of the large model and the size of the knowledge base are important factors affecting the generation accuracy of the model. A computing power network is a new type of information infrastructure, which allocates and flexibly schedules computing resources, storage resources and network resources according to business needs among the cloud, edge and end. Its core idea is to connect geographically distributed computing power center nodes through new network technologies, dynamically and in real time perceive the state of computing power resources, and then allocate and schedule computing tasks, transmit data, and form a network that perceives, allocates and schedules computing power globally.

[0003] At present, the deployment of a large model mainly has the following two ways: The first way is to completely separate the large model from the computing power network. The large model is independently deployed to a centralized computer room, or deployed to a distributed computer room according to the region. When the deployment computer room of the large model reaches the request upper limit, the computing power of the computing power network is rented through a standardized interface to realize the corresponding service. In this deployment mode, the computing power network provides a general computing power pool, and does not preinstall a model. When needed, the model is loaded and the inference task is executed.

[0004] The second way is to partially integrate the large model with the computing power network. The large model server cluster is arranged according to the distribution of the computing power network. By placing the large model server and the computing power network server in the same computer room, the integration is realized in form.

[0005] The above two deployment modes cannot realize online learning of the large model. The training iteration of the large model can only rely on the large model manufacturer to collect data independently, perform offline training, and publish the model after training. The prediction generation level of the large model is affected by the resources invested by the manufacturer. The more data collected, the better the effect. Therefore, the large model generation effect is usually insufficient, and the training cost is high. SUMMARY

[0006] The present application provides a distributed large model to solve the defects of high cost and poor effect of offline training of the large model in the prior art.

[0007] The present application provides a distributed large model, comprising the following steps: Multiple intelligent computing units (AIUs) are used for data training and inference; each AIU is distributed in the base station and central computer room of the computing power network. Multiple intelligent controllers (AICs) are used to integrate control model parameter information; each AIC is distributed and set up in the control room of the computing power network. After receiving user input data, the parameters of each AIU and each AIC are updated level by level based on the difference between the inference results of the distributed large model and the source data information; the source data information is the original output data corresponding to the input data.

[0008] According to the distributed large model provided in this application, a single hidden layer of the distributed large model includes multiple parallel blocks, which are used to extract features from the received input data; Each of the aforementioned Blocks is distributed within the computing network.

[0009] According to a distributed large model provided in this application, the input data and output data of the Block are signal-encoded data, and the signal-encoded data is bit-based encoded data.

[0010] According to a distributed large model provided in this application, after receiving user input data, the parameters of each AIU and each AIC are updated level by level based on the difference between the inference result of the distributed large model and the source data information, including: After receiving the signal encoding data corresponding to the user input data, the distributed large model performs inference based on the signal encoding data corresponding to the user input data to obtain the corresponding output signal encoding data. Based on the difference between the signal encoding data corresponding to the source data information and the output signal encoding data, the parameters of each AIU and each AIC are updated level by level.

[0011] According to a distributed large model provided in this application, the AIU includes: The AIU controller is used to synchronously distribute the signal encoding data and the encryption rules corresponding to the signal encoding data sent by the active antenna unit to the buffer and the block. The buffer is used to cache the data fingerprint of the signal-encoded data and the user identity information of the user who sent the signal-encoded data; The Block is used to extract features from the signal encoded data, send the feature extraction results to the buffer for caching, and distribute computing tasks to Blocks of the same level, or request the AIC to forward and distribute them to other Blocks.

[0012] According to a distributed large model provided in this application, if the Block uses an attention mechanism, the key and value obtained from feature extraction are sent to the cache for caching. When the intelligent computing unit receives input signal encoded data that is the same as the data fingerprint, it appends the value corresponding to the data fingerprint to the input signal encoded data to perform feature extraction.

[0013] According to a distributed large model provided in this application, if the Block uses an attention mechanism, the key and value obtained from feature extraction are sent to the cache for caching. When the intelligent computing unit receives input signal encoded data that is identical to the user identity information, it appends the Key corresponding to the user identity information to the input signal encoded data for feature extraction.

[0014] According to a distributed large model provided in this application, the AIC includes: The VNN layer is used to summarize and calculate the feature extraction results of each Block in a single hidden layer. The dimension of the VNN layer is variable. An L buffer is used to store an L list, which records the weight values ​​of the current parameters of the VNN layer. The AIC controller is used to control the dimension change of the VNN layer based on the data changes in the L buffer; A cache is used to cache the AIUs that can be called by the AIC.

[0015] According to a distributed large model provided in this application, the AIC controller controls the dimensionality change of the VNN layer based on the following steps: During the training of the distributed large model, if the value of the L list changes by a magnitude greater than the first threshold and the loss function value decreases, then the dimension of the VNN layer remains unchanged. If the value of the L list changes by a greater than the first threshold and the loss function value increases, then the dimension of the VNN layer is increased. If the change in the value of the L list is less than the second threshold, and the loss function value is less than or equal to the preset threshold, then the dimension of the VNN layer remains unchanged; the second threshold is less than the first threshold. If the change in the value of the L list is less than the second threshold, and the loss function value is greater than the preset threshold, then the dimension of the VNN layer is reduced.

[0016] According to a distributed large model provided in this application, the step of updating the parameters of each AIU and each AIC level by level based on the difference between the inference results of the distributed large model and the source data information includes: Initialize the confidence probability and model parameters of each node; the nodes include the AIU and the AIC; At each node, after inference based on the current model parameters, the loss function value of the node is calculated; The weighted loss of each node is determined based on the product of the loss function value and the confidence probability of each node. The weighted losses of each node are aggregated to obtain the total loss value; Based on the total loss value, the gradient of each node is determined; The model parameters and confidence probabilities of each node are updated based on the gradient of each node. Iteratively update the model parameters and confidence probabilities of each node until the iteration stops.

[0017] The distributed large model provided in this application includes multiple Intelligent Computing Units (AIUs) for data training and inference; each AIU is distributed in the base station and central computer room of the computing power network; multiple Intelligent Controllers (AICs) are used to control the integration of model parameter information; each AIC is distributed in the control computer room of the computing power network; after receiving user input data, based on the difference between the inference result of the distributed large model and the source data information, the parameters of each AIU and each AIC are updated level by level; the source data information is the original output data corresponding to the input data. The computing power network itself has intelligent computing and storage capabilities, and naturally possesses distributed computing characteristics, which is consistent with the parallel computing characteristics of the large model encoder. Therefore, this application decouples the basic structure of the large model and integrates it with the computing network resources to realize a distributed large model. The flexible access of nodes allows its parameter scale to easily exceed that of the existing largest model, solving the problem that the parameter scale of the large model is not easy to flexibly expand. Furthermore, the decomposition of the large model modules allows for more flexible configuration of computing power and improves the generation efficiency of the large model. Furthermore, by comparing the source data with the output data of the distributed large model, unsupervised training of the distributed large model is achieved during its operation. The model parameters are continuously and dynamically updated through a large amount of user data, reducing training costs, improving the training flexibility of the large model, and thus continuously optimizing the generation effect of the large model. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is one of the structural diagrams of the distributed large model provided in this application; Figure 2 This is a schematic diagram of the data transmission process provided in this application; Figure 3 This is the second schematic diagram of the distributed large model provided in this application; Figure 4 This is a schematic diagram comparing the encoding of the distributed large model provided in this application with existing large models; Figure 5 This is a schematic diagram of the AIU structure provided in this application; Figure 6 This is a schematic diagram of the AIC structure provided in this application; Figure 7 This is a schematic diagram of the VNN layer provided in this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] It should be noted that in the description of the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Unless otherwise expressly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly, for example, they can be fixed connections, detachable connections, or integral connections; they can be mechanical connections or electrical connections; they can be direct connections or indirect connections through an intermediate medium; and they can be internal connections between two elements. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0022] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0023] Currently, large-scale models are based on a multi-layered structure of encoders / decoders and Transformers. Models are centrally deployed in the same computing environment, with a fixed structure and computational resources determined before deployment and training. There are three main technical drawbacks: ① Large models cannot dynamically expand their structural parameters (it has been proven that the number of parameters in a large model directly determines its upper limit of accuracy); ② Large models cannot learn online and continuously update parameters (each version iteration requires offline training); ③ Large models involve repeated input and output encoding (network transmission encodes information into a data stream, the model's input needs to use the decoding result, and the output needs to be encoded again, resulting in a repetition with each access).

[0024] To address the above-mentioned problems, this application proposes the following embodiments.

[0025] The following is combined Figures 1-7 This application describes the distributed large model provided in its embodiments.

[0026] Figure 1 This is one of the structural diagrams of the distributed large model provided in this application, such as... Figure 1 As shown, in order to address the aforementioned drawback ②, the distributed large-scale model provided in this application includes the following: Multiple intelligent computing units (AIUs) are used for data training and inference; each AIU is distributed in the base station and central computer room of the computing power network. Multiple intelligent controllers (AICs) are used to integrate control model parameter information; each AIC is distributed and set up in the control room of the computing power network. After receiving user input data, the parameters of each AIU and each AIC are updated level by level based on the difference between the inference results of the distributed large model and the source data information; the source data information is the original output data corresponding to the input data.

[0027] Figure 1 A schematic diagram of an exemplary architecture that can be used in this distributed large-scale model is shown. For descriptive purposes, the illustrated architecture is merely an example of a suitable environment and does not imply any limitation on the scope or functionality of this application. Nor should this computing system be construed as an extension of... Figure 1 Any component shown or a combination thereof has any dependencies or requirements.

[0028] Here, the central computer room refers to a computer room that centrally manages and stores critical computing resources, data storage devices, and network facilities, and houses multiple servers. The control computer room is a computer room used for monitoring and controlling specific systems or processes.

[0029] like Figure 1 As shown, the dashed lines and the solid lines excluding AIU and AIC constitute the existing large-scale model service solutions. Due to the characteristics of large models—large parameter scale, large computational resource requirements, and large data requirements—the three parts of the dashed lines require a huge investment of resources, which are not shared between different organizations, resulting in ineffective waste of repeated data collection and model training. The solid lines represent the fusion solution provided in this application, proposing an intelligent computing unit (AIU) and an intelligent controller (AIC). The AIU is responsible for data training and inference, while the AIC controls the integration of parameter information, the synchronization of user status information, and the distribution of control protocols.

[0030] In existing technologies, when a user has a source (the user's query yields accurate results derived from collected raw information, not content generated by the model's own reasoning), which is the basic function of current communication networks, the user transmits data, the network transmits data, the target source receives and processes the data, and then transmits the processed results along with the original data; the network transmits the data again, and the user retrieves the data. This process is as follows: Figure 2 As shown, the data link is as follows: User inputs image, voice, text, and other information → Base station AAU (Active Antenna Unit) → Local central equipment room DU (Distributed Unit) → Core network central equipment room CU (Centralized Unit, i.e., control room) → Other central equipment rooms CU → Source data information (image, voice, text, and other information) → Other central equipment rooms CU → Core network central equipment room CU → Local central equipment room DU → Base station AAU → User receives information.

[0031] This process has a clearly defined data source, which can be used for training distributed large-scale models, with user input and source data information serving as training data. Therefore, this application adds an extra data link to the basic link: Base Station AAU → Base Station AIU → Local Center Room DU → Local Center Room AIU → Core Network Center Room CU → Core Network Center Room AIC → Core Network Center Room CU → Other Center Room CU → Other Center Room AIU. The training feedback inference results are compared and calculated with the source data information, and the weight parameters of the large model distributed on each AIU are updated level by level, thereby realizing online learning of the large model and dynamically updating the model parameters.

[0032] Specifically, starting from the terminal initiating a data acquisition request, including but not limited to input data in various modalities such as text, images, video, and voice, the base station's AAU is responsible for radio frequency processing and signal modulation, including data encoding and encryption. The encoded data and encryption rules are transmitted to the base station AIU, which performs basic feature extraction and then forwards the processed parameters to the AIU in the local central computer room. The AIU in the local central computer room retains the link latent space encoding information of the input data and undertakes a portion of the computation. The remaining computational workload and data are transmitted to the core network central control room via the backbone network. The core network central control room's AIC distributes the computational requirements and data to other central computer rooms for computation and synchronization. The CU is responsible for handling non-real-time protocols and services, while the DU, as a distributed unit, is responsible for handling physical layer protocols and real-time services.

[0033] After each node completes its calculation, the core network central control (AIC) is responsible for synchronizing the distributed cluster data and calculations. The final calculation results are transmitted level by level through the core network bearer network to the corresponding local central computer room. The response data of the CU and DU are transmitted back using the same link. The local AIU is responsible for calculating the difference between the model output results and the actual response data of the distributed large model by combining the local link latent space encoding information, and then transmitting the results back to the AIC. The AIC selects a certain gradient update method to update the model parameters.

[0034] When a user uses the network without access to source data, they directly utilize the distributed large-scale model of the computing power network. The DU (Data Utility Unit) and CU (Data Controller Unit) do not participate in the transmission of data sources via the network. The distributed large-scale model directly outputs the prediction results and sends them back to the user through the DU and CU. Compared to active use, passive use lacks access to source data; the distributed large-scale model directly infers without updating model parameters.

[0035] The distributed large model provided in this application includes multiple Intelligent Computing Units (AIUs) for data training and inference; each AIU is distributed in the base station and central computer room of the computing power network; multiple Intelligent Controllers (AICs) are used to control the integration of model parameter information; each AIC is distributed in the control room of the computing power network; after receiving user input data, based on the difference between the inference result of the distributed large model and the source data information, the parameters of each AIU and each AIC are updated level by level; the source data information is the original output data corresponding to the input data. The computing power network itself has intelligent computing and storage capabilities, and naturally possesses distributed computing characteristics, which is consistent with the parallel computing characteristics of the large model encoder. Therefore, this application decouples the basic structure of the large model and integrates it with the computing network resources to realize a distributed large model. The flexible access of nodes allows its parameter scale to easily exceed that of the existing largest model, solving the problem that the parameter scale of the large model is not easy to flexibly expand. Furthermore, the decomposition of the large model modules allows for more flexible configuration of computing power and improves the generation efficiency of the large model. Furthermore, by comparing the source data with the output data of the distributed large model, unsupervised training of the distributed large model is achieved during its operation. The model parameters are continuously and dynamically updated through a large amount of user data, reducing training costs, improving the training flexibility of the large model, and thus continuously optimizing the generation effect of the large model.

[0036] To address the aforementioned drawbacks①, the distributed large-scale model in this application decomposes a single layer of the large-scale model into multiple blocks connected in parallel. Specifically: In an optional embodiment, a single hidden layer of the distributed large model includes multiple parallel blocks, which are used to extract features from the received input data; Each of the aforementioned Blocks is distributed within the computing network.

[0037] Figure 3 This is the second structural diagram of the distributed large model provided in this application, as shown below. Figure 3 As shown, the Block serves as the basic unit, and distributed large models are realized through Blocks in different environments of the parallel computing network. The same-level expansion (increasing the width of each layer without increasing the number of network layers) increases the number of Blocks in a single layer, which can improve the parameter scale of a single layer of the large model and achieve flexible scaling.

[0038] It should be noted that, Figure 3 This is just an illustrative diagram; the specific structure of a Block is not limited here. Blocks can use different basic components according to actual usage requirements. For example, a Transformer can be used to replace the Query-Key-Value part of the Block in the diagram, allowing for flexible replacement.

[0039] The distributed large model provided in this application splits the single layer of the large model into multiple blocks connected in parallel, thereby enabling dynamic expansion of the structural parameters of the large model through peer-to-peer expansion. This allows for the creation of large models that exceed the existing maximum parameter scale, thereby improving model accuracy.

[0040] To address the aforementioned drawback ③, the distributed large model in this application is directly trained and inferred based on bit encoding. Specifically: In an optional embodiment, the input and output data of the Block are signal-encoded data, which are bit-based encoded data.

[0041] During network transmission, multiple modal data such as language, speech, and images naturally need to be encoded into bit data for compressed transmission, which is very similar to the basic encoder-decoder structure of large models. Therefore, in this application embodiment, the round-trip data in network transmission is used as the knowledge source for model training.

[0042] like Figure 3 As shown, the input and output of Block are both signal-coded data, thus solving the problem of repeated encoding of data into signal-coded data in large models at this stage.

[0043] Furthermore, after receiving user input data, the step of updating the parameters of each AIU and each AIC level by level based on the difference between the inference results of the distributed large model and the source data information includes: After receiving the signal encoding data corresponding to the user input data, the distributed large model performs inference based on the signal encoding data corresponding to the user input data to obtain the corresponding output signal encoding data. Based on the difference between the signal encoding data corresponding to the source data information and the output signal encoding data, the parameters of each AIU and each AIC are updated level by level.

[0044] Figure 4 This is a diagram comparing the encoding of the distributed large-scale model provided in this application with existing large-scale models, as shown below. Figure 4 As shown above, the existing large model encoding process encodes user-input images, videos, voice, text, and other content into bit-based signal encodings for transmission. The device deploying the large model receives the signal encodings via the network, decodes them, and then performs format encoding to convert the data into a format that the large model can process. The large model performs inference based on the format encodings, generates format-encoded outputs, and then converts them back into signal encodings for transmission. The user receives the signal encodings, decodes them, and obtains the processing results from the large model.

[0045] The following processing flow of the distributed large-scale model is the encoding flow provided in the embodiments of this application. Specifically, the input and output of the Block are both signal-encoded data. After receiving the bit-form signal encoding, the distributed large-scale model directly uses the signal encoding for inference, solving the problem of repeated encoding of data into signal-encoded data in current large-scale models. By comparing the output signal-encoded data of the distributed large-scale model with the signal-encoded data obtained by the user through active access, unsupervised training of the distributed large-scale model can be achieved during operation, solving the problems of online learning and data acquisition difficulties of the model.

[0046] The distributed large model provided in this application utilizes the similarity between bit signal encoding and the basic encoder-decoder structure of the large model. The input and output of the block are both signal-encoded data. The large model directly uses the signal-encoded data for training, inference, and generation, avoiding the problem of repeated encoding of data into signal-encoded data by the large model, thereby improving data processing efficiency and resource utilization.

[0047] In an optional embodiment, the AIU includes: The AIU controller is used to synchronously distribute the signal encoding data and the encryption rules corresponding to the signal encoding data sent by the active antenna unit to the buffer and the block. The buffer is used to cache the data fingerprint of the signal-encoded data and the user identity information of the user who sent the signal-encoded data; The Block is used to extract features from the signal encoded data, send the feature extraction results to the buffer for caching, and distribute computing tasks to Blocks of the same level, or request the AIC to forward and distribute them to other Blocks.

[0048] It should be noted that the AIU provided in this application is a software-defined parallel computing component that can be deployed on general-purpose computing devices, just like AAU, DU, and CU. It is responsible for receiving signal-encoded data and performing multi-layer encoding and decoding calculations for large model components (Blocks), extracting the latent space key (Key) of the input information query, outputting the value information after encoding, and connecting the query and Key results during computation to adapt to the input-output connection problems unique to distributed large models.

[0049] Figure 5 This is a schematic diagram of the AIU structure provided in this application, such as... Figure 5 As shown, the left side is the AIU structure diagram, the upper right side is the existing large model computation process, and the lower right side is the distributed large model computation process provided in this application.

[0050] The terminal initiates a data acquisition request, including but not limited to input data in various modalities such as text, images, video, and voice. The base station, with the AAU responsible for radio frequency processing and signal modulation, including data encoding and encryption, transmits the encoded data and encryption rules to the AIU. The AIU controller receives the data and synchronously distributes it to the buffer and the Block. The buffer caches the data fingerprint and user identity information. The Block performs basic feature extraction and records the Block's UUID (Universally Unique Identifier) ​​in the first row and first column of the model's path encoding table as the start identifier for the operation. At this time, the Block can also distribute the computation task to other peer Blocks with direct management authority, or request the AIC to relay access to other Blocks, perform computation on the current layer of the model, and send the Key and result Value to the buffer for caching.

[0051] When the path encoding table is full of UUIDs, the distributed large model computation of the computing network is completed. Finally, all block computation data is synchronized to the AIC. The AIC then sends the data and the resources accessed by the active block to the starting block. When the controller receives all the resources and model output data, it indicates that the training process has begun. The multilayer perceptron (MLP) performs a final summary output on the model output data and uses the loss module to calculate the difference between the output result and the resource information. The gradient information is then transmitted back to each block through the AIC, and each block updates its weight parameters. When the controller only receives the model output data, the MLP summarizes and outputs it directly without updating the weight parameters.

[0052] Here, a data fingerprint refers to a fixed-length string generated by applying a specific hash function to data. This string is a compact representation of the original dataset and is used to verify data integrity, identify data, or serve as a unique identifier for the data.

[0053] The distributed large-scale model provided in this application uses Intelligent Computing Units (AIUs) to perform calculations through parallel blocks. This distributed parallel processing of computational tasks via AIU blocks improves computational efficiency. Furthermore, the number of computing units can be dynamically expanded by adding AIUs, allowing for flexible selection of the computational scale based on the complexity of the application scenario. In practical applications, training can begin with a small number of nodes, gradually adding nodes horizontally and vertically for slow evolution, resulting in a more stable training process.

[0054] In an optional embodiment, if the Block uses an attention mechanism, the key and value obtained from feature extraction are sent to the cache for caching; When the intelligent computing unit receives input signal encoded data that is the same as the data fingerprint, it appends the value corresponding to the data fingerprint to the input signal encoded data to perform feature extraction.

[0055] In this embodiment of the application, when the same or similar data fingerprint is requested again, the cached value corresponding to the data fingerprint is appended to the input data, and the data with the appended value is used as a whole for calculation.

[0056] The distributed large model provided in this application records data fingerprints through a cache. When input data with the same data fingerprint is received, the stored value is appended to the input data, thereby providing richer input data and optimizing the output of the large model.

[0057] In an optional embodiment, if the Block uses an attention mechanism, the key and value obtained from feature extraction are sent to the cache for caching; When the intelligent computing unit receives input signal encoded data that is identical to the user identity information, it appends the Key corresponding to the user identity information to the input signal encoded data for feature extraction.

[0058] In this embodiment of the application, when the same user identity information is requested again, the cached key is appended to the input data and calculated as a whole. For example... Figure 5 As shown in the upper right corner, Q represents the query, V represents the value, and K represents the key. Existing large models require multiple rounds of dialogue to supplement input information, thereby improving the accuracy of the model's output. For example... Figure 5As shown in the lower right corner, the distributed large model provided in this application caches Key and Value through a cache, which can reduce the number of multi-turn dialogues. Unlike the existing QKV structure that directly obtains the final Value, the Block in this application uses the cached implicit space Key to continuously supplement the Query and improve the input to replace the guidance work of the long context model in multi-turn dialogues.

[0059] Existing large-scale models, being offline and stateless, cannot resolve ambiguity issues (e.g., "Xiaomi" can mean both a food brand and a mobile phone brand) or issues related to abbreviations in professional fields. However, the distributed large-scale model provided in this application, when the same user identity information is requested again, appends the key to the input information, continuously supplements the query, reduces dialogue rounds, retains rich historical information in queries with long contexts, and can provide answers for specific fields based on user identity information, providing more accurate reasoning and generation results, and efficiently realizing personalized intelligent question answering for users.

[0060] In an optional embodiment, the AIC includes: The VNN layer is used to summarize and calculate the feature extraction results of each Block in a single hidden layer. The dimension of the VNN layer is variable. An L buffer is used to store an L list, which records the weight values ​​of the current parameters of the VNN layer. The AIC controller is used to control the dimension change of the VNN layer based on the data changes in the L buffer; A cache is used to cache the AIUs that can be called by the AIC.

[0061] It should be noted that AIC is a software-defined parallel computing control component that can be deployed on x86 general-purpose computing devices, just like AAU, DU, and CU. It is responsible for controlling the data and computation synchronization of distributed blocks.

[0062] like Figure 3 As shown, the distributed large-scale computational network model decomposes a single layer of the large model into multiple blocks connected in parallel. In the deep computation of the large model, the AIC controls the sequential computation of these blocks. Specifically, the current-layer computation information of all parallel blocks is input to the AIC.

[0063] Figure 6 This is a schematic diagram of the AIC structure provided in this application, such as... Figure 6As shown, the AIC obtains the outputs of multiple blocks from the distributed large model as input. If the controller does not obtain the loss of a block, it indicates that it is inference and directly distributes the computation task to other blocks. If it receives the loss, it indicates that it is training and the loss data is placed in the loss buffer. The AIC controller first calculates the changes in the L buffer data to determine whether to change the dimensionality of the VNN (Variable Neural Network). The loss calculated by the VNN is then distributed by the controller to the next layer of computation blocks. The controller is also responsible for resource management of distributed computing. The cascading buffer caches the AIU computing resources that can be called. Idle AIU computing power is actively pushed to the AIC, which caches it in the cascading buffer. When the AIC needs to compute, it can access the cascading AIU for computation.

[0064] Understandably, to improve the strength of distributed large-scale models, if the number of model parameters in the AIC layer is too small, computational forgetting will occur after very deep computation. Conversely, if the number of parameters is too large, training will be difficult to converge, resulting in significant waste of computational resources. The distributed large-scale model provided in this application caches the dimensions of the VNN layer through an L-cacher, enabling variable dimensions of the aggregation layer. This allows the dimensions of the VNN layer to be adjusted according to the prediction accuracy during model training, thereby obtaining a suitable number of parameters and balancing model computational cost and accuracy.

[0065] In an optional embodiment, the AIC controller controls the dimensionality change of the VNN layer based on the following steps: During the training of the distributed large model, if the value of the L list changes by a magnitude greater than the first threshold and the loss function value decreases, then the dimension of the VNN layer remains unchanged. If the value of the L list changes by a greater than the first threshold and the loss function value increases, then the dimension of the VNN layer is increased. If the change in the value of the L list is less than the second threshold, and the loss function value is less than or equal to the preset threshold, then the dimension of the VNN layer remains unchanged; the second threshold is less than the first threshold. If the change in the value of the L list is less than the second threshold, and the loss function value is greater than the preset threshold, then the dimension of the VNN layer is reduced.

[0066] Here, the first threshold is a large value, indicating that the value of L changes drastically. Its specific value can be adaptively set according to actual usage requirements. The second threshold is a small value, indicating that the value of L changes little or even remains unchanged, and the model parameters tend to be stable. Its specific value can also be adaptively set according to actual usage requirements. The preset threshold is a value close to 0, indicating that the loss function value is extremely small, and the model has been trained to the optimal solution or a local optimum. Its specific value can also be adaptively set according to actual usage requirements.

[0067] Figure 7 This is a schematic diagram of the VNN layer provided in this application, such as... Figure 7 As shown, a variable-dimensional base model layer (VNN) is used to address the adaptability issue of distributed large models in the aggregation computation layer (AIC). The VNN layer of the AIC adds an L list to record the weight values ​​of the current VNN parameters. This layer can flexibly add new nodes to increase the parameter scale of the current AIC layer aggregation, thereby realizing a variable-dimensional model layer.

[0068] Specifically, by monitoring the parameter change trends of the model layers, we attempt to add or remove parameter dimensions to make the training process more stable. The dimensionality adjustment strategy is as follows: if the value of L changes more than a first threshold during training (i.e., L changes drastically), and the network's loss function value is decreasing, it indicates normal training and VNN dimensionality adjustment is unnecessary. If the network's loss function value is increasing, it indicates that knowledge has been forgotten during training, the current VNN parameter size is too small to adapt to broader data fitting requirements, and dimensionality increase is needed. If the value of L changes little, and the network's loss function value is relatively stable and less than or equal to a preset threshold (tending towards 0), it indicates that training has converged and VNN dimensionality adjustment is unnecessary; conversely, it indicates that training has fallen into a local optimum, the VNN parameter dimension is too high, making gradient updates difficult, and VNN dimensionality reduction is needed.

[0069] The distributed large model provided in this application determines the training stage and training effect of the model based on the change magnitude of the L list and the loss function value, thereby dynamically adjusting the dimension of the VNN layer to obtain a suitable dimension value, avoiding the computational forgetting problem caused by too few parameters, and avoiding the problem of high training cost caused by too many parameters.

[0070] In an optional embodiment, the step of updating the parameters of each AIU and each AIC level by level based on the differences between the inference results of the distributed large model and the source data information includes: Initialize the confidence probability and model parameters of each node; the nodes include the AIU and the AIC; At each node, after inference based on the current model parameters, the loss function value of the node is calculated; The weighted loss of each node is determined based on the product of the loss function value and the confidence probability of each node. The weighted losses of each node are aggregated to obtain the total loss value; Based on the total loss value, the gradient of each node is determined; The model parameters and confidence probabilities of each node are updated based on the gradient of each node. Iteratively update the model parameters and confidence probabilities of each node until the iteration stops.

[0071] In a computing-powered network environment, data transmission distances are long, computing power varies across different environments, and computational reliability is inconsistent. Current large-scale model training methods cannot solve the problems of computational fault tolerance and consistency, making them unsuitable for such environments. Therefore, this application introduces a trust probability, which is also a parameter that needs to be optimized for the entire network (i.e., it assumes that the results of Block computations are not 100% reliable and may even result in computational loss). The overall calculation formula is expressed as follows: ; Here, AI Model is a large distributed model that has been trained; Optimize is the optimization function, which usually uses gradient descent or other optimization algorithms; Loss Function is the loss function used to measure the difference between the model's predicted value and the actual value; Parameters are the model parameters, namely the weights and biases of the neural network. It is a set of trust probabilities for distributed nodes.

[0072] Specifically, the model parameters and confidence probabilities are first initialized randomly, the model output is calculated at each node using the current parameters, and then the loss function is used to calculate the loss at each node.

[0073] Preferably, the standard cross-entropy loss function is used to calculate the local loss for each node i: ; in, It is the data encoding predicted by the model. This retrieves the source data, where j is the output encoding position within the node. It is the data encoding output by the corresponding node. This refers to the source data for the corresponding node, meaning minimizing the difference between the predicted value and the true label of the local node. Since the output source of a node in a distributed large-scale computing network model is only one, it is not possible to know... The true value of a node at a given location can be used in actual training as follows: and They are equal. Then, the loss of each node is compared with the confidence probability. Multiply to obtain the weighted loss. To calculate the total loss during training, the weighted loss needs to be aggregated across all nodes, for example, using a weighted average: .

[0074] After obtaining the training loss value, the gradient of each node is calculated through backpropagation, and then the model parameters of the nodes are updated using the gradient descent algorithm, and the process is repeated continuously.

[0075] Here, the iteration stopping conditions may include, but are not limited to, reaching a predetermined number of iterations, or the loss value falling below a certain threshold.

[0076] The distributed large model provided in this application introduces trust probability. During the model training process, the trust probability is optimized, enabling the distributed large model to simultaneously possess surface reasoning and deep reasoning. This enhances training controllability and solves the problem of difficulty in model miniaturization.

[0077] In summary, the distributed large-scale model provided in this application allows for flexible node access, enabling flexible upgrades to the model's parameter scale and achieving models exceeding the current maximum parameter scale. Directly introducing signal encoding for computation reduces computational redundancy caused by repeated data encoding and makes the multimodal input-output format more unified. Training can be enhanced by adding new nodes after a small number of nodes have been trained, resulting in a more stable training process. Local computing link relationships and network user authentication enable the model to possess personalized reasoning capabilities, enhancing the effectiveness of the personalized large-scale model and reducing redundant interactions in multi-turn dialogue-guided models.

[0078] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A distributed large-scale model, characterized in that, include: Multiple intelligent computing units (AIUs) are used for data training and inference; each AIU is distributed in the base station and central computer room of the computing power network. Multiple intelligent controllers (AICs) are used to integrate control model parameter information; each AIC is distributed and set up in the control room of the computing power network. After receiving user input data, based on the differences between the inference results of the distributed large model and the source data information, the parameters of each AIU and each AIC are updated level by level; The source data information is the original output data corresponding to the input data.

2. The distributed large model according to claim 1, characterized in that, A single hidden layer of the distributed large model comprises multiple parallel blocks, which are used to extract features from the received input data. Each of the aforementioned Blocks is distributed within the computing network.

3. The distributed large-scale model according to claim 2, characterized in that, The input and output data of the Block are signal-encoded data, which are bit-based encoded data.

4. The distributed large model according to claim 3, characterized in that, After receiving user input data, the parameters of each AIU and each AIC are updated level by level based on the differences between the inference results of the distributed large model and the source data information, including: After receiving the signal encoding data corresponding to the user input data, the distributed large model performs inference based on the signal encoding data corresponding to the user input data to obtain the corresponding output signal encoding data. Based on the difference between the signal encoding data corresponding to the source data information and the output signal encoding data, the parameters of each AIU and each AIC are updated level by level.

5. The distributed large model according to claim 3, characterized in that, The AIU includes: The AIU controller is used to synchronously distribute the signal encoding data and the encryption rules corresponding to the signal encoding data sent by the active antenna unit to the buffer and the block. The buffer is used to cache the data fingerprint of the signal-encoded data and the user identity information of the user who sent the signal-encoded data; The Block is used to extract features from the signal encoded data, send the feature extraction results to the buffer for caching, and distribute computing tasks to Blocks of the same level, or request the AIC to forward and distribute the tasks to other Blocks.

6. The distributed large-scale model according to claim 5, characterized in that, If the Block uses an attention mechanism, the key and value obtained from feature extraction are sent to the cache for caching. When the intelligent computing unit receives input signal encoded data that is the same as the data fingerprint, it appends the value corresponding to the data fingerprint to the input signal encoded data to perform feature extraction.

7. The distributed large-scale model according to claim 5, characterized in that, If the Block uses an attention mechanism, the key and value obtained from feature extraction are sent to the cache for caching. When the intelligent computing unit receives input signal encoded data that is identical to the user identity information, it appends the Key corresponding to the user identity information to the input signal encoded data for feature extraction.

8. The distributed large model according to claim 5, characterized in that, The AIC includes: The VNN layer is used to summarize and calculate the feature extraction results of each Block in a single hidden layer. The dimension of the VNN layer is variable. An L buffer is used to store an L list, which records the weight values ​​of the current parameters of the VNN layer. The AIC controller is used to control the dimension change of the VNN layer based on the data changes in the L buffer; A cache is used to cache the AIUs that can be called by the AIC.

9. The distributed large model according to claim 8, characterized in that, The AIC controller controls the dimensionality change of the VNN layer based on the following steps: During the training of the distributed large model, if the value of the L list changes by a magnitude greater than the first threshold and the loss function value decreases, then the dimension of the VNN layer remains unchanged. If the value of the L list changes by a greater than the first threshold and the loss function value increases, then the dimension of the VNN layer is increased. If the change in the value of the L list is less than the second threshold, and the loss function value is less than or equal to the preset threshold, then the dimension of the VNN layer remains unchanged; the second threshold is less than the first threshold. If the change in the value of the L list is less than the second threshold, and the loss function value is greater than the preset threshold, then the dimension of the VNN layer is reduced.

10. The distributed large model according to any one of claims 1-9, characterized in that, The method of updating the parameters of each AIU and each AIC level by level based on the differences between the inference results and source data information of the distributed large model includes: Initialize the confidence probability and model parameters of each node; the nodes include the AIU and the AIC; At each node, after inference based on the current model parameters, the loss function value of the node is calculated; The weighted loss of each node is determined based on the product of the loss function value and the confidence probability of each node. The weighted losses of each node are aggregated to obtain the total loss value; Based on the total loss value, the gradient of each node is determined; The model parameters and confidence probabilities of each node are updated based on the gradient of each node. Iteratively update the model parameters and confidence probabilities of each node until the iteration stops.

Images