Cloud management platform-based data processing method and related apparatus
Patent Information
- Application Number
- PCT/CN2026/076882
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-02-27
- Filing Date
- 2026-02-03
- Publication Date
- 2026-09-03
Smart Images
Figure CN2026076882_03092026_PF_FP_ABST
Abstract
Description
A data processing method and related apparatus based on a cloud management platform
[0001] This application claims priority to Chinese patent application No. 202510235307.8, filed on February 27, 2025, entitled "A Data Processing Method and Related Device Based on a Cloud Management Platform", the entire contents of which are incorporated herein by reference. Technical Field
[0002] This application relates to the field of communication technology, and in particular to a data processing method and related apparatus based on a cloud management platform. Background Technology
[0003] As large language models (LLMs) are increasingly used in various tasks, their powerful reasoning capabilities have become a core competitive advantage in complex scenarios such as code generation, mathematical proof, and problem-solving. However, current large language models rely on stepwise reasoning, which typically requires significant computational resources and inference time, causing the inference cost to increase exponentially with model size and task complexity. In practical applications, this high inference cost and latency severely limit the widespread adoption of large language models, especially in scenarios requiring real-time responses, such as dialogue systems, intelligent assistants, and autonomous driving.
[0004] Current inference techniques typically improve inference accuracy by increasing the number of inference steps and time. However, while this process can improve inference quality, it also significantly increases computational costs and response time, failing to meet the real-time and cost-effectiveness requirements of many practical applications.
[0005] Therefore, while improving the reasoning ability of large language models, how to reduce the reasoning cost has become an urgent problem to be solved. Summary of the Invention
[0006] This application provides a data processing method based on a cloud management platform, which can reduce the computational overhead of model inference while increasing inference speed.
[0007] In a first aspect, this application provides a data processing method based on a cloud management platform, wherein the cloud management platform is used to manage the infrastructure providing cloud services, the infrastructure including a storage system and a cluster of computing nodes, and the method includes:
[0008] The cloud management platform acquires a question sent by a user; when the cloud management platform determines that there is a first inference result matching the question in the first inference result set pre-stored in the storage system, it determines the answer to the question based on the first inference result; each inference result in the first inference result set is obtained by the computing node cluster through historical inference using a large language model; when the cloud management platform determines that there is no inference result matching the question in the first inference result set, the computing node cluster determines the answer to the question based on the question using a large language model.
[0009] The idea behind this application is that, for some problems, the large language model may have already performed reasoning on similar problems in the past. The reasoning results can be directly reused without re-reasoning, thereby reducing the computational overhead of model reasoning. For example, in this application embodiment, the storage system can store a first set of reasoning results obtained from the large language model's historical reasoning. For a user-inputted question, it can first be determined whether there is a reasoning result in the first set that matches the user's input question. If it is determined that there is a first reasoning result in the pre-stored first set that matches the question, the first reasoning result can be used directly to determine the answer to the question, without needing to use the large language model to reason about the question. This increases reasoning speed while reducing the computational overhead of model reasoning. Only when it is determined that there is no reasoning result in the first set that matches the question is the answer determined by the large language model based on the question.
[0010] In one possible implementation, the first inference result includes a query and the answer to the query; the first inference result matches the question when a preset condition is met between the first inference result and the question, the preset condition including at least one of the following: the similarity between the query and the question is greater than a threshold, and the uncertainty of the first inference result itself is lower than a threshold.
[0011] For example, the similarity between the query and the question can be quantified by calculating the distance between the feature vectors of the question and the feature vectors of the query. The uncertainty of the first inference result itself can be determined by the probability distribution of multiple inference results that are connected to the question. When the distribution is very uniform, the uncertainty of the first inference result itself can be considered to be relatively large.
[0012] In one possible implementation, the first inference result includes a query and the answer to the query; determining the answer to the question based on the target inference result includes: obtaining a supplemented question based on the answer to the query; determining the answer to the question based on the supplemented question when a second inference result matching the supplemented question exists in a pre-stored set of second inference results, wherein each inference result in the second inference result set has an association with the first inference result; and determining the answer to the question based on the supplemented question using a large language model when no inference result matching the supplemented question exists in the second inference result set.
[0013] In this embodiment of the application, if the answer in the first reasoning result can already answer the user's question, then the reply can be made directly based on the answer in the first reasoning result. If the answer in the first reasoning result is insufficient to answer the user's question, then the question can be supplemented directly based on the answer in the first reasoning result to obtain the supplemented question, and then the next round of processing can be carried out. In this embodiment, each inference result in the second inference result set is derived from historical reasoning by the large language model based on the query (or similar query) in the first inference result set. That is, each inference result in the second inference result set may serve as an answer to the supplemented question. In this embodiment, it can be assumed that there is an association between each inference result in the second inference result set and the first inference result set. This association can be represented by directed edges in a directed graph. The role of this association is that after determining a query, inference results associated with that query (in the inference result set) can be directly used as candidates for inference results in subsequent rounds of that query. Then, it can be determined from the second inference result set whether there is a matching second inference result for the supplemented question. If it is determined that there is a second inference result in the pre-stored second inference result set that matches the supplemented question, the answer to the question is determined based on the second inference result. If it is determined that there is no inference result in the second inference result set that matches the supplemented question, the answer to the question is determined based on the supplemented question using the large language model.
[0014] In one possible implementation, each inference result in the second inference result set has an association with the first inference result and a probability value corresponding to the association; the method further includes: determining the second inference result from the second inference result set by sampling based on the probability value; and matching the second inference result with the supplemented question when a preset condition is met between the second inference result and the supplemented question.
[0015] In one possible implementation, the method further includes: increasing the probability value corresponding to the association between the second inference result and the first inference result. By autonomously maintaining the probability value of the association between the inference results, memory retrieval and large language model behaviors are similar, while the inference chain is more efficient.
[0016] In one possible implementation, determining the answer to the question based on the supplemented question using a large language model includes: obtaining a third inference result based on the supplemented question using a large language model; adding a correlation between the third inference result and the first inference result; and determining the answer to the question based on the third inference result.
[0017] By autonomously maintaining the probability values of the relationships between inference results, memory retrieval and large language models can behave similarly, while the inference chain is more efficient.
[0018] Secondly, this application provides a data processing device based on a cloud management platform, wherein the cloud management platform is used to manage the infrastructure providing cloud services, the infrastructure including a storage system and a cluster of computing nodes, and the device includes:
[0019] The retrieval module is used to retrieve questions;
[0020] The reasoning result determination module is used to determine the answer to the question based on the first reasoning result when a first reasoning result matching the question exists in a pre-stored first reasoning result set; each reasoning result is obtained through reasoning in the history of the large language model;
[0021] The model reasoning module is used to determine the answer to the question based on the question and through a large language model when it is determined that there is no reasoning result matching the question in the first reasoning result set.
[0022] In one possible implementation, the first inference result includes a query and the answer to the query; the first inference result matches the question when a preset condition is met between the first inference result and the question, the preset condition including at least one of the following:
[0023] The similarity between the query and the question is greater than a threshold, and the uncertainty of the first reasoning result itself is less than a threshold.
[0024] In one possible implementation, the first inference result includes a query and the answer to the query; the inference result determination module is configured to:
[0025] Based on the answer to the query, the supplemented question is obtained;
[0026] Based on the supplemented question, when it is determined that there is a second reasoning result in the pre-stored second reasoning result set that matches the supplemented question, the answer to the question is determined based on the second reasoning result, and each reasoning result in the second reasoning result set has an association with the first reasoning result;
[0027] If it is determined that there is no reasoning result in the second set of reasoning results that matches the supplemented question, the answer to the question is determined by a large language model based on the supplemented question.
[0028] In one possible implementation, there is a correlation between each inference result in the second inference result set and the first inference result, as well as a probability value corresponding to the correlation.
[0029] The reasoning result determination module is also used for:
[0030] Based on the probability value, the second inference result is determined by sampling from the second inference result set;
[0031] When a preset condition is met between the second reasoning result and the supplemented question, the second reasoning result matches the supplemented question.
[0032] In one possible implementation, the reasoning result determination module is further configured to:
[0033] Increase the probability value corresponding to the correlation between the second inference result and the first inference result.
[0034] In one possible implementation, the reasoning result determination module is configured to:
[0035] Based on the supplemented question, a third reasoning result is obtained through a large language model;
[0036] Increase the correlation between the third reasoning result and the first reasoning result; and,
[0037] Based on the third reasoning result, the answer to the question is determined.
[0038] Thirdly, this application provides a computing device cluster. The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory. The at least one processor and the at least one memory communicate with each other. The at least one processor is used to execute instructions stored in the at least one memory to cause the computing device or the computing device cluster to perform a method as described in the first aspect or any implementation thereof.
[0039] Fourthly, this application provides a computer-readable storage medium storing instructions that instruct a computing device or a cluster of computing devices to execute the method of the database system described in the first aspect or any implementation thereof.
[0040] Fifthly, this application provides a computer program product containing instructions that, when run on a computing device or a cluster of computing devices, causes the computing device or cluster of computing devices to execute the database system execution method described in the first aspect or any implementation thereof.
[0041] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description
[0042] Figure 1 illustrates an architecture provided in this application;
[0043] Figures 2 to 9 are schematic flowcharts of a data processing method based on a cloud management platform provided in this application;
[0044] Figures 10 to 13 are schematic diagrams of the hardware architecture provided in this application. Detailed Implementation
[0045] First, some expressions that may appear in this application will be explained.
[0046] "First" and "second" are used to distinguish different objects or to differentiate different treatments of the same object, rather than to describe a specific order of objects.
[0047] "At least one" means one or more, while "more" means two or more.
[0048] "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone. A and B can be singular or plural.
[0049] The character " / " generally indicates that the objects before and after it are in an "or" relationship. For example, A / B can mean A or B.
[0050] Furthermore, the terms "comprising," "including," and "having" used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.
[0051] It should be noted that in this application, the terms "exemplary" or "for example" are used to indicate that something is being described or illustrated. Any implementation or design scheme described as "exemplary" or "for example" (such as the embodiments in this application) should not be construed as being more preferred or advantageous than other implementations or design schemes. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.
[0052] In the specification and drawings of this application, the terms "of", "relevant", and "corresponding" may sometimes be used interchangeably. It should be noted that when the distinction is not emphasized, they have the same meaning.
[0053] Below is a brief description of some of the terms used in this application.
[0054] (1) Neural Network
[0055] A neural network can be composed of neural units, which can be operational units that take xs and an intercept of 1 as inputs, and whose output can be:
[0056] Where s = 1, 2, ..., n, where n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input of the next convolutional layer, and the activation function can be the sigmoid function. A neural network is a network formed by connecting multiple of the above-mentioned individual neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, which can be a region composed of several neural units.
[0057] (2) Deep Neural Networks
[0058] Deep Neural Networks (DNNs), also known as multilayer neural networks, can be understood as neural networks with many hidden layers, though there's no specific metric for "many." DNNs can be categorized into three layers based on their position: input layers, hidden layers, and output layers. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. All layers are fully connected, meaning that any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer. Although DNNs appear complex, the operation of each layer is actually quite simple, resembling a linear relationship as follows: in, It is the input vector. It is the output vector. α is the offset vector, W is the weight matrix (also called coefficients), and α() is the activation function. Each layer is simply an adjustment of the input vector. The output vector is obtained through such a simple operation. Because DNNs have many layers, the coefficients W and the offset vector... The number of these parameters is therefore quite large. The definitions of these parameters in a DNN are as follows: Taking the coefficient W as an example: Assuming a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as... The superscript 3 represents the layer number where coefficient W resides, while the subscript corresponds to the output third layer index 2 and the input second layer index 4. In summary, the coefficients from the k-th neuron in layer L-1 to the j-th neuron in layer L are defined as follows: It's important to note that the input layer does not have a W parameter. In deep neural networks, more hidden layers allow the network to better represent complex real-world situations. Theoretically, the more parameters a model has, the higher its complexity and "capacity," meaning it can perform more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrix of all layers in the trained deep neural network (a weight matrix formed by the vectors W from many layers).
[0059] (3) Large language models (LLMs) are deep learning-based natural language processing models designed to understand and generate natural language text. A large language model is an artificial intelligence model trained on massive amounts of text data, utilizing deep learning techniques to capture the complexity and diversity of language. These models process and understand enormous amounts of language data, typically ranging from billions to hundreds of billions. Through large-scale unsupervised training, large language models can learn patterns and structures in natural language, thus simulating, to some extent, the human language cognition and generation process.
[0060] As large language models (LLMs) are increasingly used in various tasks, their powerful reasoning capabilities have become a core competitive advantage in complex scenarios such as code generation, mathematical proof, and problem-solving. However, current large language models rely on stepwise reasoning, which typically requires significant computational resources and inference time, causing the inference cost to increase exponentially with model size and task complexity. In practical applications, this high inference cost and latency severely limit the widespread adoption of large language models, especially in scenarios requiring real-time responses, such as dialogue systems, intelligent assistants, and autonomous driving.
[0061] Current inference techniques typically improve inference accuracy by increasing the number of inference steps and time. However, while this process can improve inference quality, it also significantly increases computational costs and response time, failing to meet the real-time and cost-effectiveness requirements of many practical applications.
[0062] Therefore, while improving the reasoning ability of large language models, how to reduce the reasoning cost has become an urgent problem to be solved.
[0063] To address the aforementioned issues, this application provides a data processing method based on a cloud management platform. Referring to Figure 1, the cloud management platform is used to manage the infrastructure providing cloud services, and the infrastructure includes a storage system and a cluster of computing nodes.
[0064] A cloud management platform can centrally manage the infrastructure of the entire cloud service system (e.g., within the infrastructure, based on a user's question, determining whether to directly read inference results from the storage system or utilize a cluster of compute nodes to provide a large language model for re-inference). The cloud management platform can also be accessible to users outside the cloud service system and respond to their requests. For example, the cloud management platform can provide various interfaces, such as login and model inference interfaces, for user clients (e.g., the user's terminal device or the browser on that device). The cloud management platform can authenticate a user's client through the login interface, allowing the client to log in after successful authentication. Furthermore, the cloud management platform can allow the user's client to send model inference requests to the cloud management platform through the model inference interface, indicating the questions the user needs to answer.
[0065] The infrastructure includes a cluster of computing nodes and a storage system that provide cloud services to the user. The computing node cluster may contain a large number of computing nodes, each of which may occupy a certain amount of physical resources. For example, any one of these computing nodes may occupy any one or any combination of at least one computing resource of the physical server (e.g., a central processing unit (CPU) and a graphics processing unit (GPU), etc.), at least one storage resource of the physical server (e.g., memory and disk), and at least one network resource of the physical server (e.g., a network interface card, etc.), without any limitation.
[0066] Referring to Figure 2, which is a flowchart illustrating a data processing method based on a cloud management platform according to an embodiment of this application, the method includes:
[0067] 201. The cloud management platform receives questions sent by users;
[0068] In the context of LLM, user input questions are typically statements seeking information, answering questions, performing tasks, or engaging in conversation. These questions can be very diverse, covering various fields and topics. Below are some examples of question types that users can choose to input:
[0069] 1. Information Inquiry:
[0070] How's the weather today?
[0071] Where is the nearest movie theater?
[0072] Can you tell me about the latest developments in artificial intelligence?
[0073] 2. Q&A:
[0074] Why is the sky blue?
[0075] Why do humans need sleep?
[0076] 3. Task execution:
[0077] "Please set an alarm for me at 7 a.m. tomorrow."
[0078] "Could you help me check train tickets from Beijing to Shanghai?"
[0079] 4. Dialogue and communication:
[0080] "How was your day today?"
[0081] What kind of movies do you like to watch?
[0082] What do you think of this idea?
[0083] 5. Personalized suggestions:
[0084] "Based on my preferences, can you recommend some restaurants?"
[0085] What gift should I buy for my friend?
[0086] Based on my browsing history, do you have any new book recommendations?
[0087] The questions users input can be simple, complex, concrete, or abstract. The task of the cloud management platform is to understand these questions and generate appropriate responses to meet the user's needs.
[0088] 202. When the cloud management platform determines that there is a first inference result matching the question in the first inference result set pre-stored in the storage system, it determines the answer to the question based on the first inference result; each inference result in the first inference result set is obtained by the computing node cluster through historical inference of the large language model;
[0089] In existing technologies, after a user inputs a question, the large language model can directly perform inference. To ensure the accuracy of the inference results, multiple rounds of inference are often required. Each round yields a query and a corresponding answer. For example, if the user's question is "What's the weather like today?", the large language model can infer the query: "What is the current location?", with the answer being "Shenzhen." In the next round of inference, if the input question is "What's the weather like in Shenzhen?", the answer is "Sunny," then "Sunny" becomes the answer to the user's question. This approach requires the large language model to perform many rounds of inference, resulting in significant computational overhead. The approach of this application is that for some questions, the large language model may have already performed inference for similar questions in the past. The inference results can be directly reused without re-inference, thereby reducing the computational overhead of the model's inference. For example, in this embodiment of the application, the storage system can store a first set of reasoning results obtained from historical reasoning by the large language model. For a question input by the user, it can first determine whether there is a reasoning result in the first set of reasoning results that matches the question input by the user. When it is determined that there is a first reasoning result in the pre-stored first set of reasoning results that matches the question, the first reasoning result can be used directly, that is, the answer to the question can be determined based on the first reasoning result, without using the large language model to reason about the question. This can reduce the computational cost of model reasoning while increasing the reasoning speed. Only when it is determined that there is no reasoning result in the first set of reasoning results that matches the question, the answer to the question is determined based on the question and through the large language model.
[0090] In this embodiment of the application, the first inference result set may include multiple inference results, and each inference result may include a query and the corresponding answer.
[0091] In one possible implementation, different inference results can be represented as a directed graph. This graph can include multiple nodes, each containing a query and its corresponding answer. Optionally, each node can also include the model's derivation of the answer and any uncertainties. Edges represent probability values pointing to the next node. The inference route (discriminator) is responsible for determining, based on the retrieved memory results and the current question, whether to query the large language model or terminate the inference process.
[0092] In one possible implementation, the first inference result includes a query and the answer to the query; the first inference result matches the question when a preset condition is met between the first inference result and the question, the preset condition including at least one of the following: the similarity between the query and the question is greater than a threshold, and the uncertainty of the first inference result itself is lower than a threshold.
[0093] For example, the similarity between the query and the question can be quantified by calculating the distance between the feature vectors of the question and the feature vectors of the query. The uncertainty of the first inference result itself can be determined by the probability distribution of multiple inference results that are connected to the question. When the distribution is very uniform, the uncertainty of the first inference result itself can be considered to be relatively large.
[0094] In this embodiment of the application, if the answer in the first reasoning result can already answer the user's question, then the reply can be made directly based on the answer in the first reasoning result. If the answer in the first reasoning result is insufficient to answer the user's question, then the question can be supplemented directly based on the answer in the first reasoning result to obtain the supplemented question, and then the next round of processing can be carried out.
[0095] In one possible implementation, the first inference result includes a query and the answer to the query. Based on the answer to the query, a supplemented question can be obtained. The answer to the query is the information needed to obtain an answer to the question. For example, if a user's question is "What is the weather like today?", the first inference result may include the query: "What is the current location?", with the answer being: "Shenzhen". In the next round of inference, the supplemented question is "What is the weather like in Shenzhen?". Based on the supplemented question, if it is determined that a second inference result matching the supplemented question exists in a pre-stored set of second inference results, the answer to the question is determined based on the second inference result. Each inference result in the second inference result set has an association with the first inference result.
[0096] 203. When the cloud management platform determines that there is no reasoning result matching the question in the first reasoning result set, the computing node cluster determines the answer to the question based on the question and through a large language model.
[0097] In one possible implementation, a third inference result can be obtained through a large language model based on the supplemented question; the correlation between the third inference result and the first inference result can be increased; and the answer to the question can be determined based on the third inference result.
[0098] In this embodiment, each inference result in the second inference result set is derived from historical reasoning by the large language model based on the query (or similar query) in the first inference result set. That is, each inference result in the second inference result set may serve as an answer to the supplemented question. In this embodiment, it can be assumed that there is an association between each inference result in the second inference result set and the first inference result set. This association can be represented by directed edges in a directed graph. The role of this association is that after determining a query, inference results associated with that query (in the inference result set) can be directly used as candidates for inference results in subsequent rounds of that query. Then, it can be determined from the second inference result set whether there is a matching second inference result for the supplemented question. If it is determined that there is a second inference result in the pre-stored second inference result set that matches the supplemented question, the answer to the question is determined based on the second inference result. If it is determined that there is no inference result in the second inference result set that matches the supplemented question, the answer to the question is determined based on the supplemented question using the large language model.
[0099] In one possible implementation, there is a correlation between each inference result in the second inference result set and the first inference result, as well as a probability value corresponding to the correlation; the second inference result can be determined by sampling from the second inference result set based on the probability value; when the second inference result meets a preset condition with the supplemented question, the second inference result matches the supplemented question.
[0100] For example, the second inference result can be determined by sampling from the second inference result set through probability sampling (refer to Figure 3, probability values can be set between adjacent nodes in the directed graph, and node selection can be based on probability values). When the second inference result meets a preset condition with the supplemented question, the second inference result is determined to match the supplemented question. For example, the second inference result can be determined to match the supplemented question when the similarity between the query included in the second inference result and the supplemented question is greater than a threshold and / or the uncertainty of the second inference result itself is lower than a threshold.
[0101] For example, referring to Figure 4, similar nodes can be extracted from the plastic memory through vector retrieval. The reasoning route determines whether to re-reason based on the similarity between the node and the question and the uncertain value of the node itself. If re-reasoning is required, the large language model is used to re-reason and obtain the answer and the next query question. Otherwise, the answer is obtained based on the node extraction, and the edge probability is sampled to obtain the further query question. The process returns to the previous steps until the answer is obtained.
[0102] In this embodiment of the application, the correlation between the inference results can be updated and supplemented. For example, in one possible implementation, after determining that the second inference result matches the supplemented question, the probability value corresponding to the correlation between the second inference result and the first inference result can be increased.
[0103] For example, referring to Figure 5, the storage system already maintains the connection relationship between node A and node B. If node A and node B are determined to be a match in a certain iteration, the probability value of the connection relationship between node A and node B can be increased.
[0104] For example, referring to Figure 5, if node C is processed through the language model during a certain iteration to obtain node E, then the connection between node C and node E can be increased.
[0105] In this embodiment, the relationships between inference relations can be maintained through online memory and offline memory. In online memory maintenance, when the large language model generates new inference results, corresponding nodes and edges are added to the plastic memory. If an outgoing edge of a node already exists, its weight is increased and renormalized; if the outgoing edge does not exist, it is initialized to a fixed value, and all outgoing edges of that node are normalized. In offline memory maintenance, counterfactual inference is performed using the large language model, as shown in Figure 6, to determine whether a node can be skipped. If so, a skip connection is added. Optionally, the utility values of each node and edge can be evaluated using, but not limited to, Monte Carlo Tree Search (MCTS), and the weights on the edges are normalized and updated based on these utility values.
[0106] For example, referring to Figure 7, after a user poses a problem, the inference route first selects to use plastic memory or a large language model for inference. If the large language model is used for inference, online memory maintenance is triggered to update the plastic memory; otherwise, the plastic memory is used directly to replace this step of inference. In the offline stage, the offline memory maintenance module calls the large language model to continuously maintain the plastic memory.
[0107] For example, referring to Figure 8, which is a schematic diagram of an online memory update process, it includes:
[0108] 1. User raises a question
[0109] 2. Reasoning route: Query plastic memory based on the problem.
[0110] 3. Reasoning Route: Based on the retrieved memories, determine whether re-reasoning is necessary.
[0111] 4. If re-reasoning is required, then send the current problem to the large language model.
[0112] 5. Large language models return inference results and follow-up questions.
[0113] 6. Inference route updates plastic memory.
[0114] 7. Inferring the route updates the current problem to a subsequent problem, returning to step 2.
[0115] 8. If no further reasoning is required
[0116] 9. The reasoning route adopts the results of memory retrieval and returns to step 2 based on subsequent questions.
[0117] 10. After determining the completion of the inference routing task, submit the code and generate the results to the user.
[0118] For example, referring to Figure 9, which illustrates a process for offline memory updates. Compared to the online memory update in Figure 8, offline memory maintenance can be triggered after the result is submitted.
[0119] The above is a detailed description of the data processing method based on a cloud management platform provided in the embodiments of this application. The following will introduce the cloud management platform provided in the embodiments of this application. Figure 10 is a structural schematic diagram of the cloud management platform provided in the embodiments of this application. The cloud management platform is used to manage the infrastructure providing cloud services. The infrastructure includes a computing node cluster and a storage system. The cloud management platform includes:
[0120] The acquisition module 1001 is used to acquire the problem; for a detailed description of the acquisition module 1001, please refer to the description of step 201 in the above embodiment.
[0121] The reasoning result determination module 1002 is used to determine the answer to the question based on the first reasoning result when it is determined that there is a first reasoning result matching the question in the pre-stored first reasoning result set; each reasoning result is obtained through reasoning in the history of the large language model; wherein, a detailed description of the reasoning result determination module 1002 can be referred to the description of step 202 in the above embodiment.
[0122] The model reasoning module 1003 is used to determine the answer to the question based on the question and through a large language model when it is determined that there is no reasoning result matching the question in the first reasoning result set. For a detailed description of the model reasoning module 1003, please refer to the description of step 203 in the above embodiments.
[0123] In one possible implementation, the first inference result includes a query and the answer to the query; the first inference result matches the question when a preset condition is met between the first inference result and the question, the preset condition including at least one of the following:
[0124] The similarity between the query and the question is greater than a threshold, and the uncertainty of the first reasoning result itself is less than a threshold.
[0125] In one possible implementation, the first inference result includes a query and the answer to the query; the inference result determination module is configured to:
[0126] Based on the answer to the query, the supplemented question is obtained;
[0127] Based on the supplemented question, when it is determined that there is a second reasoning result in the pre-stored second reasoning result set that matches the supplemented question, the answer to the question is determined based on the second reasoning result, and each reasoning result in the second reasoning result set has an association with the first reasoning result;
[0128] If it is determined that there is no reasoning result in the second set of reasoning results that matches the supplemented question, the answer to the question is determined by a large language model based on the supplemented question.
[0129] In one possible implementation, there is a correlation between each inference result in the second inference result set and the first inference result, as well as a probability value corresponding to the correlation.
[0130] The reasoning result determination module is also used for:
[0131] Based on the probability value, the second inference result is determined by sampling from the second inference result set;
[0132] When a preset condition is met between the second reasoning result and the supplemented question, the second reasoning result matches the supplemented question.
[0133] In one possible implementation, the reasoning result determination module is further configured to:
[0134] Increase the probability value corresponding to the correlation between the second inference result and the first inference result.
[0135] In one possible implementation, the reasoning result determination module is configured to:
[0136] Based on the supplemented question, a third reasoning result is obtained through a large language model;
[0137] Increase the correlation between the third reasoning result and the first reasoning result; and,
[0138] Based on the third reasoning result, the answer to the question is determined.
[0139] It should be noted that the information interaction and implementation process between the modules / units of the above-mentioned device are based on the same concept as the method embodiments of this application, and the resulting technical effects are the same as those of the method embodiments of this application. For details, please refer to the description in the method embodiments shown above in the embodiments of this application, and will not be repeated here.
[0140] Please refer to Figure 11, which is a schematic diagram of a computing device provided in an embodiment of this application. As shown in Figure 11, the computing device 1100 (which can be used to present the aforementioned cloud management platform) includes: a processor 1101, a memory 1102, a communication interface 1103, and a bus 1104. The processor 1101, the memory 1102, and the communication interface 1103 are coupled through the bus (not labeled in the figure). The memory 1102 stores instructions. When the execution instructions in the memory 1102 are executed, the computing device 1100 executes the method performed by the cloud management platform in the above method embodiment.
[0141] The computing device 1100 may be one or more integrated circuits configured to implement the methods described above, such as: one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these forms of integrated circuits. Furthermore, when the units in the device can be implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Alternatively, these units may be integrated together and implemented as a system-on-a-chip (SOC).
[0142] The processor 1101 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.
[0143] The memory 1102 can be volatile memory or non-volatile memory, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).
[0144] The memory 1102 stores executable program code, and the processor 1101 executes the executable program code to implement the functions of the aforementioned acquisition module, execution inference result determination module, model inference module, etc., thereby realizing the aforementioned data processing method based on the cloud management platform. That is, the memory 1102 stores instructions for executing the aforementioned data processing method based on the cloud management platform.
[0145] The communication interface 1103 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1100 and other devices or communication networks.
[0146] In addition to the data bus, the 1104 bus can also include a power bus, a control bus, and a status signal bus. The bus can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The bus can be divided into address bus, data bus, and control bus.
[0147] Please refer to Figure 12, which is a schematic diagram of a computing device cluster provided in an embodiment of this application. As shown in Figure 12, the computing device cluster 1200 includes at least one computing device 1100.
[0148] As shown in Figure 12, the computing device cluster 1200 includes at least one computing device 1100. The memory 1102 of one or more computing devices 1100 in the computing device cluster 1200 may store the same instructions for executing the above-described data processing method based on the cloud management platform.
[0149] In some possible implementations, the memory 1102 of one or more computing devices 1100 in the computing device cluster 1200 may also store partial instructions for executing the aforementioned data processing method based on the cloud management platform. In other words, a combination of one or more computing devices 1100 can jointly execute the aforementioned data processing method based on the cloud management platform.
[0150] It should be noted that the memory 1102 in different computing devices 1100 within the computing device cluster 1200 can store different instructions, which are used to execute certain functions of the aforementioned cloud management platform. That is, the instructions stored in the memory 1102 of different computing devices 1100 can implement the functions of one or more modules, such as the acquisition module, the inference result determination module, and the model inference module.
[0151] In some possible implementations, one or more computing devices 1100 in the computing device cluster 1200 can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc.
[0152] Please refer to Figure 13, which is a schematic diagram of computer devices in a computer cluster provided in an embodiment of this application being connected via a network. As shown in Figure 13, two computing devices 1100A and 1100B are connected via a network. Specifically, they are connected to the network through the communication interfaces in each computing device.
[0153] In one possible implementation, the memory in computing device 1100A stores instructions for performing the functions of modules such as the acquisition module. Meanwhile, the memory in computing device 1100B stores instructions for performing the functions of modules such as the inference result determination module and the model inference module.
[0154] It should be understood that the functions of computing device 1100A shown in Figure 13 can also be performed by multiple computing devices. Similarly, the functions of computing device 1100B can also be performed by multiple computing devices.
[0155] This application also relates to a computer storage medium storing a program for signal processing, which, when run on a computer, causes the computer to perform the steps executed by the cloud management platform in the embodiment shown in FIG2.
[0156] This application also relates to a computer program product that stores instructions that, when executed by a computer, cause the computer to perform the steps performed by the cloud management platform in the embodiment shown in FIG2.
[0157] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0158] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.
[0159] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0160] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0161] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
Claims
1. A data processing method based on a cloud management platform, characterized in that, The cloud management platform is used to manage the infrastructure that provides cloud services, the infrastructure including a storage system and a cluster of computing nodes, and the method includes: The cloud management platform receives questions sent by users; When the cloud management platform determines that there is a first inference result matching the question in the first inference result set pre-stored in the storage system, it determines the answer to the question based on the first inference result; each inference result in the first inference result set is obtained by the computing node cluster through historical inference of a large language model; When the cloud management platform determines that there is no reasoning result matching the question in the first set of reasoning results, the computing node cluster determines the answer to the question based on the question and through a large language model.
2. The method according to claim 1, characterized in that, The first reasoning result includes a query and the answer to the query; when the first reasoning result and the question satisfy a preset condition, the first reasoning result matches the question, and the preset condition includes at least one of the following: The similarity between the query and the question is greater than a threshold, and the uncertainty of the first reasoning result itself is less than a threshold.
3. The method according to claim 1 or 2, characterized in that, The first reasoning result includes a query and the answer to the query; determining the answer to the question based on the target reasoning result includes: Based on the answer to the query, the supplemented question is obtained; Based on the supplemented question, when it is determined that there is a second reasoning result in the pre-stored second reasoning result set that matches the supplemented question, the answer to the question is determined based on the second reasoning result, and each reasoning result in the second reasoning result set has an association with the first reasoning result; If it is determined that there is no reasoning result in the second set of reasoning results that matches the supplemented question, the answer to the question is determined by a large language model based on the supplemented question.
4. The method according to claim 3, characterized in that, There is a correlation between each inference result in the second inference result set and the first inference result, as well as a probability value corresponding to the correlation. The method further includes: Based on the probability value, the second inference result is determined by sampling from the second inference result set; When a preset condition is met between the second reasoning result and the supplemented question, the second reasoning result matches the supplemented question.
5. The method according to claim 4, characterized in that, The method further includes: Increase the probability value corresponding to the correlation between the second inference result and the first inference result.
6. The method according to any one of claims 3 to 5, characterized in that, The step of determining the answer to the question based on the supplemented question using a large language model includes: Based on the supplemented question, a third reasoning result is obtained through a large language model; Increase the correlation between the third reasoning result and the first reasoning result; and, Based on the third reasoning result, the answer to the question is determined.
7. A data processing device based on a cloud management platform, characterized in that, The cloud management platform is used to manage the infrastructure that provides cloud services, the infrastructure including a storage system and a cluster of computing nodes, and the device includes: The retrieval module is used to retrieve questions; The reasoning result determination module is used to determine the answer to the question based on the first reasoning result when a first reasoning result matching the question exists in a pre-stored first reasoning result set; each reasoning result is obtained through reasoning in the history of the large language model; The model reasoning module is used to determine the answer to the question based on the question and through a large language model when it is determined that there is no reasoning result matching the question in the first reasoning result set.
8. The apparatus according to claim 7, characterized in that, The first reasoning result includes a query and the answer to the query; when the first reasoning result and the question satisfy a preset condition, the first reasoning result matches the question, and the preset condition includes at least one of the following: The similarity between the query and the question is greater than a threshold, and the uncertainty of the first reasoning result itself is less than a threshold.
9. The apparatus according to claim 7 or 8, characterized in that, The first reasoning result includes a query and the answer to the query; the reasoning result determination module is used for: Based on the answer to the query, the supplemented question is obtained; Based on the supplemented question, when it is determined that there is a second reasoning result in the pre-stored second reasoning result set that matches the supplemented question, the answer to the question is determined based on the second reasoning result, and each reasoning result in the second reasoning result set has an association with the first reasoning result; If it is determined that there is no reasoning result in the second set of reasoning results that matches the supplemented question, the answer to the question is determined by a large language model based on the supplemented question.
10. The apparatus according to claim 9, characterized in that, There is a correlation between each inference result in the second inference result set and the first inference result, as well as a probability value corresponding to the correlation. The reasoning result determination module is also used for: Based on the probability value, the second inference result is determined by sampling from the second inference result set; When a preset condition is met between the second reasoning result and the supplemented question, the second reasoning result matches the supplemented question.
11. The apparatus according to claim 10, characterized in that, The reasoning result determination module is also used for: Increase the probability value corresponding to the correlation between the second inference result and the first inference result.
12. The apparatus according to any one of claims 9 to 11, characterized in that, The reasoning result determination module is used for: Based on the supplemented question, a third reasoning result is obtained through a large language model; Increase the correlation between the third reasoning result and the first reasoning result; as well as, Based on the third reasoning result, the answer to the question is determined.
13. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 6.
14. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1 to 6.
15. A computer-readable storage medium, characterized in that, Includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1 to 6.