Large model reasoning method and system, computer equipment and storage medium

Abstract

The invention discloses a large model reasoning method and system, computer equipment and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: segmenting a pre-trained target large model, and obtaining an input layer, a middle layer and an output layer corresponding to the target large model; deploying the input layer and the output layer at a client, and deploying the intermediate layer at a server; receiving a Prompt input by a target user at the client, and encoding the Prompt through the input layer to obtain a corresponding numerical vector; uploading the numerical value vector to the server, and reasoning the numerical value vector through the intermediate layer to obtain a corresponding result vector; and returning the result vector to the client, and decoding the result vector through the output layer to obtain a corresponding result text. By means of the method, privacy protection in the large model reasoning stage can be achieved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large model reasoning method, system, computer device and non-volatile computer-readable storage medium. Background Art

[0002] At present, with the breakthrough progress of deep learning technology, large model reasoning has become a hot topic in the field of artificial intelligence. Large models, namely large language models (LLM), such as GPT, BERT, etc., have been widely used in natural language processing, image recognition and complex decision-making tasks due to their excellent performance. These models usually require a large amount of data for pre-training to learn complex data representations. Once the training is completed, the model can be used to reason about new data, thereby providing accurate prediction or generation capabilities in various applications.

[0003] However, in practical applications, large model reasoning faces multiple challenges. First, the reasoning process is computationally intensive and requires high-performance hardware support, which requires that the relevant hardware and software infrastructure must be able to effectively support large-scale parallel computing. Second, large models usually need to be optimized to adapt to specific tasks, which may include fine-tuning model parameters or performing knowledge distillation to simplify the model structure. In addition, and most importantly, the large model reasoning process must take into account privacy and security issues, especially when processing sensitive data in the financial field, it is particularly important to protect the privacy of data.

[0004] Based on this, how to provide a large model reasoning method, system, computer device and non-volatile computer-readable storage medium to achieve privacy protection in the large model reasoning stage is a problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a large model reasoning method, system, computer device and non-volatile computer-readable storage medium that can be used in financial technology or other related fields, aiming to solve the problem of how to achieve privacy protection in the large model reasoning stage.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A large model reasoning method, comprising:

[0008] Segmenting the pre-trained target large model to obtain an input layer, an intermediate layer, and an output layer corresponding to the target large model;

[0009] The input layer and the output layer are both deployed on the client, and the middle layer is deployed on the server;

[0010] Receiving a Prompt input by a target user on the client, and encoding the Prompt through the input layer to obtain a corresponding numerical vector;

[0011] Uploading the numerical vector to the server, and performing reasoning on the numerical vector through the intermediate layer to obtain a corresponding result vector;

[0012] The result vector is returned to the client, and the result vector is decoded through the output layer to obtain a corresponding result text.

[0013] In a further technical solution, the large model inference method, wherein the pre-trained target large model is segmented to obtain an input layer, an intermediate layer, and an output layer corresponding to the target large model, includes:

[0014] Predetermine the segmentation strategy based on user needs;

[0015] The segmentation strategy is used to segment the pre-trained target large model to obtain the input layer, the middle layer, and the output layer corresponding to the target large model.

[0016] In a further technical solution, the large model reasoning method, wherein the input layer and the output layer are both deployed on the client, and the intermediate layer is deployed on the server, comprises:

[0017] Predetermining a deployment strategy based on the user needs;

[0018] The deployment strategy is adopted to deploy the input layer and the output layer on the client, and to deploy the intermediate layer on the server.

[0019] In a further technical solution, the large model inference method, wherein the receiving of the Prompt input by the target user on the client and encoding the Prompt through the input layer to obtain a corresponding numerical vector, includes:

[0020] Receive a prompt input by the target user on the client;

[0021] Encode the Prompt by the input layer deployed on the client to obtain a corresponding initial numerical vector;

[0022] The initial numerical vector is searched and enhanced by using a preset private knowledge base to obtain a corresponding numerical vector.

[0023] In a further technical solution, the large model reasoning method, wherein the numerical vector is uploaded to the server, and the numerical vector is reasoned through the intermediate layer to obtain a corresponding result vector, includes:

[0024] Uploading the numerical vector obtained after the retrieval enhancement processing to the server;

[0025] Inferring the numerical vector by the intermediate layer deployed on the server to obtain a corresponding initial result vector;

[0026] The initial result vector is normalized to obtain a corresponding result vector.

[0027] In a further technical solution, the large model reasoning method, wherein the returning the result vector to the client and decoding the result vector through the output layer to obtain the corresponding result text, includes:

[0028] Returning the result vector to the client;

[0029] Decoding the result vector through the output layer deployed on the client to obtain a corresponding initial result text;

[0030] Correction processing is performed on the initial result text to obtain a corresponding result text.

[0031] In a further technical solution, the large model inference method, wherein the pre-trained target large model is segmented to obtain the input layer, the intermediate layer, and the output layer corresponding to the target large model, wherein:

[0032] The target large model is a federated large model, and the input layer, the intermediate layer, and the output layer are respectively an Embedding layer, a Transformer layer, and a Decoder layer.

[0033] A large model reasoning system, comprising:

[0034] A segmentation module is used to segment the pre-trained target large model to obtain an input layer, an intermediate layer, and an output layer corresponding to the target large model;

[0035] A deployment module, used to deploy the input layer and the output layer on the client, and deploy the middle layer on the server;

[0036] An encoding module, used for receiving a Prompt input by a target user on the client, and encoding the Prompt through the input layer to obtain a corresponding numerical vector;

[0037] An inference module, used for uploading the numerical vector to the server, and performing inference on the numerical vector through the intermediate layer to obtain a corresponding result vector;

[0038] The decoding module is used to return the result vector to the client and decode the result vector through the output layer to obtain a corresponding result text.

[0039] A computer device, wherein the computer device comprises at least one processor; and

[0040] a memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores a computer program that can be executed by the at least one processor. When the computer program is executed by the at least one processor, the large model reasoning method as described in any one of the above items can be implemented.

[0042] A non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, it can implement the large model reasoning method as described in any one of the above items.

[0043] Compared with the prior art, the present invention provides a large model reasoning method, system, computer device and non-volatile computer-readable storage medium, wherein the method includes: segmenting the pre-trained target large model to obtain the input layer, intermediate layer, and output layer corresponding to the target large model; deploying the input layer and the output layer on the client, and deploying the intermediate layer on the server; receiving the prompt input by the target user on the client, and encoding the prompt through the input layer to obtain the corresponding numerical vector; uploading the numerical vector to the server, and reasoning the numerical vector through the intermediate layer to obtain the corresponding result vector; returning the result vector to the client, and decoding the result vector through the output layer to obtain the corresponding result text. In this way, the method of the present invention divides the target large model into an input layer, an intermediate layer, and an output layer, and distributes them on the client and the server in the actual application scenario, while ensuring that the user's plaintext data privacy does not leave the local area, and can better request the service of the server; in this way, the method of the present invention not only realizes the protection of the user's data privacy, that is, realizes the privacy protection for the large model reasoning stage, but also makes full use of the powerful computing power of the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A flowchart of a large model reasoning method provided in an embodiment of the present invention.

[0046] Figure 2 A schematic diagram of the functional modules of a large model reasoning system provided in an embodiment of the present invention.

[0047] Figure 3 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] In the description of the present invention, the terms "include", "including", "have", "contain", etc. are all open terms, which mean including but not limited to. The descriptions with reference to the terms "one embodiment", "a specific embodiment", "some embodiments", "for example", etc. mean that the specific features, structures or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. The order of steps involved in each embodiment is used to schematically illustrate the implementation of the present application, and the order of steps therein is not limited and can be appropriately adjusted as needed.

[0050] Various non-limiting embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0051] See also Figure 1 The embodiment of the present invention provides a large model reasoning method, wherein the method comprises the steps of:

[0052] S100, segmenting the pre-trained target large model to obtain an input layer, an intermediate layer, and an output layer corresponding to the target large model;

[0053] S200, deploying the input layer and the output layer on the client, and deploying the middle layer on the server;

[0054] S300, receiving a Prompt input by a target user at the client, and encoding the Prompt through the input layer to obtain a corresponding numerical vector;

[0055] S400, uploading the numerical vector to the server, and performing reasoning on the numerical vector through the intermediate layer to obtain a corresponding result vector;

[0056] S500: Return the result vector to the client, and decode the result vector through the output layer to obtain a corresponding result text.

[0057] Specifically, in the present invention, Prompt is a prompt text that guides the large model to understand and generate specific content.

[0058] Furthermore, the large model inference method, wherein the step S100, segmenting the pre-trained target large model to obtain the input layer, the intermediate layer, and the output layer corresponding to the target large model, includes:

[0059] Predetermine the segmentation strategy based on user needs;

[0060] The segmentation strategy is used to segment the pre-trained target large model to obtain the input layer, the middle layer, and the output layer corresponding to the target large model.

[0061] In the specific implementation, in this embodiment, a segmentation strategy is predetermined according to user needs, and the segmentation strategy is used to segment the pre-trained target large model to obtain the input layer, the intermediate layer, and the output layer corresponding to the target large model. The specific implementation process is as follows:

[0062] 1) Demand analysis and segmentation strategy determination

[0063] First, analyze and determine an appropriate model segmentation strategy based on user needs and application scenarios; this may include deciding which layers need to be segmented and how they work together in the system architecture after segmentation;

[0064] 2) Model segmentation implementation

[0065] The pre-trained target large model (which can be a federated large model) is segmented using a predetermined segmentation strategy; segmentation usually involves the following steps:

[0066] Identify the input layer (such as the Embedding layer), which is usually the first layer of the model to receive raw data;

[0067] Identify the intermediate layers, which may include multiple hidden layers or specific layers (such as Transformer layers) that are responsible for feature extraction and processing in the model;

[0068] Identify the output layer (such as the Decoder layer), which is responsible for generating the final inference results or outputs;

[0069] Finally, the input layer, intermediate layer, and output layer corresponding to the target large model are obtained;

[0070] 3) Deployment of model layers after segmentation

[0071] Each layer after segmentation will be deployed to the corresponding computing environment according to the established deployment strategy, which may be the client, server or a node in the distributed system;

[0072] 4) Model Collaboration

[0073] Ensure that the segmented model layers can work together to complete the entire reasoning process from input to output; this may require defining clear data transfer protocols and interfaces;

[0074] 5) Testing and verification

[0075] Test and validate the segmented model to ensure that its performance is comparable to the original unsegmented model and meets user needs and expectations;

[0076] 6) Optimization and adjustment

[0077] According to the test results and actual operation conditions, make necessary optimization and adjustments to the segmentation strategy and deployment;

[0078] Through the above process, the flexibility of model deployment can be improved, the requirements for computing resources can be reduced, and the privacy protection capabilities of the model can be enhanced; at the same time, this method also helps to achieve customized deployment of the model to meet the needs of specific application scenarios.

[0079] Furthermore, the large model reasoning method, wherein the step S200, deploying both the input layer and the output layer on the client, and deploying the intermediate layer on the server, comprises:

[0080] Predetermining a deployment strategy based on the user needs;

[0081] The deployment strategy is adopted to deploy the input layer and the output layer on the client, and to deploy the intermediate layer on the server.

[0082] In the specific implementation, in this embodiment, according to the user's needs, a deployment strategy is pre-determined, and the input layer and the output layer are deployed on the client side, and the intermediate layer is deployed on the server side using the deployment strategy. The specific implementation process is as follows:

[0083] 1) Requirements analysis and deployment strategy formulation

[0084] Based on the above user needs and application scenarios, analyze and determine a suitable deployment strategy, clarify the deployment details of the input layer and output layer on the client, and the deployment plan of the middle layer on the server;

[0085] 2) Model segmentation and optimization

[0086] After using the model segmentation technology to split the pre-trained target large model into independent components, the segmented model components are optimized to adapt to different operating environments and ensure that they can work efficiently together on the client and server.

[0087] 3) Security and communication protocol design

[0088] Design security protocols for data transmission between the client and the server, including data encryption and authentication mechanisms, to ensure data security and privacy protection during transmission; at the same time, define clear API interfaces and data formats so that the client can send requests to the server and transmit the result vector back to the client;

[0089] 4) Distributed deployment implementation

[0090] Deploy the input layer and output layer on the client to ensure that they can process the user's input data and present the final output of the model; deploy the middle layer on the server and use the server's computing resources to perform the main reasoning tasks of the model; configure the server environment to support high-performance operation and scalability of the model;

[0091] 5) System integration testing and iteration

[0092] Conduct integration testing on the entire system to verify the end-to-end performance after model segmentation and deployment; check whether the client and server work smoothly together, whether data transmission is secure, and whether the model output is accurate; make necessary adjustments and optimizations based on the test results to ensure that the system meets user needs and provides high-quality services;

[0093] Through the above process, it can be ensured that the deployment of the model not only meets user needs but also guarantees the security, performance and scalability of the system.

[0094] Furthermore, the large model inference method, wherein the step S300, receiving the Prompt input by the target user in the client, and encoding the Prompt through the input layer to obtain a corresponding numerical vector, includes:

[0095] Receive a prompt input by the target user on the client;

[0096] Encode the Prompt by the input layer deployed on the client to obtain a corresponding initial numerical vector;

[0097] The initial numerical vector is searched and enhanced by using a preset private knowledge base to obtain a corresponding numerical vector.

[0098] In specific implementation, in this embodiment, a prompt input by a target user on the client is received, the prompt is encoded by the input layer deployed on the client to obtain a corresponding initial numerical vector, and a search enhancement process is performed on the initial numerical vector through a preset private knowledge base to obtain a corresponding numerical vector, wherein the specific implementation process is as follows:

[0099] 1) User input reception and encoding

[0100] The target user enters a prompt on the client. The client interface design should ensure the convenience and accuracy of user input. After receiving the prompt, the input layer converts it into an initial numerical vector. This usually involves steps such as word segmentation and word embedding search to convert the text into a numerical form that the model can understand.

[0101] 2) Enhanced private knowledge base retrieval

[0102] After the initial numerical vector is generated, the client uses the private knowledge base for retrieval enhancement; this step may include querying the knowledge base to obtain additional information or contextual data related to the prompt, using retrieval algorithms (such as vector similarity search) to find the most relevant knowledge fragments, and converting them into numerical vectors or features;

[0103] 3) Vector fusion and enhancement processing

[0104] The retrieved target numerical vector is fused with the initial numerical vector to form an enhanced numerical vector; the fusion method may include simple concatenation, weighted averaging, or more complex feature interaction methods, the purpose of which is to integrate information from prompts and private knowledge bases to enrich the model input;

[0105] 4) Security and privacy protection

[0106] Ensure the security and privacy of user data throughout the entire processing process; this may include encrypting data in transit and performing local processing on the client side to reduce exposure of sensitive information;

[0107] 5) Result vector output

[0108] The enhanced numeric vectors are output, which now contain the information of the original prompt as well as the relevant information retrieved from the private knowledge base; the client prepares these vectors for secure transmission to the server for further reasoning processing;

[0109] Through the above process, the client not only converts the user's natural language prompt into numerical representations that the model can understand, but also enhances these numerical representations through a private knowledge base, providing richer and more relevant input data for the server's deep learning model.

[0110] Furthermore, the large model reasoning method, wherein the step S400, uploading the numerical vector to the server, and reasoning the numerical vector through the intermediate layer to obtain a corresponding result vector, includes:

[0111] Uploading the numerical vector obtained after the retrieval enhancement processing to the server;

[0112] Inferring the numerical vector by the intermediate layer deployed on the server to obtain a corresponding initial result vector;

[0113] The initial result vector is normalized to obtain a corresponding result vector.

[0114] In specific implementation, in this embodiment, the numerical vector obtained after the search enhancement processing is uploaded to the server, the numerical vector is inferred by the intermediate layer deployed on the server to obtain a corresponding initial result vector, and the initial result vector is normalized to obtain a corresponding result vector, wherein the specific implementation process is as follows:

[0115] 1) Upload numerical vector

[0116] The client uploads the retrieved enhanced numerical vector to the server through a secure network connection; this step requires ensuring the security of data transmission, which may involve the use of SSL encryption or other security protocols;

[0117] 2) Server-side reasoning processing

[0118] After receiving the numerical vector, the server inputs it into the middle layer (such as the Transformer layer) deployed on the server. The middle layer performs deep processing on the numerical vector, uses the parameters and algorithms of the model for reasoning, and generates an initial result vector.

[0119] 3) Result vector generation

[0120] After the server-side middle layer completes the calculation, the generated initial result vector contains the model's response to the input numerical vector; these vectors are usually in a raw state and require further processing to obtain the final output;

[0121] 4) Standardized processing

[0122] Normalize the initial result vector to ensure consistency and accuracy of the result; normalization may include scaling the value range of the vector, standardizing the vector to fit a specific distribution, or applying a specific mathematical transformation to optimize the quality of the result;

[0123] 5) Final result output

[0124] The normalized result vector is the final output, which the server can send back to the client for decoding to generate understandable result text; this step needs to ensure the accuracy and usability of the result;

[0125] Through the above process, the server not only completes the deep reasoning of complex numerical vectors, but also ensures the quality and availability of the results through normalization processing, providing the client with reliable and accurate output.

[0126] Furthermore, the large model reasoning method, wherein the step S500, returning the result vector to the client, and decoding the result vector through the output layer to obtain the corresponding result text, includes:

[0127] Returning the result vector to the client;

[0128] Decoding the result vector through the output layer deployed on the client to obtain a corresponding initial result text;

[0129] Correction processing is performed on the initial result text to obtain a corresponding result text.

[0130] In the specific implementation, in this embodiment, the result vector is returned to the client; the result vector is decoded by the output layer deployed on the client to obtain the corresponding initial result text; the initial result text is corrected to obtain the corresponding result text; wherein the specific implementation process is as follows:

[0131] 1) Result vector is returned

[0132] After the server completes the processing, it sends the normalized result vector back to the client securely over the network; this step needs to ensure the integrity and confidentiality of the data during transmission;

[0133] 2) Client decoding

[0134] After receiving the result vector, the client uses the locally deployed output layer to decode the vector; the output layer usually includes one or more neural network layers, which are responsible for converting the numerical vector into text form;

[0135] 3) Initial result text generation

[0136] The output layer generates the initial result text from the decoded results; this may be one or more candidate sentences, depending on how the model is decoded and the type of task;

[0137] 4) Text correction processing

[0138] Apply natural language processing techniques to the initial result text for correction and optimization; this may include grammar checking, spelling correction, contextual word sense disambiguation, and improving sentence flow and logic;

[0139] 5) Final result output

[0140] The corrected text is presented to the target user as the final result text; this step ensures the quality of the output text and improves the usability of the model and user satisfaction.

[0141] Through the above process, the client not only receives and decodes the result vector from the server, but also further improves the quality of the result through text correction processing, ensuring the accuracy and readability of the output.

[0142] Furthermore, in the large model inference method, the target large model is a federated large model, and the input layer, the intermediate layer, and the output layer are respectively an Embedding layer, a Transformer layer, and a Decoder layer;

[0143] The private knowledge base is deployed on the client and / or the server.

[0144] In the specific implementation, in this embodiment, the target large model is a federated large model, the input layer, the intermediate layer, and the output layer are respectively an Embedding layer, a Transformer layer, and a Decoder layer; the private knowledge base is deployed on the client and / or the server; wherein the specific implementation process is roughly described as follows:

[0145] 1) Deployment of private knowledge base

[0146] Determine the deployment location of the private knowledge base based on data privacy and security requirements; if the privacy requirements are extremely high, the knowledge base can be deployed on the client; if more powerful computing resources are needed to support the retrieval and processing of the knowledge base, it can also be deployed on the server, or in a distributed deployment, partly on the client and partly on the server;

[0147] 2) User input and Embedding layer encoding

[0148] The target user enters a prompt on the client; the client's Embedding layer converts the prompt into an initial numerical vector to prepare for the next step of retrieval and reasoning;

[0149] 3) Retrieval enhancement processing

[0150] Use a private knowledge base to perform retrieval enhancement on the initial numeric vector generated by the Embedding layer; this may involve querying the knowledge base to obtain relevant information and fusing this information into the initial numeric vector to enrich the context of the input data and obtain the corresponding numeric vector;

[0151] 4) Server-side Transformer layer reasoning

[0152] The client uploads the retrieved enhanced numerical vectors to the server; the server's Transformer layer receives these vectors and performs further reasoning to generate preliminary initial result vectors;

[0153] 5) Normalization of the result vector and decoding at the Decoder layer

[0154] The server normalizes the initial result vector obtained by reasoning to optimize the quality of the result and obtain the corresponding result vector;

[0155] The server sends the result vector back to the client for decoding. After receiving the result vector, the client uses the locally deployed Decoder layer to decode the result vector to generate an initial result text. The client corrects and optimizes the initial result text to generate an understandable result text.

[0156] 6) Final result output

[0157] The resulting text after correction is presented to the target user as the final output; this step ensures the accuracy, fluency and readability of the output text;

[0158] Through the above process, the federated big model can achieve efficient distributed processing while ensuring the privacy and security of user data; this architecture is particularly suitable for application scenarios with strict requirements on data privacy, such as finance, medical care and other fields.

[0159] It can be seen from the above method embodiments that the large model reasoning method provided by the present invention at least includes: segmenting the pre-trained target large model to obtain the input layer, intermediate layer, and output layer corresponding to the target large model; deploying the input layer and the output layer on the client, and deploying the intermediate layer on the server; receiving the Prompt input by the target user on the client, and encoding the Prompt through the input layer to obtain the corresponding numerical vector; uploading the numerical vector to the server, and reasoning the numerical vector through the intermediate layer to obtain the corresponding result vector; returning the result vector to the client, and decoding the result vector through the output layer to obtain the corresponding result text; etc. In this way, the privacy protection for the large model reasoning stage can be achieved through the method of the present invention.

[0160] Specifically, the large model inference method provided by the embodiment of the present invention innovatively splits the large model into an input layer, an intermediate layer, and an output layer, and distributes them on the client and server in actual application scenarios. While ensuring that the user's plaintext data privacy does not leave the local area, the large model service of the server can be better requested. This not only protects the user's data privacy, but also makes full use of the powerful computing power of the server, making it easier to implement and helping to serve all walks of life.

[0161] It should be understood that, although the present application provides method operation steps as described in the embodiments or flowcharts, more or less operation steps may be included based on conventional or non-creative labor, and these operation steps are not necessarily performed in sequence according to the embodiment or flowchart. The order of steps listed in the embodiment or flowchart is only one way of executing the order of many steps, and does not represent the only execution order. It should be noted that there is not necessarily a certain order between the above steps. A person of ordinary skill in the art can understand from the description of the embodiment of the present invention that in different embodiments, the above steps may have different execution orders, that is, they may be executed in parallel, or they may be executed in exchange, etc. Moreover, at least a part of the steps in the embodiment or flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but may be executed in turn, alternately or synchronously with other steps or at least a part of the sub-steps or stages of other steps.

[0162] Based on the above method embodiment, please refer to Figure 2 Another embodiment of the present invention further provides a large model reasoning system, wherein the system includes:

[0163] A segmentation module 11 is used to segment the pre-trained target large model to obtain an input layer, an intermediate layer, and an output layer corresponding to the target large model;

[0164] A deployment module 12, configured to deploy the input layer and the output layer on the client, and deploy the intermediate layer on the server;

[0165] The encoding module 13 is used to receive the Prompt input by the target user on the client, and encode the Prompt through the input layer to obtain a corresponding numerical vector;

[0166] The reasoning module 14 is used to upload the numerical vector to the server, and to reason the numerical vector through the intermediate layer to obtain a corresponding result vector;

[0167] The decoding module 15 is used to return the result vector to the client, and decode the result vector through the output layer to obtain a corresponding result text.

[0168] Furthermore, the large model inference system, wherein the pre-trained target large model is segmented to obtain an input layer, an intermediate layer, and an output layer corresponding to the target large model, includes:

[0169] Predetermine the segmentation strategy based on user needs;

[0170] The segmentation strategy is used to segment the pre-trained target large model to obtain the input layer, the middle layer, and the output layer corresponding to the target large model.

[0171] In the specific implementation, in this embodiment, a segmentation strategy is predetermined according to user needs, and the segmentation strategy is used to segment the pre-trained target large model to obtain the input layer, the intermediate layer, and the output layer corresponding to the target large model. The specific implementation process is as follows:

[0172] 1) Demand analysis and segmentation strategy determination

[0173] First, analyze and determine an appropriate model segmentation strategy based on user needs and application scenarios; this may include deciding which layers need to be segmented and how they work together in the system architecture after segmentation;

[0174] 2) Model segmentation implementation

[0175] The pre-trained target large model (which can be a federated large model) is segmented using a predetermined segmentation strategy; segmentation usually involves the following steps:

[0176] Identify the input layer (such as the Embedding layer), which is usually the first layer of the model to receive raw data;

[0177] Identify the intermediate layers, which may include multiple hidden layers or specific layers (such as Transformer layers) that are responsible for feature extraction and processing in the model;

[0178] Identify the output layer (such as the Decoder layer), which is responsible for generating the final inference results or outputs;

[0179] Finally, the input layer, intermediate layer, and output layer corresponding to the target large model are obtained;

[0180] 3) Deployment of model layers after segmentation

[0181] Each layer after segmentation will be deployed to the corresponding computing environment according to the established deployment strategy, which may be the client, server or a node in the distributed system;

[0182] 4) Model Collaboration

[0183] Ensure that the split model layers can work together to complete the entire reasoning process from input to output; this may require defining clear data transmission protocols and interfaces;

[0184] 5) Testing and verification

[0185] Test and validate the segmented model to ensure that its performance is comparable to the original unsegmented model and meets user needs and expectations;

[0186] 6) Optimization and adjustment

[0187] According to the test results and actual operation conditions, make necessary optimization and adjustments to the segmentation strategy and deployment;

[0188] Through the above process, the flexibility of model deployment can be improved, the requirements for computing resources can be reduced, and the privacy protection capabilities of the model can be enhanced; at the same time, this method also helps to achieve customized deployment of the model to meet the needs of specific application scenarios.

[0189] Furthermore, the large model reasoning system, wherein the input layer and the output layer are both deployed on the client, and the intermediate layer is deployed on the server, comprises:

[0190] Predetermining a deployment strategy based on the user needs;

[0191] The deployment strategy is adopted to deploy the input layer and the output layer on the client, and to deploy the intermediate layer on the server.

[0192] In the specific implementation, in this embodiment, according to the user's needs, a deployment strategy is pre-determined, and the input layer and the output layer are deployed on the client side, and the intermediate layer is deployed on the server side using the deployment strategy. The specific implementation process is as follows:

[0193] 1) Requirements analysis and deployment strategy formulation

[0194] Based on the above user needs and application scenarios, analyze and determine a suitable deployment strategy, clarify the deployment details of the input layer and output layer on the client, and the deployment plan of the middle layer on the server;

[0195] 2) Model segmentation and optimization

[0196] After using the model segmentation technology to split the pre-trained target large model into independent components, the segmented model components are optimized to adapt to different operating environments and ensure that they can work efficiently together on the client and server.

[0197] 3) Security and communication protocol design

[0198] Design security protocols for data transmission between the client and the server, including data encryption and authentication mechanisms, to ensure data security and privacy protection during transmission; at the same time, define clear API interfaces and data formats so that the client can send requests to the server and transmit the result vector back to the client;

[0199] 4) Distributed deployment implementation

[0200] Deploy the input layer and output layer on the client to ensure that they can process the user's input data and present the final output of the model; deploy the middle layer on the server and use the server's computing resources to perform the main reasoning tasks of the model; configure the server environment to support high-performance operation and scalability of the model;

[0201] 5) System integration testing and iteration

[0202] Conduct integration testing on the entire system to verify the end-to-end performance after model segmentation and deployment; check whether the client and server work smoothly together, whether data transmission is secure, and whether the model output is accurate; make necessary adjustments and optimizations based on the test results to ensure that the system meets user needs and provides high-quality services;

[0203] Through the above process, it can be ensured that the deployment of the model not only meets user needs but also guarantees the security, performance and scalability of the system.

[0204] Furthermore, the large model reasoning system, wherein the receiving of the Prompt input by the target user on the client, and encoding the Prompt through the input layer to obtain a corresponding numerical vector, includes:

[0205] Receive a prompt input by the target user on the client;

[0206] Encode the Prompt by the input layer deployed on the client to obtain a corresponding initial numerical vector;

[0207] The initial numerical vector is searched and enhanced by using a preset private knowledge base to obtain a corresponding numerical vector.

[0208] In specific implementation, in this embodiment, a prompt input by a target user on the client is received, the prompt is encoded by the input layer deployed on the client to obtain a corresponding initial numerical vector, and a search enhancement process is performed on the initial numerical vector through a preset private knowledge base to obtain a corresponding numerical vector, wherein the specific implementation process is as follows:

[0209] 1) User input reception and encoding

[0210] The target user enters a prompt on the client. The client interface design should ensure the convenience and accuracy of user input. After receiving the prompt, the input layer converts it into an initial numerical vector. This usually involves steps such as word segmentation and word embedding search to convert the text into a numerical form that the model can understand.

[0211] 2) Enhanced private knowledge base retrieval

[0212] After the initial numerical vector is generated, the client uses the private knowledge base for retrieval enhancement; this step may include querying the knowledge base to obtain additional information or contextual data related to the prompt, using retrieval algorithms (such as vector similarity search) to find the most relevant knowledge fragments, and converting them into numerical vectors or features;

[0213] 3) Vector fusion and enhancement processing

[0214] The retrieved target numerical vector is fused with the initial numerical vector to form an enhanced numerical vector; the fusion method may include simple concatenation, weighted averaging, or more complex feature interaction methods, the purpose of which is to integrate information from prompts and private knowledge bases to enrich the model input;

[0215] 4) Security and privacy protection

[0216] Ensure the security and privacy of user data throughout the entire processing process; this may include encrypting data in transit and performing local processing on the client side to reduce exposure of sensitive information;

[0217] 5) Result vector output

[0218] The enhanced numeric vectors are output, which now contain the information of the original prompt as well as the relevant information retrieved from the private knowledge base; the client prepares these vectors for secure transmission to the server for further reasoning processing;

[0219] Through the above process, the client not only converts the user's natural language prompt into numerical representations that the model can understand, but also enhances these numerical representations through a private knowledge base, providing richer and more relevant input data for the server's deep learning model.

[0220] Furthermore, in the large model reasoning system, the uploading of the numerical vector to the server and the reasoning of the numerical vector through the intermediate layer to obtain the corresponding result vector include:

[0221] Uploading the numerical vector obtained after the retrieval enhancement processing to the server;

[0222] Inferring the numerical vector by the intermediate layer deployed on the server to obtain a corresponding initial result vector;

[0223] The initial result vector is normalized to obtain a corresponding result vector.

[0224] In specific implementation, in this embodiment, the numerical vector obtained after the search enhancement processing is uploaded to the server, the numerical vector is inferred by the intermediate layer deployed on the server to obtain a corresponding initial result vector, and the initial result vector is normalized to obtain a corresponding result vector, wherein the specific implementation process is as follows:

[0225] 1) Upload numerical vector

[0226] The client uploads the retrieved enhanced numerical vector to the server through a secure network connection; this step requires ensuring the security of data transmission, which may involve the use of SSL encryption or other security protocols;

[0227] 2) Server-side reasoning processing

[0228] After receiving the numerical vector, the server inputs it into the middle layer (such as the Transformer layer) deployed on the server. The middle layer performs deep processing on the numerical vector, uses the parameters and algorithms of the model for reasoning, and generates an initial result vector.

[0229] 3) Result vector generation

[0230] After the server-side middle layer completes the calculation, the generated initial result vector contains the model's response to the input numerical vector; these vectors are usually in a raw state and require further processing to obtain the final output;

[0231] 4) Standardized processing

[0232] Normalize the initial result vector to ensure consistency and accuracy of the result; normalization may include scaling the value range of the vector, standardizing the vector to fit a specific distribution, or applying a specific mathematical transformation to optimize the quality of the result;

[0233] 5) Final result output

[0234] The normalized result vector is the final output, which the server can send back to the client for decoding to generate understandable result text; this step needs to ensure the accuracy and usability of the result;

[0235] Through the above process, the server not only completes the deep reasoning of complex numerical vectors, but also ensures the quality and availability of the results through normalization processing, providing the client with reliable and accurate output.

[0236] Furthermore, the large model reasoning system, wherein the returning the result vector to the client and decoding the result vector through the output layer to obtain the corresponding result text, includes:

[0237] Returning the result vector to the client;

[0238] Decoding the result vector through the output layer deployed on the client to obtain a corresponding initial result text;

[0239] Correction processing is performed on the initial result text to obtain a corresponding result text.

[0240] In the specific implementation, in this embodiment, the result vector is returned to the client; the result vector is decoded by the output layer deployed on the client to obtain the corresponding initial result text; the initial result text is corrected to obtain the corresponding result text; wherein the specific implementation process is as follows:

[0241] 1) Result vector is returned

[0242] After the server completes the processing, it sends the normalized result vector back to the client securely over the network; this step needs to ensure the integrity and confidentiality of the data during transmission;

[0243] 2) Client decoding

[0244] After receiving the result vector, the client uses the locally deployed output layer to decode the vector; the output layer usually includes one or more neural network layers, which are responsible for converting the numerical vector into text form;

[0245] 3) Initial result text generation

[0246] The output layer generates the initial result text from the decoded results; this may be one or more candidate sentences, depending on how the model is decoded and the type of task;

[0247] 4) Text correction processing

[0248] Apply natural language processing techniques to the initial result text for correction and optimization; this may include grammar checking, spelling correction, contextual word sense disambiguation, and improving sentence flow and logic;

[0249] 5) Final result output

[0250] The corrected text is presented to the target user as the final result text; this step ensures the quality of the output text and improves the usability of the model and user satisfaction.

[0251] Through the above process, the client not only receives and decodes the result vector from the server, but also further improves the quality of the result through text correction processing, ensuring the accuracy and readability of the output.

[0252] Furthermore, in the large model reasoning system, the target large model is a federated large model, and the input layer, the intermediate layer, and the output layer are respectively an Embedding layer, a Transformer layer, and a Decoder layer;

[0253] The private knowledge base is deployed on the client and / or the server.

[0254] In the specific implementation, in this embodiment, the target large model is a federated large model, the input layer, the intermediate layer, and the output layer are respectively an Embedding layer, a Transformer layer, and a Decoder layer; the private knowledge base is deployed on the client and / or the server; wherein the specific implementation process is roughly described as follows:

[0255] 1) Deployment of private knowledge base

[0256] Determine the deployment location of the private knowledge base based on data privacy and security requirements; if the privacy requirements are extremely high, the knowledge base can be deployed on the client; if more powerful computing resources are needed to support the retrieval and processing of the knowledge base, it can also be deployed on the server, or in a distributed deployment, partly on the client and partly on the server;

[0257] 2) User input and Embedding layer encoding

[0258] The target user enters a prompt on the client; the client's Embedding layer converts the prompt into an initial numerical vector to prepare for the next step of retrieval and reasoning;

[0259] 3) Retrieval enhancement processing

[0260] Use a private knowledge base to perform retrieval enhancement on the initial numeric vector generated by the Embedding layer; this may involve querying the knowledge base to obtain relevant information and fusing this information into the initial numeric vector to enrich the context of the input data and obtain the corresponding numeric vector;

[0261] 4) Server-side Transformer layer reasoning

[0262] The client uploads the retrieved enhanced numerical vectors to the server; the server's Transformer layer receives these vectors and performs further reasoning to generate preliminary initial result vectors;

[0263] 5) Normalization of the result vector and decoding at the Decoder layer

[0264] The server normalizes the initial result vector obtained by reasoning to optimize the quality of the result and obtain the corresponding result vector;

[0265] The server sends the result vector back to the client for decoding. After receiving the result vector, the client uses the locally deployed Decoder layer to decode the result vector to generate an initial result text. The client corrects and optimizes the initial result text to generate an understandable result text.

[0266] 6) Final result output

[0267] The resulting text after correction is presented to the target user as the final output; this step ensures the accuracy, fluency and readability of the output text;

[0268] Through the above process, the federated big model can achieve efficient distributed processing while ensuring the privacy and security of user data; this architecture is particularly suitable for application scenarios with strict requirements on data privacy, such as finance, medical care and other fields.

[0269] It can be seen from the above system embodiments that the large model inference system provided by the present invention at least includes: segmenting the pre-trained target large model to obtain the input layer, intermediate layer, and output layer corresponding to the target large model; deploying the input layer and the output layer on the client, and deploying the intermediate layer on the server; receiving the Prompt input by the target user on the client, and encoding the Prompt through the input layer to obtain the corresponding numerical vector; uploading the numerical vector to the server, and inferring the numerical vector through the intermediate layer to obtain the corresponding result vector; returning the result vector to the client, and decoding the result vector through the output layer to obtain the corresponding result text; and so on. In this way, the system of the present invention can achieve privacy protection for the large model inference stage.

[0270] Specifically, the large model inference system provided by the embodiment of the present invention innovatively splits the large model into an input layer, an intermediate layer, and an output layer, and distributes it on the client and server in the actual application scenario. While ensuring that the user's plaintext data privacy does not leave the local area, it can better request the large model service of the server. This not only protects the user's data privacy, but also makes full use of the powerful computing power of the server, making it easier to implement and helping to serve all walks of life.

[0271] Based on the above method embodiment, please refer to Figure 3 Another embodiment of the present invention further provides a computer device, wherein the computer device 10 includes:

[0272] memory 120 and one or more processors 110, Figure 3 A processor 110 is used as an example for description. The processor 110 and the memory 120 may be connected via a communication bus or other means. Figure 3 The example of connection via a communication bus is taken as an example.

[0273] The processor 110 is used to complete various control logics of the computer device 10. It can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of these components. In addition, the processor 110 can also be any traditional processor, microprocessor or state machine. The processor 110 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP, or any other such configuration.

[0274] The memory 120, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the computer program corresponding to the large model reasoning method in the embodiment of the present invention. The processor 110 executes various functional applications and data processing of the computer device 10 by running the non-volatile software programs, instructions and units stored in the memory 120, that is, implements the large model reasoning method in the above method embodiment.

[0275] The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store an application required for operating the device or at least one function; the data storage area may store data created according to the use of the computer device 10, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include a memory remotely arranged relative to the processor 110, and these remote memories may be connected to the computer device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0276] One or more units are stored in the memory 120, and when executed by one or more processors 110, a large model reasoning method as in any of the above method embodiments can be implemented, for example, the above described Figure 1 The method comprises steps S100 to S500.

[0277] Those skilled in the art will understand that Figure 3 The hardware structure diagram shown in the figure is only a schematic diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0278] Based on the above method embodiments, another embodiment of the present invention further provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the large model reasoning method in any of the above method embodiments can be implemented, for example, the above described Figure 1 The method comprises steps S100 to S500.

[0279] As an example, the non-volatile storage medium can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable ROM (EEPROM), or a flash memory. Volatile memory can include a random access memory (RAM) as an external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM, (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synch link DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0280] Based on the above method embodiments, another embodiment of the present invention further provides a computer program product, the computer program product includes a computer program stored on a non-volatile computer-readable storage medium, the computer program includes program instructions, when the program instructions are executed by a processor, the large model reasoning method in any of the above method embodiments can be implemented, for example, the above described Figure 1 The method comprises steps S100 to S500.

[0281] The embodiments described above are merely illustrative, in which the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0282] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can exist in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0283] Conditional language such as "can," "can," "might," or "may," unless specifically stated otherwise or otherwise understood within the context as used, is generally intended to convey, among other things, that a particular embodiment can include (while other embodiments do not) a particular feature, element, and / or operation. Thus, such conditional language is also generally intended to imply that a feature, element, and / or operation is required for one or more embodiments anyway or that one or more embodiments must include logic for determining, with or without input or prompting, whether such features, elements, and / or operations are included or will be performed in any particular embodiment.

[0284] What has been described herein, in this specification and in the accompanying drawings includes examples that can provide a large model reasoning method, system, computer device, and non-volatile computer-readable storage medium. Of course, it is not possible to describe every conceivable combination of elements and / or methods for the purpose of describing the various features of the present disclosure, but it can be recognized that many other combinations and permutations of the disclosed features are possible. Therefore, it is obvious that various modifications can be made to the present disclosure without departing from the scope or spirit of the present disclosure, but all these various modifications should belong to the scope of protection of the claims attached to the present invention. In addition, or in an alternative, other embodiments of the present disclosure may be obvious from consideration of this specification and the accompanying drawings and from the practice of the present disclosure as presented herein. It is intended that the examples proposed in this specification and the accompanying drawings are considered to be illustrative and not restrictive in all respects. Although specific terms are used herein, they are used in a general and descriptive sense and are not used for limiting purposes.

Claims

1. A large model reasoning method, characterized in that: include: Segmenting the pre-trained target large model to obtain an input layer, an intermediate layer, and an output layer corresponding to the target large model; The input layer and the output layer are both deployed on the client, and the middle layer is deployed on the server; Receiving a Prompt input by a target user on the client, and encoding the Prompt through the input layer to obtain a corresponding numerical vector; Uploading the numerical vector to the server, and performing reasoning on the numerical vector through the intermediate layer to obtain a corresponding result vector; The result vector is returned to the client, and the result vector is decoded through the output layer to obtain a corresponding result text.

2. The large model reasoning method according to claim 1, characterized in that: The pre-trained target large model is segmented to obtain an input layer, an intermediate layer, and an output layer corresponding to the target large model, including: Predetermine the segmentation strategy based on user needs; The segmentation strategy is used to segment the pre-trained target large model to obtain the input layer, the middle layer, and the output layer corresponding to the target large model.

3. The large model reasoning method according to claim 2, characterized in that: The step of deploying both the input layer and the output layer on the client and the middle layer on the server includes: Predetermining a deployment strategy based on the user needs; The deployment strategy is adopted to deploy the input layer and the output layer on the client, and to deploy the intermediate layer on the server.

4. The large model reasoning method according to claim 3, characterized in that: The receiving the Prompt input by the target user at the client, and encoding the Prompt through the input layer to obtain a corresponding numerical vector, includes: Receive a prompt input by the target user on the client; Encode the Prompt by the input layer deployed on the client to obtain a corresponding initial numerical vector; The initial numerical vector is searched and enhanced by using a preset private knowledge base to obtain a corresponding numerical vector.

5. The large model reasoning method according to claim 4, characterized in that: The uploading of the numerical vector to the server and inferring the numerical vector through the intermediate layer to obtain a corresponding result vector includes: Uploading the numerical vector obtained after the retrieval enhancement processing to the server; Inferring the numerical vector by the intermediate layer deployed on the server to obtain a corresponding initial result vector; The initial result vector is normalized to obtain a corresponding result vector.

6. The large model reasoning method according to claim 5, characterized in that: The returning the result vector to the client, and decoding the result vector through the output layer to obtain the corresponding result text, includes: Returning the result vector to the client; Decoding the result vector through the output layer deployed on the client to obtain a corresponding initial result text; Correction processing is performed on the initial result text to obtain a corresponding result text.

7. The large model reasoning method according to any one of claims 1 to 6, characterized in that: The pre-trained target large model is segmented to obtain an input layer, an intermediate layer, and an output layer corresponding to the target large model, wherein: The target large model is a federated large model, and the input layer, the intermediate layer, and the output layer are respectively an Embedding layer, a Transformer layer, and a Decoder layer.

8. A large model reasoning system, characterized in that: include: A segmentation module is used to segment the pre-trained target large model to obtain an input layer, an intermediate layer, and an output layer corresponding to the target large model; A deployment module, used to deploy the input layer and the output layer on the client, and deploy the middle layer on the server; An encoding module, used for receiving a Prompt input by a target user on the client, and encoding the Prompt through the input layer to obtain a corresponding numerical vector; An inference module, used for uploading the numerical vector to the server, and performing inference on the numerical vector through the intermediate layer to obtain a corresponding result vector; The decoding module is used to return the result vector to the client and decode the result vector through the output layer to obtain a corresponding result text.

9. A computer device, characterized in that: The computer device comprises at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor. When the computer program is executed by the at least one processor, the large model reasoning method as described in any one of claims 1 to 7 can be implemented.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, it can implement the large model reasoning method as described in any one of claims 1-7.

Images