Large model service system and method, computer equipment and readable storage medium

Abstract

The invention provides a large model service system and method, computer equipment and a readable storage medium. A cue word engineering module configures a cue word template; generating a target text according to the prompt word selected by the user; the retrieval enhancement generation module receives a basic text uploaded by a user, converts the basic text into word vectors, stores the word vectors in a database, and retrieves prompt words from the database according to question information input by the user to generate a target text; the plug-in development module receives the configuration file, analyzes the configuration file to obtain API interface information, calls extended data to retrieve cue words, and generates a target text based on the cue words; and the agent module defines the execution combination and sequence of the cue word engineering module, the retrieval enhancement generation module and the plug-in development module according to the user-defined operation of the user on the system execution chain. By adopting the method, the flexibility of large model service, the efficiency of model retrieval and the safety of model service are improved, diversified business scene requirements are further met, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer model technology, and in particular to a large model service system, method, computer equipment and readable storage medium. Background Art

[0002] In the field of artificial intelligence, the rise of large language model service systems is like a technological revolution. It has penetrated into all walks of life with an unstoppable momentum and reshaped the technology ecosystem. These models, with their powerful language processing capabilities, have not only attracted widespread attention from ordinary people, but have also become an indispensable technical pillar for large enterprises.

[0003] However, the study found that the existing model service system has the following defects: lack of flexibility. Traditional text processing or generation systems often lack sufficient flexibility and cannot be customized according to the specific needs of users. These systems can usually only run according to preset processes or algorithms and are difficult to adapt to diverse application scenarios; low retrieval efficiency. In terms of text retrieval, traditional systems may rely on simple methods such as keyword matching, resulting in low retrieval efficiency and low accuracy. The limitations of this method are particularly obvious when processing large-scale text data; low security. It performs poorly in controlling the input and output of large models, authority control, channel management, security control, and tenant management, and cannot meet the growing security needs. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a large model service system, method, computer equipment and readable storage medium to improve the flexibility of large model services, the efficiency of model retrieval, and the security of model services, further meet the needs of diversified business scenarios, and improve user experience.

[0005] In a first aspect, an embodiment of the present application provides a large model service system, the system comprising a prompt word engineering module, a search enhancement generation module, a plug-in development module and an intelligent agent module;

[0006] The prompt word engineering module is used to configure a plurality of prompt word templates and display them to a target user; and is also used to generate a first target text based on a first prompt word in the plurality of prompt word templates in response to a selection operation of the target user on the first prompt word;

[0007] The retrieval enhancement generation module is used to receive the basic text uploaded by the target user, convert the basic text into a number of high-dimensional word vectors, and store each high-dimensional word vector in a vector database to form a retrieval database; it is also used to receive the question information input by the target user, retrieve a second prompt word from the retrieval database according to the question information, and generate a second target text based on the second prompt word;

[0008] The plug-in development module is used to receive the configuration file and question information input by the target user, parse the configuration file, and obtain detailed information of the third-party API interface based on the parsing result; it is also used to determine whether the third-party API interface needs to be called based on the question information, and when the third-party API interface needs to be called, call the extended data from the third-party API interface; it is also used to retrieve a third prompt word from the extended data based on the question information, and generate a third target text based on the third prompt word;

[0009] The intelligent agent module is used to receive the target user's custom operations on the system execution chain, and define the execution combination and / or execution order of the prompt word engineering module, the retrieval enhancement generation module and the plug-in development module based on the custom operations.

[0010] Optionally, generating a first target text based on the first prompt word includes:

[0011] Using the target macro model, generating the first target text according to the first prompt word;

[0012] The generating a second target text based on the second prompt word includes:

[0013] Using the target macro model, generating the second target text according to the second prompt word;

[0014] The step of generating a third target text based on the third prompt word includes:

[0015] The target macro model is used to generate the third target text according to the third prompt word.

[0016] Optionally, the system further includes a model training module and a model management module;

[0017] The model training module is used to perform model training on the initial large model;

[0018] The model management module is used to test the trained initial large model and deploy the initial large model that passes the test as the target large model.

[0019] Optionally, converting the basic text into a number of high-dimensional word vectors includes:

[0020] The basic text is parsed through an embedding model to obtain a number of high-dimensional word vectors, wherein the embedding model includes an M3E model, a GTE model, and a BGE model.

[0021] Optionally, the target large model includes a GLM model and a Qwen model.

[0022] Optionally, retrieving a second prompt word from the search database according to the question information includes:

[0023] Performing text vectorization on the question information to obtain an initial word vector;

[0024] Calculating the similarity between the initial word vector and each high-dimensional word vector in the retrieval database respectively;

[0025] A high-dimensional word vector in the retrieval database whose similarity with the initial word vector exceeds a preset threshold is determined as the second prompt word.

[0026] In a second aspect, an embodiment of the present application provides a large model service method, which is applied to a large model service system. The system includes a prompt word engineering module, a search enhancement generation module, a plug-in development module, and an agent module. The method also includes:

[0027] The prompt word engineering module configures a plurality of prompt word templates and displays them to a target user; and is further configured to generate a first target text based on a first prompt word in the plurality of prompt word templates in response to a selection operation of the target user on the first prompt word;

[0028] The retrieval enhancement generation module receives the basic text uploaded by the target user, converts the basic text into a number of high-dimensional word vectors, and stores each high-dimensional word vector in a vector database to form a retrieval database; it is also used to receive question information input by the target user, retrieve a second prompt word from the retrieval database according to the question information, and generate a second target text based on the second prompt word;

[0029] The plug-in development module receives the configuration file and question information input by the target user, parses the configuration file, and obtains detailed information of the third-party API interface based on the parsing result; determines whether the third-party API interface needs to be called based on the question information, and when the third-party API interface needs to be called, calls the extended data from the third-party API interface; retrieves a third prompt word from the extended data based on the question information, and generates a third target text based on the third prompt word;

[0030] The intelligent agent module receives the target user's custom operation on the system execution chain, and defines the execution combination and / or execution order of the prompt word engineering module, the retrieval enhancement generation module and the plug-in development module based on the custom operation.

[0031] Optionally, generating a first target text based on the first prompt word includes:

[0032] Using the target macro model, generating the first target text according to the first prompt word;

[0033] The generating a second target text based on the second prompt word includes:

[0034] Using the target macro model, generating the second target text according to the second prompt word;

[0035] The step of generating a third target text based on the third prompt word includes:

[0036] The target macro model is used to generate the third target text according to the third prompt word.

[0037] Optionally, the system further includes a model training module and a model management module, and the method further includes:

[0038] The model training module performs model training on the initial large model;

[0039] The model management module tests the trained initial large model and deploys the initial large model that passes the test as the target large model.

[0040] Optionally, converting the basic text into a number of high-dimensional word vectors includes:

[0041] The basic text is parsed through an embedding model to obtain a number of high-dimensional word vectors, wherein the embedding model includes an M3E model, a GTE model, and a BGE model.

[0042] Optionally, the target large model includes a GLM model and a Qwen model.

[0043] Optionally, retrieving a second prompt word from the search database according to the question information includes:

[0044] Performing text vectorization on the question information to obtain an initial word vector;

[0045] Calculating the similarity between the initial word vector and each high-dimensional word vector in the retrieval database respectively;

[0046] A high-dimensional word vector in the retrieval database whose similarity with the initial word vector exceeds a preset threshold is determined as the second prompt word.

[0047] In the third aspect, an embodiment of the present application provides a computer device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the large model service method described in any optional implementation manner of the second aspect are performed.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the large model service method described in any optional implementation manner in the second aspect above are executed.

[0049] The technical solution provided by this application includes but is not limited to the following beneficial effects:

[0050] This application constructs a large model service system, and the prompt word engineering module configures several prompt word templates and displays them to the target user; it is also used to respond to the target user's selection operation of the first prompt word in several prompt word templates, and generate a first target text based on the first prompt word, which can improve the efficiency and effect of the user's interaction with the large model, so that the large model can better understand and respond to the needs of the user, thereby generating high-quality official documents, reports and other contents. The retrieval enhancement generation module receives the basic text uploaded by the target user, converts the basic text into several high-dimensional word vectors, and stores each high-dimensional word vector in the vector database to form a retrieval database; it is also used to receive the question information input by the target user, retrieve the second prompt word from the retrieval database according to the question information, and generate the second target text based on the second prompt word, which can achieve fast and accurate knowledge retrieval and generation, and provide real-time knowledge support for users. By setting up a plug-in development module, users can quickly develop and integrate custom functions according to their own business needs, greatly expanding the application scope of the platform and adapting to different business scenarios. By setting up intelligent phantoms to autonomously perform tasks in various application scenarios, human intervention is reduced, and work efficiency and automation level are improved. The intelligent agent module receives the target user's customized operations on the system execution chain, and defines the execution order of the prompt word engineering module and the retrieval enhancement generation module based on the customized operations. It can realize fast and accurate knowledge retrieval and generation, and provide users with real-time knowledge support. It is suitable for information-intensive work scenarios, such as legal document writing, scientific research report generation, etc.

[0051] By adopting the above solution, the complex technical details of the big model are abstracted, and by providing concise and clear interfaces and tools, the threshold for use is lowered, and the application and implementation of big model capabilities are promoted. At the same time, by building an open ecosystem for big models, more developers are attracted to join and a prosperous ecosystem is formed to meet the needs of different business scenarios. The survey found that the industry's leading companies have formed a trend of model service platformization. Through the mutual cooperation of various modules in the big model service system, the flexibility of big model services, the efficiency of model retrieval, and the security of model services can be improved, further meeting the needs of diverse business scenarios and improving user experience.

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 A schematic diagram of the structure of a large model service system provided by the first embodiment of the present invention is shown;

[0055] Figure 2 A schematic diagram showing the structure of a second large model service system provided by the first embodiment of the present invention is shown;

[0056] Figure 3 A schematic diagram showing a large model service platform architecture provided by the first embodiment of the present invention;

[0057] Figure 4 A flow chart of a large model service method provided by the second embodiment of the present invention is shown;

[0058] Figure 5 A schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention is shown. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0060] Embodiment 1

[0061] To facilitate understanding of this application, Figure 1 The content described in the structural schematic diagram of a large model service system provided by the first embodiment of the present invention is described in detail.

[0062] See also Figure 1 As shown, Figure 1 A structural schematic diagram of a large model service system provided by the first embodiment of the present invention is shown, wherein the system includes a prompt word engineering module 101, a retrieval enhancement generation module 102, a plug-in development module 103 and an intelligent agent module 104.

[0063] The prompt word engineering module is used to configure a plurality of prompt word templates and display them to a target user; and is also used to generate a first target text based on a first prompt word in the plurality of prompt word templates in response to a selection operation of the target user on the first prompt word.

[0064] Specifically, the module has a built-in prompt word template square, which is configured with several prompt word templates to display to users, so that users can select and apply corresponding prompt word templates according to different needs. In addition, the module uses multiple frameworks such as CO-STAR framework, ICIO framework, CRISPE framework, etc. to optimize prompt words, and can dynamically adjust and optimize prompt words according to user input to improve the accuracy and relevance of model response. After the user selects the initial prompt word, the module responds to the target user's selection operation of the initial prompt word in several prompt word templates, and generates the first target text using the preset large model.

[0065] The retrieval enhancement generation module is used to receive the basic text uploaded by the target user, convert the basic text into a number of high-dimensional word vectors, and store each high-dimensional word vector in a vector database to form a retrieval database; it is also used to receive the question information input by the target user, retrieve a second prompt word from the retrieval database according to the question information, and generate a second target text based on the second prompt word.

[0066] Specifically, users can upload basic text through this module, and the module converts the basic text into high-dimensional word vectors through the embedding model. These word vectors are stored in an efficient vector database to obtain a retrieval database, providing data support for subsequent text retrieval. When performing text retrieval, users input question information through natural language. The module can quickly find the second prompt word from the vector database based on the question information through the large model, and provide enhanced responses in combination with the generation capabilities of the large model to generate target reply text, thereby achieving fast and accurate knowledge retrieval and generation.

[0067] The plug-in development module is used to receive the configuration file and question information input by the target user, parse the configuration file, and obtain detailed information of the third-party API interface based on the parsing result; it is also used to determine whether the third-party API interface needs to be called based on the question information, and when the third-party API interface needs to be called, call the extended data from the third-party API interface; it is also used to retrieve a third prompt word from the extended data based on the question information, and generate a third target text based on the third prompt word.

[0068] Specifically, the module defines the plug-in interface by writing a yaml configuration file, and uses the function call capability of the big model to parse and execute these configuration files, so that the big model service system has an API interface that can communicate with the third-party system. The big model service system obtains extended data from the third-party system through the API interface based on the user's question information through the big model. When performing text retrieval, the user inputs the question information through natural language. The module can quickly find the second prompt word from the extended data based on the question information through the big model, and generate the target reply text in combination with the generation capability of the big model. In addition, the system supports hot loading and real-time updating of plug-ins, which improves the efficiency of plug-in development and deployment.

[0069] The intelligent agent module is used to receive the target user's custom operations on the system execution chain, and define the execution combination and / or execution order of the prompt word engineering module, the retrieval enhancement generation module and the plug-in development module based on the custom operations.

[0070] Specifically, the agent module provides the target user with an execution chain definition interface, so that the user can define the execution chain and process arrangement through the module to create a customized agent, and the agent can autonomously execute complex tasks and workflows, and define the execution order and execution combination of the prompt word engineering module, the search enhancement generation module, and the extended data. The constructed large model agent supports the flexible combination and mixing of multiple capabilities (including prompt words, search enhancement generation RAG, plug-in development), and realizes the openness and sharing of Agent agents through standardized service release processes to meet the needs of different business scenarios.

[0071] The agent module is also used to provide a system display interface for the target user, and display the target reply text to the target user through the system display interface. Specifically, the module provides a system display interface and a system customization tool, supports service publishing and channel management, including displaying the target reply text to the target user through the system display interface, and provides a system customization tool so that the user can customize the system through the system customization tool.

[0072] In a feasible implementation manner, the generating the first target text based on the first prompt word includes:

[0073] Using the target macro model, generating the first target text according to the first prompt word;

[0074] The generating a second target text based on the second prompt word includes:

[0075] Using the target macro model, generating the second target text according to the second prompt word;

[0076] The step of generating a third target text based on the third prompt word includes:

[0077] The target macro model is used to generate the third target text according to the third prompt word.

[0078] Specifically, the target big model used in this application includes several open source big models and commercial big models. The first prompt word / the second prompt word / the third prompt word is input into the target big model, and the first target text / the second target text / the third target text is obtained based on the text generation capability of the big model itself.

[0079] In one possible embodiment, see Figure 2 As shown, Figure 2 A schematic diagram of the structure of a second large model service system provided by the first embodiment of the present invention is shown, wherein the system further includes a model training module 201 and a model management module 202.

[0080] The model training module is used to perform model training on the initial large model.

[0081] Specifically, the model training module provides model training functions, including obtaining a model training sample set, and performing model training and optimization on a large model based on the model training sample set.

[0082] The model management module is used to test the trained initial large model and deploy the initial large model that passes the test as the target large model.

[0083] Specifically, the model management module provides functions such as model testing, version management, and performance monitoring, and supports users to deploy models based on model testing results.

[0084] In a feasible implementation scheme, the converting the basic text into a number of high-dimensional word vectors includes:

[0085] The basic text is parsed through an embedding model to obtain a number of high-dimensional word vectors, wherein the embedding model includes an M3E model, a GTE model, and a BGE model.

[0086] Specifically, a model is randomly selected from the above embedding models to map the basic text into the vector space for text vectorization processing, thereby obtaining several high-dimensional word vectors.

[0087] In a feasible implementation manner, the target macro model includes a GLM model and a Qwen model.

[0088] Specifically, a model is randomly selected from the above target large model for text retrieval.

[0089] In a feasible implementation manner, the retrieving the second prompt word from the search database according to the question information includes:

[0090] The question information is subjected to text vectorization processing to obtain an initial word vector; the similarity between the initial word vector and each high-dimensional word vector in the retrieval database is calculated respectively; and the high-dimensional word vector in the retrieval database whose similarity with the initial word vector exceeds a preset threshold is determined as the second prompt word.

[0091] Specifically, the embedding module performs text vectorization on the question information to obtain an initial word vector, and then calculates the similarity between the initial word vector and the high-dimensional word vector in the retrieval database. The high-dimensional word vector in the retrieval database whose similarity with the initial word vector exceeds a preset threshold or ranks in the top preset position is selected as the second prompt word.

[0092] The large model service system provided in this application is deployed as a platform layer in the AI ​​large model service platform. Figure 3 As shown, Figure 3 A schematic diagram of a large model service platform architecture provided by Embodiment 1 of the present invention is shown, wherein the large model service platform combines a large-parameter commercial basic model and multiple open source large models to form a multi-dimensional AI large model basic base. On this base, we are committed to providing rich large model basic resources while ensuring data security to adapt to various types of large model application scenarios in various fields and meet the needs of future technological development. The AI ​​large model service platform proposed in this application has a four-layer architecture, which realizes the efficient development and management of intelligent applications by integrating multiple large models and providing rich functional modules. The platform can be widely used in multiple fields such as intelligent assistants, office assistants, robots, and document generation. The large model service platform includes an infrastructure layer, a model layer, a platform layer, and an intelligent application layer.

[0093] The infrastructure layer is used to provide basic computing resources such as GPU computing power, high-performance storage systems, and high-speed network bandwidth as the computing foundation of the entire platform. These infrastructures ensure that the platform can efficiently process large-scale data and complex computing tasks.

[0094] The model layer contains multiple inference models and embedding models. The inference models include multiple open source models (such as the GLM model, Qwen model, and Llama model) and multiple large commercial models. The embedding models include the M3E model, GTE model, BGE model, etc. These models are used to convert text into high-dimensional word vectors for subsequent processing.

[0095] The platform layer is used to integrate the large model capabilities of the model layer. As the large model service system provided by this application, the prompt word engineering module provides users with accurate prompt words through the built-in prompt word template and prompt word Prompt optimization technology of the template square, so that the large model can generate high-quality responses. A knowledge base is established through the RAG retrieval generation enhancement module, a front-end knowledge base management module is provided, and the uploaded documents are converted into word vectors using an embedded model and stored in a vector database for document management, document parsing, document segmentation, and knowledge recall, etc., to achieve retrieval enhancement based on the generation capability of the large model. Through the function calling capability of the plug-in development module based on the underlying large model, users are allowed to customize plug-in development. Users only need to write a yaml configuration file that defines the interface, and the large model can parse and perform corresponding operations to realize functions such as API management, debugging optimization, script writing, and interactive cards. Through the model training module and the model management module, data management, model fine-tuning, distributed training, model evaluation, one-click deployment, model testing, version management, performance monitoring and other functions are provided to support users to train and optimize models according to their needs. Based on the big model capability, the agent module supports user-defined agent agents and provides functions such as execution chain, process orchestration, service publishing, agent agent management, and channel management. The platform layer implements big model services through a variety of service forms, including cloud assistants, APIs, H5, and JS.

[0096] The intelligent application layer is used to provide a variety of intelligent applications, including global AI assistant, AI assistant, cloud assistant robot, document generator, etc. These applications use the functions provided by the platform layer to provide intelligent services to end users and improve office efficiency and user experience.

[0097] The large model service platform also provides platform management functions, including authority control, operation statistics, security management, channel control, tenant management, and online network testing.

[0098] Specifically, the AI ​​big model service platform is based on different basic general AI big models, and through the Prompt prompt word engineering segmentation, it encapsulates different specific atomic AI capabilities. By adjusting the input to the model, it relies on the AI ​​big model to realize customized AI capabilities, and empowers upper-level product applications in the form of different channels. Upper-level product applications can directly apply the open capabilities of the AI ​​big model through the AI ​​big model service platform, and build intelligent applications on this basis. At the same time, the intelligent capabilities of the big model are combined with technologies such as Embedding vectorization to greatly improve the processing efficiency of traditional types of knowledge structures such as documents and knowledge bases. It supports background upload and management of multiple types of documents, and one-click import of documents to automatically parse questions and answers, which greatly reduces the requirements for building a knowledge base, easily realizes intelligent questions and answers of own knowledge, and effectively reduces the workload of manual replies. In addition, the big model service plug-in capability allows the big model to access the latest information, run calculations, or use third-party services, so that the big model service can obtain relevant knowledge or complete task execution through external APIs, open standard plug-in development frameworks, support various product applications to access in accordance with standards, and expand the capabilities of the big model. Furthermore, we build a large model Agent intelligent body, support the open use of a single capability or a combination of multiple capabilities (including RAG knowledge base and plug-ins), and realize the openness of Agent intelligent body capabilities through a standardized service release process to meet the needs of different business scenarios. In addition, we jointly build an open ecosystem for large model applications through the AI ​​large model service platform, and have now realized the implementation of large model application scenarios in different fields such as content creation, collaborative office, sales operation, intelligent underwriting, development and operation and maintenance.

[0099] The AI ​​big model service platform has completed the integrated access of a variety of basic big language models with different parameter quantities, and unified management through the platform layer, providing rich big model basic resources upward, supporting users to freely select and switch the underlying basic big models on the AI ​​big model service platform, and adapting to various types of big model application scenarios in various fields. The AI ​​big model service platform is based on a diverse big model foundation, encapsulating prompt word engineering, RAG retrieval enhancement, plug-ins and other big model capabilities, supporting users to customize their own model capabilities through the platform, abstracting the complex technical details of the big model, and lowering the threshold for docking and use by providing concise and clear interfaces and tools, helping to quickly implement big model application scenarios. The AI ​​big model service platform supports opening up the empowerment of big model capabilities through a variety of different channels such as interface calls, JS code embedding, robot assistants, etc., supporting the company's various product teams to dock and integrate big model capabilities according to their own personalized needs, and integrating big model artificial intelligence capabilities into various technology products. Building an AI big model service platform, abstracting the complex technical details of big models, lowering the threshold for use by providing concise and clear interfaces and tools, promoting the application and implementation of big model capabilities, and at the same time attracting more developers to join by building an open ecosystem for big models, forming a prosperous ecosystem to meet the needs of different business scenarios. The survey found that the industry's leading companies have formed a trend of model service platformization.

[0100] Embodiment 2

[0101] See also Figure 4 As shown, Figure 4 A flowchart of a large model service method provided by the second embodiment of the present invention is shown, wherein the method is applied to a large model service system, the system includes a prompt word engineering module, a search enhancement generation module, a plug-in development module and an intelligent agent module, and the method includes steps S401 to S404:

[0102] S401: the prompt word engineering module configures a plurality of prompt word templates and displays them to a target user; in response to the target user selecting a first prompt word from the plurality of prompt word templates, generates a first target text based on the first prompt word;

[0103] S402: The retrieval enhancement generation module receives the basic text uploaded by the target user, converts the basic text into a number of high-dimensional word vectors, and stores each high-dimensional word vector in a vector database to form a retrieval database; receives the question information input by the target user, retrieves a second prompt word from the retrieval database according to the question information, and generates a second target text based on the second prompt word;

[0104] S403: the plug-in development module receives the configuration file and question information input by the target user, parses the configuration file, and obtains detailed information of the third-party API interface based on the parsing result; determines whether the third-party API interface needs to be called based on the question information, and when the third-party API interface needs to be called, calls the extended data from the third-party API interface; retrieves a third prompt word from the extended data based on the question information, and generates a third target text based on the third prompt word;

[0105] S404: The intelligent agent module receives the target user's custom operation on the system execution chain, and defines the execution combination and / or execution order of the prompt word engineering module, the retrieval enhancement generation module and the plug-in development module based on the custom operation.

[0106] In a feasible implementation manner, the generating the first target text based on the first prompt word includes:

[0107] Using the target macro model, generating the first target text according to the first prompt word;

[0108] The generating a second target text based on the second prompt word includes:

[0109] Using the target macro model, generating the second target text according to the second prompt word;

[0110] The step of generating a third target text based on the third prompt word includes:

[0111] The target macro model is used to generate the third target text according to the third prompt word.

[0112] In a feasible implementation scheme, the system further includes a model training module and a model management module, and the method further includes:

[0113] The model training module performs model training on the initial large model;

[0114] The model management module tests the trained initial large model and deploys the initial large model that passes the test as the target large model.

[0115] In a feasible implementation scheme, the converting the basic text into a number of high-dimensional word vectors includes:

[0116] The basic text is parsed through an embedding model to obtain a number of high-dimensional word vectors, wherein the embedding model includes an M3E model, a GTE model, and a BGE model.

[0117] In a feasible implementation manner, the target macro model includes a GLM model and a Qwen model.

[0118] In a feasible implementation manner, the retrieving the second prompt word from the search database according to the question information includes:

[0119] Performing text vectorization on the question information to obtain an initial word vector;

[0120] Calculating the similarity between the initial word vector and each high-dimensional word vector in the retrieval database respectively;

[0121] A high-dimensional word vector in the retrieval database whose similarity with the initial word vector exceeds a preset threshold is determined as the second prompt word.

[0122] Embodiment 3

[0123] Based on the same application concept, see Figure 5 As shown, Figure 5 FIG. 4 shows a schematic diagram of the structure of a computer device provided by Embodiment 3 of the present invention, wherein: Figure 5 As shown, a computer device 500 provided in Embodiment 3 of the present application includes:

[0124] A processor 501, a memory 502 and a bus 503, wherein the memory 502 stores machine-readable instructions executable by the processor 501. When the computer device 500 is running, the processor 501 communicates with the memory 502 via the bus 503. When the processor 501 is running, the machine-readable instructions execute the steps of a large model service method shown in the above-mentioned embodiment 2.

[0125] Embodiment 4

[0126] Based on the same application concept, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a large model service method described in any one of the above-mentioned embodiments 2 are executed.

[0127] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the method described above can refer to the corresponding process in the aforementioned system embodiment, and will not be repeated here.

[0128] The computer program product for performing large model services provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0129] A large model service system provided by an embodiment of the present invention can be specific hardware on a device or software or firmware installed on a device. The system provided by an embodiment of the present invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment. Technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, the corresponding processes of the aforementioned methods can all refer to the specific working processes in the aforementioned system embodiment, and will not be repeated here.

[0130] In the embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some communication interface, device or unit, which can be electrical, mechanical or other forms.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] In addition, each functional unit in the embodiment provided by the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0133] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0134] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0135] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or perform equivalent replacements on some of the technical features thereof; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A large model service system, characterized in that: The system includes a prompt word engineering module, a search enhancement generation module, a plug-in development module and an intelligent agent module; The prompt word engineering module is used to configure a plurality of prompt word templates and display them to a target user; and is also used to generate a first target text based on a first prompt word in the plurality of prompt word templates in response to a selection operation of the target user on the first prompt word; The retrieval enhancement generation module is used to receive the basic text uploaded by the target user, convert the basic text into a number of high-dimensional word vectors, and store each high-dimensional word vector in a vector database to form a retrieval database; it is also used to receive the question information input by the target user, retrieve a second prompt word from the retrieval database according to the question information, and generate a second target text based on the second prompt word; The plug-in development module is used to receive the configuration file and question information input by the target user, parse the configuration file, and obtain detailed information of the third-party API interface based on the parsing result; it is also used to determine whether the third-party API interface needs to be called based on the question information, and when the third-party API interface needs to be called, call the extended data from the third-party API interface; it is also used to retrieve a third prompt word from the extended data based on the question information, and generate a third target text based on the third prompt word; The intelligent agent module is used to receive the target user's custom operations on the system execution chain, and define the execution combination and / or execution order of the prompt word engineering module, the retrieval enhancement generation module and the plug-in development module based on the custom operations.

2. The system according to claim 1, characterized in that The generating a first target text based on the first prompt word includes: Using the target macro model, generating the first target text according to the first prompt word; The generating a second target text based on the second prompt word includes: Using the target macro model, generating the second target text according to the second prompt word; The step of generating a third target text based on the third prompt word includes: The target macro model is used to generate the third target text according to the third prompt word.

3. The system according to claim 2, characterized in that The system also includes a model training module and a model management module; The model training module is used to perform model training on the initial large model; The model management module is used to test the trained initial large model and deploy the initial large model that passes the test as the target large model.

4. The system according to claim 1, characterized in that The step of converting the basic text into a number of high-dimensional word vectors includes: The basic text is parsed through an embedding model to obtain a number of high-dimensional word vectors, wherein the embedding model includes an M3E model, a GTE model, and a BGE model.

5. The system according to claim 2, characterized in that The target large model includes GLM model and Qwen model.

6. The system according to claim 1, characterized in that The step of retrieving a second prompt word from the search database according to the question information includes: Performing text vectorization on the question information to obtain an initial word vector; Calculating the similarity between the initial word vector and each high-dimensional word vector in the retrieval database respectively; A high-dimensional word vector in the retrieval database whose similarity with the initial word vector exceeds a preset threshold is determined as the second prompt word.

7. A large model service method, characterized in that: Applied to a large model service system, the system includes a prompt word engineering module, a search enhancement generation module, a plug-in development module and an intelligent agent module, and the method also includes: The prompt word engineering module configures a plurality of prompt word templates and displays them to a target user; and is further configured to generate a first target text based on a first prompt word in the plurality of prompt word templates in response to a selection operation of the target user on the first prompt word; The retrieval enhancement generation module receives the basic text uploaded by the target user, converts the basic text into a number of high-dimensional word vectors, and stores each high-dimensional word vector in a vector database to form a retrieval database; it is also used to receive question information input by the target user, retrieve a second prompt word from the retrieval database according to the question information, and generate a second target text based on the second prompt word; The plug-in development module receives the configuration file and question information input by the target user, parses the configuration file, and obtains detailed information of the third-party API interface based on the parsing result; determines whether the third-party API interface needs to be called based on the question information, and when the third-party API interface needs to be called, calls the extended data from the third-party API interface; retrieves a third prompt word from the extended data based on the question information, and generates a third target text based on the third prompt word; The intelligent agent module receives the target user's custom operation on the system execution chain, and defines the execution combination and / or execution order of the prompt word engineering module, the retrieval enhancement generation module and the plug-in development module based on the custom operation.

8. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate through the bus. When the machine-readable instructions are executed by the processor, the steps of the large model service method as described in claim 7 are performed.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the large model service method as described in claim 7.

Images