Large language model development platform

Abstract

The invention belongs to the field of large language model development, and particularly relates to a large language model development platform which comprises application management, data set management, tool management, a model square and a dialogue platform. The application management is used for managing a plurality of applications and defining configuration of the applications, including overview, planning, roles, scenes, output, memory and access; the data set management is used for managing a plurality of data sets and documents, and comprises data set index setting, recall setting, document management, hit debugging and metadata maintenance; the tool management is used for managing a plurality of external tools, including a large model tool and an AP I tool; the model square is used for debugging, verifying and comparing the answer effect and response time of each large model; the dialogue platform is used for configuring the application, developers can be helped to quickly construct the business-level AI native application, the platform can automatically process vigor and accumulator, and the developers only need to concentrate on innovation, solve problems for individuals and improve efficiency for enterprises.

Description

Technical Field

[0001] The present invention relates to the technical field of large language model development, and in particular to a large language model development platform. Background Art

[0002] Large language models are deep learning models trained using large amounts of text data, enabling them to generate natural language text or understand the meaning of language text. These models can provide in-depth knowledge and language production on a variety of topics by training on large datasets. The core idea is to learn the patterns and structure of natural language through large-scale unsupervised training, simulating the human language cognition and generation process to a certain extent.

[0003] In the existing technology, there is a lack of a platform for quickly developing large language models, and the development efficiency of large language models is low. Therefore, we propose a large language model development platform to solve the above problems. Summary of the Invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art, such as the lack of a platform for rapidly developing large language models and the low efficiency of large language model development, by proposing a large language model development platform.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A large language model development platform, including application management, dataset management, tool management, model platform, and dialogue platform:

[0007] Application management is used to manage multiple applications and define application configurations, including: profile, planning, roles, scenarios, output, memory, and access;

[0008] Dataset management is used to manage multiple datasets and documents, including: dataset index settings, recall settings, document management, hit debugging, and metadata maintenance;

[0009] Tool management is used to manage multiple external tools, including large model tools and API tools;

[0010] The Model Square is used to debug, verify, and compare the answer effects and response times of various models;

[0011] The dialogue platform is used to configure applications and provide a web-based dialogue implementation for easy preview and questioning.

[0012] Preferably, the summary includes an application list and an application configuration. The application list manages the addition, deletion, modification and query of applications. The application configuration creates an application, which requires an application avatar, name and description. A welcome message and guiding questions provide guidance to users. The guiding questions configure the questions that users most want to try.

[0013] Preferably, the roles include:

[0014] Define the core prompt words for your application. If you are unsure how to fill in the prompt words, use the prompt word template. The prompt word template includes:

[0015] Role prompts: Define the role played by the application, describing the work that this role can complete, the conversation style, and the temperament and personality;

[0016] Command prompt words: define the key commands for the role to serve users.

[0017] Preferably, the scenario includes:

[0018] Create one or more scenarios. Each scenario corresponds to using a certain tool to complete a certain type of user intention. The execution of the scenario includes the following modes:

[0019] Precision mode: hits a scenario based on user input and returns the result immediately; suitable for scenarios with a single result requirement;

[0020] Composite mode: hits one or more scenarios based on user input, and returns the result after integrating the results of multiple scenarios; suitable for scenarios that require comprehensive results;

[0021] Parallel mode: Forces all scenarios to be hit simultaneously based on user input, and returns the result after combining the results of multiple scenarios; suitable for complex scenarios that require multiple operation scenarios to be executed once.

[0022] Preferably, the index setting is used to set the index-related parameters of the data set, including: related file parsing method, setting whether to enable metadata parsing, whether to generate a full-text summary, whether to generate an intelligent index, and whether to generate question-answer pairs. The recall setting is used to set the relevant parameters of the data set during the recall process, including: recall threshold, recall quantity, whether to perform entity filtering, and whether to perform time period filtering.

[0023] Preferably, the API tool can encapsulate an external API call in the simplest way and expose it in the form of a RESTful API. Any external API that can be accessed can be implemented through the API tool and provided to the application for use. The name, description, prompt words, input parameters and model are configured through the interface. A debugging tool is also provided, and debugging can be performed directly through input parameters in the interface.

[0024] Preferably, the model debugging includes: preparing a clean data set related to the target task to ensure coverage of diverse scenarios and potential input types, setting hyperparameters and conducting small-scale pre-training to verify model convergence, adjusting the number of Transformer layers and attention head structures according to task requirements, optimizing the mask strategy or generating targets in the pre-training phase through self-supervised learning, verifying input and output format compatibility, monitoring GPU / CPU utilization during training, and optimizing distributed training strategies to improve efficiency.

[0025] Preferably, the response time test includes: measuring single inference latency, recording GPU memory usage, using the tool JMeter to simulate high-concurrency requests, evaluating throughput limits, comparing the impact of different hardware configurations on response time, and compressing the model size to reduce deployment costs.

[0026] Preferably, the model answer effect test specifically includes automated benchmark testing, adversarial testing, and scenario-based manual evaluation; the automated benchmark test includes using standardized data sets for batch input, statistical accuracy and F1 value; and monitoring the model's prediction confidence for professional terms through perplexity.

[0027] Adversarial testing involves implanting semantic interference items to test the model's error correction capabilities; constructing knowledge conflict scenarios to observe the self-consistency of the answers;

[0028] Scenario-based manual evaluation includes a multi-round dialogue stress test: after 10 consecutive rounds of questions, irrelevant topics are inserted to verify context retention;

[0029] Cross-language proficiency verification: Mixed Chinese and English questions to assess multilingual collaborative comprehension ability.

[0030] Preferably, the dialogue platform uses Docker to package model services and dependent environments, implements automatic scaling through Kubernetes to cope with traffic fluctuations, connects to Prometheus to monitor API response time and error rate, sets up a log analysis system, and captures abnormal dialogue samples for iterative optimization.

[0031] In the present invention, the large language model development platform has the following beneficial effects:

[0032] Application management is used to manage multiple applications and define application configurations, including overview, planning, roles, scenarios, output, memory, and access. Dataset management is used to manage multiple datasets and documents, including dataset index settings, recall settings, document management, hit debugging, and metadata maintenance. Tool management is used to manage multiple external tools, including large model tools and API tools. The Model Plaza is used to debug, verify, and compare the answer effects and response times of various models. The dialogue platform is used to provide a web-based dialogue implementation based on the configured application, facilitating preview and asking questions.

[0033] This invention can help developers quickly build commercial-grade AI native applications. The platform will automatically handle the dirty work and developers only need to focus on innovation, solving problems for individuals and improving efficiency for enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A block diagram of a large language model development platform proposed by the present invention;

[0035] Figure 2 A block diagram of application management of a large language model development platform proposed by the present invention;

[0036] Figure 3 A block diagram of data set management for a large language model development platform proposed in the present invention;

[0037] Figure 4 A block diagram of tool management for a large language model development platform proposed by the present invention;

[0038] Figure 5 A block diagram of the model square of a large language model development platform proposed in the present invention. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0040] Example 1

[0041] Reference Figure 1-Figure 5 , a large language model development platform, including application management, dataset management, tool management, model square and dialogue platform:

[0042] Application management is used to manage multiple applications and define application configurations, including: profile, planning, roles, scenarios, output, memory, and access;

[0043] Dataset management is used to manage multiple datasets and documents, including: dataset index settings, recall settings, document management, hit debugging, and metadata maintenance;

[0044] Tool management is used to manage multiple external tools, including large model tools and API tools;

[0045] The Model Square is used to debug, verify, and compare the answer effects and response times of various models;

[0046] The dialogue platform is used to configure applications and provide a web-based dialogue implementation for easy preview and questioning.

[0047] In this embodiment, the summary includes an application list and an application configuration. The application list manages the addition, deletion, modification, and query of applications. The application configuration creates an application, which requires an avatar, name, and description of the application. A welcome message and guiding questions provide guidance to users. The guiding questions configure the questions that users most want to try.

[0048] In this embodiment, the roles include:

[0049] Define the core prompt words for your application. If you are unsure how to fill in the prompt words, use the prompt word template. The prompt word template includes:

[0050] Role prompts: Define the role played by the application, describing the work that this role can complete, the conversation style, and the temperament and personality;

[0051] Command prompt words: define the key commands for the role to serve users.

[0052] In this embodiment, the scenarios include:

[0053] Create one or more scenarios. Each scenario corresponds to using a certain tool to complete a certain type of user intention. The execution of the scenario includes the following modes:

[0054] Precision mode: hits a scenario based on user input and returns the result immediately; suitable for scenarios with a single result requirement;

[0055] Composite mode: hits one or more scenarios based on user input, and returns the result after integrating the results of multiple scenarios; suitable for scenarios that require comprehensive results;

[0056] Parallel mode: Forces all scenarios to be hit simultaneously based on user input, and returns the result after combining the results of multiple scenarios; suitable for complex scenarios that require multiple operation scenarios to be executed once.

[0057] In this embodiment, the index setting is used to set the index-related parameters of the data set, including: the relevant file parsing method, whether to enable metadata parsing, whether to generate a full-text summary, whether to generate an intelligent index, and whether to generate question-answer pairs. The recall setting is used to set the relevant parameters of the data set during the recall process, including: the recall threshold, the number of recalls, whether to perform entity filtering, and whether to perform time period filtering.

[0058] In this embodiment, the API tool can encapsulate an external API call in the simplest way and expose it in the form of a RESTful API. Any external API that can be accessed can be implemented through the API tool and provided to the application for use. The name, description, prompt words, input parameters and model are configured through the interface. A debugging tool is also provided, and debugging can be performed directly through input parameters on the interface.

[0059] In this embodiment, model debugging includes: preparing a clean data set related to the target task to ensure coverage of diverse scenarios and potential input types, setting hyperparameters and performing small-scale pre-training to verify model convergence, adjusting the number of Transformer layers and attention head structures according to task requirements, optimizing the mask strategy or generating targets in the pre-training phase through self-supervised learning, verifying input and output format compatibility, monitoring GPU / CPU utilization during training, and optimizing distributed training strategies to improve efficiency.

[0060] In this embodiment, the response time test includes: measuring the delay of a single inference, recording the GPU memory usage, using the tool JMeter to simulate high-concurrency requests, evaluating the throughput limit, comparing the impact of different hardware configurations on response time, and compressing the model size to reduce deployment costs.

[0061] In this embodiment, the model answer effectiveness test specifically includes automated benchmark testing, adversarial testing, and scenario-based manual evaluation; the automated benchmark test includes using standardized data sets for batch input, statistical accuracy and F1 value; and the perplexity monitoring model's prediction confidence for professional terms.

[0062] Adversarial testing involves implanting semantic interference items to test the model's error correction capabilities; constructing knowledge conflict scenarios to observe the self-consistency of the answers;

[0063] Scenario-based manual evaluation includes a multi-round dialogue stress test: after 10 consecutive rounds of questions, irrelevant topics are inserted to verify context retention;

[0064] Cross-language proficiency verification: Mixed Chinese and English questions to assess multilingual collaborative comprehension ability.

[0065] In this embodiment, the conversation platform uses Docker to package the model service and dependent environment, implements automatic scaling through Kubernetes to cope with traffic fluctuations, connects to Prometheus to monitor API response time and error rate, sets up a log analysis system, and captures abnormal conversation samples for iterative optimization.

[0066] Example 2

[0067] The difference between this embodiment and the first embodiment is that:

[0068] Including application management, dataset management, tool management, model square and dialogue platform:

[0069] Application management is used to manage multiple applications and define application configurations, including: profile, planning, roles, scenarios, output, memory, and access;

[0070] Dataset management is used to manage multiple datasets and documents, including: dataset index settings, recall settings, document management, hit debugging, and metadata maintenance;

[0071] Tool management is used to manage multiple external tools, including large model tools and API tools;

[0072] The Model Plaza is used to debug, verify, and compare the answer effects and response time of various models. The Model Plaza also includes model architecture optimization, specifically: using parameter quantization to compress the model volume by 4 times, increasing the inference speed by 2-3 times, implementing structural pruning to remove up to 50% of redundant parameters, setting a maximum output length constraint of 50-100 tokens, shortening the generation time, adjusting temperature parameters to reduce randomness, and reducing invalid generation rounds.

[0073] The dialogue platform is used to configure applications and provide a web-based dialogue implementation for easy preview and questioning.

[0074] The rest is the same as that of the first embodiment.

[0075] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A large language model development platform, characterized by: Including application management, dataset management, tool management, model square and dialogue platform: Application management is used to manage multiple applications and define application configurations, including: profile, planning, roles, scenarios, output, memory, and access; Dataset management is used to manage multiple datasets and documents, including: dataset index settings, recall settings, document management, hit debugging, and metadata maintenance; Tool management is used to manage multiple external tools, including large model tools and API tools; The Model Square is used to debug, verify, and compare the answer effects and response times of various models; The dialogue platform is used to configure applications and provide a web-based dialogue implementation for easy preview and questioning.

2. A large language model development platform according to claim 1, characterized in that: The summary includes an application list and application configuration. The application list manages the addition, deletion, modification and query of applications. The application configuration creates an application, which requires an application avatar, name and description. A welcome message and guiding questions provide guidance to users. The guiding questions configure the questions that users most want to try.

3. A large language model development platform according to claim 2, characterized in that: The roles include: Define the core prompt words for your application. If you are unsure how to fill in the prompt words, use the prompt word template. The prompt word template includes: Role prompts: Define the role played by the application, describing the work that this role can complete, the conversation style, and the temperament and personality; Command prompt words: define the key commands for the role to serve users.

4. A large language model development platform according to claim 3, characterized in that: The scenarios include: Create one or more scenarios. Each scenario corresponds to using a certain tool to complete a certain type of user intention. The execution of the scenario includes the following modes: Precision mode: hits a scenario based on user input and returns the result immediately; suitable for scenarios with a single result requirement; Composite mode: hits one or more scenarios based on user input, and returns the result after integrating the results of multiple scenarios; suitable for scenarios that require comprehensive results; Parallel mode: Forces all scenarios to be hit simultaneously based on user input, and returns the result after combining the results of multiple scenarios; suitable for complex scenarios that require multiple operation scenarios to be executed once.

5. A large language model development platform according to claim 4, characterized in that: The index settings are used to set the index-related parameters of the dataset, including: related file parsing method, whether to enable metadata parsing, whether to generate full-text abstracts, whether to generate intelligent indexes, and whether to generate question-answer pairs. The recall settings are used to set the relevant parameters of the dataset during the recall process, including: recall threshold, recall quantity, whether to perform entity filtering, and whether to perform time period filtering.

6. A large language model development platform according to claim 5, characterized in that: The API tool can encapsulate an external API call in the simplest way and expose it in the form of a RESTful API. Any external API that can be accessed can be implemented through the API tool and provided to the application for use. The name, description, prompt words, input parameters and model can be configured through the interface. A debugging tool is also provided, which allows debugging to be performed directly through input parameters on the interface.

7. A large language model development platform according to claim 6, characterized in that: The model debugging includes: preparing a clean dataset related to the target task to ensure coverage of diverse scenarios and potential input types, setting hyperparameters and conducting small-scale pre-training to verify model convergence, adjusting the number of Transformer layers and attention head structures according to task requirements, optimizing the mask strategy or generating targets in the pre-training phase through self-supervised learning, verifying input and output format compatibility, monitoring GPU / CPU utilization during training, and optimizing distributed training strategies to improve efficiency.

8. A large language model development platform according to claim 7, characterized in that: The response time test includes: measuring single inference latency, recording GPU memory usage, using the JMeter tool to simulate high-concurrency requests, evaluating throughput limits, comparing the impact of different hardware configurations on response time, and compressing the model size to reduce deployment costs.

9. A large language model development platform according to claim 8, characterized in that: The model answer effectiveness test specifically includes automated benchmark testing, adversarial testing, and scenario-based manual evaluation; Automated benchmarking includes using standardized data sets for batch input and statistical accuracy and F1 values; Monitor the model's confidence in its predictions for specialized terms through perplexity; Adversarial testing involves implanting semantic interference items to test the model's error correction capabilities; Construct knowledge conflict scenarios and observe the consistency of answers; Scenario-based manual evaluation includes a multi-round dialogue stress test: after 10 consecutive rounds of questions, irrelevant topics are inserted to verify context retention; Cross-language proficiency verification: Mixed Chinese and English questions to assess multilingual collaborative comprehension ability.

10. A large language model development platform according to claim 9, characterized in that: The dialogue platform uses Docker to package model services and dependent environments, implements automatic scaling through Kubernetes to cope with traffic fluctuations, connects to Prometheus to monitor API response time and error rate, and sets up a log analysis system to capture abnormal dialogue samples for iterative optimization.

Images