Large model cue word intelligent design and verification system and method supporting multi-person cooperation

Abstract

The invention discloses a large model cue word intelligent design and verification system supporting multi-person cooperation, which designs cue words in a manner of cue word project management and multi-person cooperation to realize specialized design of the cue words. The difficulty of cue word design is simplified by using AI aided design; the operation stability and accuracy of cue words in different models and different computing power are verified through single-point testing and batch testing, so that cue word verification and iteration form a closed loop, and the cue word quality is improved; after the cue word is published, directly outputting an interface with an access key for external calling, and storing the interface as a cue word case into a cue word knowledge base; compared with an existing system and technology, the cue word version management difficulty is effectively reduced, the workload of cue word designers is reduced, the cue word quality is improved, and the risk of cue word asset loss is reduced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a large-scale model prompt word intelligent design and verification system that supports multi-person collaboration. Background Technology

[0002] Current prompt design tools face numerous problems, which seriously affect the development and application of prompts.

[0003] In terms of multi-person collaboration, the lack of effective collaboration mechanisms, such as permission management, real-time collaboration, and review processes, leads to frequent conflicts, unclear responsibilities, and low efficiency when multiple people are editing prompts. Regarding version management, the lack of a standardized version control system makes it difficult to track historical modifications of prompts, compare differences, roll back errors, or establish baselines, resulting in a chaotic and untraceable iteration process. Adapting to large models is costly; optimizing or migrating prompts for different LLMs (Large Language Models) requires a significant amount of repetitive work, such as adjusting formats, parameters, and examples, significantly increasing the development cost and time for cross-model applications. Prompt design is inefficient; existing tools or processes fail to effectively support the rapid construction, testing, reuse, and combination of prompts, relying on inefficient manual operations and hindering overall development progress. The lack of a closed loop for verification and iteration, the absence of a systematic method for quantitatively evaluating prompt effectiveness, diagnosing problems, and conducting rapid iterative optimization based on feedback, leads to a blind and inefficient optimization process. Furthermore, there is a risk of knowledge asset loss; after project completion, valuable prompts and their iterative experience are not effectively preserved, organized, and shared to form a team knowledge base, resulting in knowledge silos and reinventing the wheel. Summary of the Invention

[0004] In view of the aforementioned shortcomings of existing technologies, this invention aims to solve the problems of chaotic prompt word versions, low prompt word design efficiency, and lack of verification and iteration closed loops, thereby significantly improving prompt word development efficiency and deployment speed. To achieve the above objective, this invention provides a large-scale intelligent prompt word design and verification system supporting multi-user collaboration, comprising: The prompt word project management module is used to organize projects as the core unit. It supports the creation, editing, and deletion of projects. A single project can contain multiple independent prompt word design tasks. It provides project collaboration and permission management, branched collaborative editing, version control and trunk integration, conflict detection and merging, and version rollback functions. The prompt word design and verification module is used to support prompt word designers in selecting different models, computing cards and model parameters during the design process. It provides AI-assisted evaluation functions, generates optimization suggestions, and can generate improved prompt words with one click based on the suggestions. It supports calling the model for testing at any time, and using the test set to call the model in batches to view the accuracy and latency after the prompt words are finalized. The prompt word publishing module is used to allow designers to publish, and then review it. Once approved, it automatically generates an API interface with an access key for external systems to call. The prompt words are also stored in the system knowledge base as historical cases and displayed in the prompt word square.

[0005] Furthermore, in the project collaboration and permission management function of the prompt word project management module, after a member creates a prompt word project, other members can apply for collaboration permissions and jointly edit the project after approval by the creator.

[0006] Furthermore, in the branched collaborative editing function of the prompt word project management module, for the same prompt word file, members can create personal branches for independent editing and version iteration, save operations within the branch to generate private historical versions, and record changes and differences.

[0007] Furthermore, the AI-assisted evaluation function called in the prompt word design and verification module integrates the DeepSeek analysis engine, which can intelligently evaluate prompt words and generate multi-dimensional scoring reports, and can generate improved versions of prompt words with one click based on suggestions.

[0008] Furthermore, the prompt word design and verification module supports access to multiple mainstream and domestic large language models, including but not limited to Xingchen and Qwen, and provides comparative testing of prompt word execution effects on domestic computing power platforms and NVIDIA computing power platforms; inter-model comparison testing allows users to easily switch between different models to verify the output effect of the same prompt word, intuitively evaluate the differences between solutions, and perform batch testing. After the prompt word is finalized, it supports large-scale batch testing to ensure stability and effectiveness.

[0009] Furthermore, the API interface with access keys generated by the prompt word publishing module can be securely called by external systems, and the published prompt words will be stored in the platform knowledge base as historical prompt word cases.

[0010] Based on the above technical solutions, a method for intelligent design and verification of large-scale prompt words that supports multi-user collaboration is also included, characterized by the following steps: Step 1: Manage projects through the prompt word project management module, create projects and assign collaboration permissions, members edit prompt words on their personal branches, and perform conflict detection and merging when submitting to the main branch, supporting version rollback; Step 2: In the prompt word design and verification module, designers select the model, computing card and model parameters, use the AI-assisted evaluation function to obtain optimization suggestions and generate improved versions, call the model for testing at any time, and use the test set to batch test and check the accuracy and latency after finalization. Step 3: Designers publish prompts through the prompt publishing module. After approval, an API interface with an access key is generated for external calls, and the prompts are stored in the knowledge base and displayed in the prompt square.

[0011] Furthermore, in step 1, when members create personal branches for editing, the system records private historical versions and changes.

[0012] Furthermore, in step 2, the AI-assisted evaluation function is implemented through the integrated DeepSeek analysis engine.

[0013] Furthermore, in step 2, batch testing ensures the stability and effectiveness of the prompts under different models and computing power.

[0014] The method also includes an iterative optimization approach for prompt words based on the DeepSeek tool, characterized by the following steps: Step 1: Designers submit initial prompt words to the execution model, which runs the prompt words on a test set and obtains the LLM output results; Step 2: The evaluation model receives the LLM output results and analyzes them, outputting an evaluation report containing quantitative indicators and defect localization, where quantitative indicators include task accuracy, response stability, and latency, and defect localization includes logical contradictions, omission of key information, and instruction ambiguity; Step 3: The optimization model receives the evaluation report and automatically reconstructs the initial prompt words based on the type of defect localization, including adding constraints, correcting ambiguous expressions, and generating multiple sets of optimization schemes; Step 4: Based on the number of iterations, a new optimization scheme is selected and input into the execution model for secondary verification, and the execution model outputs the secondary verification results; Step 5: The evaluation model performs a secondary scoring on the secondary verification results. If the score meets the standard, the iteration is completed; otherwise, steps 3 to 5 are repeated.

[0015] Furthermore, in the branched collaborative editing function of the prompt word project management module, for the same prompt word file, members can optimize multiple sets of optimization schemes generated by the optimization model in step three, with different reconstruction focuses, to adapt to the needs of different scenarios.

[0016] Furthermore, in the branched collaborative editing function of the prompt word project management module, for the same prompt word file, members can, in step four, set or adjust the number of iterations according to actual needs.

[0017] Furthermore, in the branched collaborative editing function of the prompt word project management module, for the same prompt word file, members can, in step five, evaluate the secondary scoring criteria of the model, which corresponds to the quantitative indicators in step two, and the threshold for achieving the target can be customized.

[0018] Furthermore, in the branched collaborative editing function of the prompt word project management module, for the same prompt word file, members can use the execution model in step one as a large language model that supports LLM operation.

[0019] Furthermore, in the branched collaborative editing function of the prompt word project management module, for the same prompt word file, members can use natural language processing technology in the analysis process of the LLM output results of the evaluation model in step two to realize the calculation of quantitative indicators and the identification of defect location.

[0020] This invention improves design efficiency: AI-assisted pipeline and multi-person collaboration mechanism shorten the development cycle of prompt words; iteration costs are reduced, and dual-mode verification mechanism reduces manpower investment in compatibility testing.

[0021] Commercial value transformation: API monetization, high-quality suggestion keywords can be converted into paid API interfaces within 1 hour, creating new revenue channels; asset appreciation, the reuse rate of enterprise suggestion keyword library is improved, and the cost of repeated development is reduced.

[0022] Technical risk control: Atomic version rollback is used to avoid production accidents caused by the release of error message words; cross-system compatibility testing is used to identify adaptation issues in advance and reduce the risk of model switching.

[0023] Industry Standards Advancement: This invention marks the first time a standardized pipeline for prompt word development has been implemented, providing an engineering paradigm for the industry; a quantitative evaluation system (accuracy / latency) establishes an industry benchmark for prompt word quality. The following will further explain the concept, specific structure, and resulting technical effects of this invention in conjunction with the accompanying drawings, to fully understand the purpose, features, and effects of this invention. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of a preferred embodiment of the present invention. Detailed Implementation

[0025] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0026] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0027] One embodiment of the present invention is a system that mainly includes a prompt word project management module, a prompt word design and verification module, and a prompt word publishing module.

[0028] Prompt word project management module: The prompt word design and verification module allows prompt word designers to choose different models, computing cards, and model parameters. It also features AI-assisted evaluation of the prompt words, integrating the DeepSeek analysis engine to generate optimization suggestions and generate improved prompt words with a single click. During the design process, the model can be tested at any time to check if the results meet expectations. After the prompt words are finalized, the model can be used in batches with a test set to check accuracy and latency. This module supports integration with various mainstream and domestic large-scale language models, such as Xingchen and QWen.

[0029] Prompt Message Publishing Module: After designers click "Publish," reviewers will approve the prompt messages. Once approved, an API interface with an access key will be automatically generated for external systems to call, and the prompt message will be stored in the system knowledge base as a historical prompt message case and displayed in the prompt message square.

[0030] This invention also provides a method for intelligent design and verification of large-scale model prompts that supports multi-user collaboration, comprising the following steps: Step 1: Manage projects through the prompt word project management module, create projects and assign collaboration permissions to members. Members edit prompt words in their personal branches. During the editing process, the system records private historical versions and changes. Members submit the edited version to the main branch. The system performs conflict detection. If there are conflicts, they are resolved with the help of visual tools and then merged. It supports backtracking on historical versions of personal branches and the main branch.

[0031] Step 2: In the prompt word design and verification module, designers select appropriate models, computing power cards, and model parameters. They utilize the AI-assisted evaluation function integrated with the DeepSeek analysis engine to obtain optimization suggestions and generate improved prompt words. Model testing can be called at any time during the design process. After finalization, batch testing is conducted using a test set to check accuracy and latency, ensuring the stability and effectiveness of prompt words under different models and computing power.

[0032] Step 3: Designers publish prompts through the prompt publishing module. After approval, an API interface with an access key is generated for secure calls by external systems. The prompts are also stored in the system knowledge base and displayed in the prompt plaza.

[0033] This invention has many beneficial effects: Improved design efficiency: AI-assisted pipelines and multi-person collaboration mechanisms shorten the development cycle of prompt words by more than 50%; reduced iteration costs: dual-mode verification mechanism reduces manpower input for compatibility testing by 70%.

[0034] Commercial value transformation: API monetization, high-quality suggestion keywords can be converted into paid API interfaces within 1 hour, creating new revenue channels; asset appreciation, the reuse rate of enterprise suggestion keyword library increases by 60%, reducing the cost of repeated development.

[0035] Technical risk control: Atomic version rollback is used to avoid production accidents caused by the release of error message words; cross-system compatibility testing is used to identify adaptation issues in advance and reduce the risk of model switching.

[0036] Industry Standards Advancement: For the first time, a standardized pipeline for prompt word development has been implemented, providing an engineering paradigm for the industry; a quantitative evaluation system (accuracy / latency) establishes an industry benchmark for prompt word quality. Detailed Implementation The present invention will be further described in detail below with reference to specific implementation scenarios.

[0037] Scenario 1: Prompt designers use DeepSeek as an auxiliary tool to design and optimize prompts. Designers submit initial prompts to the execution model and obtain LLM output results on the test set. The evaluation model analyzes the output results, including quantitative metrics such as task accuracy, response stability, and latency, as well as defect localization such as logical contradictions, missing key information, and ambiguous instructions. The optimization model receives the evaluation report and automatically reconstructs the prompts based on defect types, such as adding constraints, correcting ambiguous expressions, and generating multiple optimized solutions. Based on the number of iterations, the new solutions are used for secondary verification in the execution model, and then scored again by the evaluation model until the iteration is completed and meets the standards. Throughout this process, designers can view the effects at any time in the prompt design and verification module and continuously optimize the prompts.

[0038] Scenario 2: Collaborative Editing of Hints and Version Management. Member A creates a hint project containing multiple hint files. Member B requests collaboration permissions, which are approved by A, allowing both to edit the project together. For one hint file, Member B creates a personal branch for independent editing. During the editing process, the system records the member's private version history and changes. After completing the editing, Member B commits the version from their personal branch to the main branch. The system automatically compares the version with the latest version on the main branch. If conflicts are detected, the system provides a visual comparison tool to assist Member B in manually resolving conflicts and merging the changes. Other members can view the latest commits on the main branch. If problems are subsequently discovered, they can revert and reuse any historical version from their personal branch or the main branch.

[0039] In the education sector, this platform allows teachers to design prompts for different courses within educational institutions. Teachers can create projects, and team members can collaboratively design prompts. AI-assisted design tools help optimize the prompts, and a multi-environment validation sandbox ensures their effectiveness on various teaching devices (such as smart classroom equipment and online teaching platforms). Validated prompts are then published to a knowledge base for other teachers to reference and use, thereby improving teaching quality.

[0040] In this enterprise knowledge management implementation example, a large amount of knowledge within an enterprise needs to be retrieved and utilized using prompts. This platform can be used to design prompts related to the enterprise knowledge base. Employees from different departments collaborate, leveraging AI-assisted design to improve the quality of prompts. Multi-environment verification ensures the applicability of prompts across different internal systems (such as office systems, knowledge management systems, etc.). Published prompts are stored in the knowledge base, improving the efficiency and accuracy of enterprise knowledge management.

[0041] In this research team implementation, when developing new algorithms, the team needs prompts to guide training. Team members collaborate on designing prompts on the platform, AI assists in the design and optimization of these prompts, and a multi-environment validation sandbox ensures the effectiveness of the prompts on different computing resources and datasets. The validated prompts are then published to a knowledge base, providing a reference for subsequent algorithm research and optimization, thus accelerating the research process.

[0042] The purpose of this invention is to address the technical limitations of existing prompt design tools, namely, the lack of a unified system or project for managing prompts, resulting in numerous and complex versions; weak compatibility between prompts and large models, low design efficiency, low prompt quality, lack of closed-loop testing, verification and iteration, and the failure to form a team prompt knowledge base with excellent prompts.

[0043] This invention proposes an intelligent design and verification system for large-scale model prompts that supports multi-user collaboration. It adopts a prompt project management approach, enabling collaborative prompt design to achieve professional prompt design. AI-assisted design simplifies the prompt design process. Single-point and batch testing verify the stability and accuracy of prompts across different models and computing power levels, creating a closed loop for prompt verification and iteration, thus improving prompt quality. After a prompt is released, an interface with an access key is directly output for external use, and the prompt is stored as a case study in the prompt knowledge base. Compared to existing systems and technologies, this system effectively reduces the difficulty of prompt version management, decreases the workload of prompt designers, improves prompt quality, and reduces the risk of prompt asset loss.

[0044] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A large-scale model prompt word intelligent design and verification system supporting multi-user collaboration, characterized in that, include: The prompt word project management module is used to organize projects as the core unit. It supports the creation, editing, and deletion of projects. A single project can contain multiple independent prompt word design tasks. It provides project collaboration and permission management, branched collaborative editing, version control and trunk integration, conflict detection and merging, and version rollback functions. The prompt word design and verification module is used to support prompt word designers in selecting different models, computing cards and model parameters during the design process. It provides AI-assisted evaluation functions, generates optimization suggestions, and can generate improved prompt words with one click based on the suggestions. It supports calling the model for testing at any time, and using the test set to call the model in batches to view the accuracy and latency after the prompt words are finalized. The prompt word publishing module is used to allow designers to publish, and then review it. Once approved, it automatically generates an API interface with an access key for external systems to call. The prompt words are also stored in the system knowledge base as historical cases and displayed in the prompt word square.

2. The system according to claim 1, characterized in that, In the project collaboration and permission management function of the prompt word project management module, after a member creates a prompt word project, other members can apply for collaboration permissions and jointly edit the project after approval by the creator.

3. The system according to claim 1, characterized in that, In the branched collaborative editing function of the prompt word project management module, members can create personal branches for independent editing and version iteration of the same prompt word file. Within the branch, operations are saved to generate private historical versions and changes are recorded.

4. The system according to claim 1, characterized in that, The prompt word design and verification module integrates the DeepSeek analysis engine into its AI-assisted evaluation function, which can intelligently evaluate prompt words and generate multi-dimensional scoring reports. It can also generate improved prompt words with one click based on suggestions.

5. The system according to claim 1, characterized in that, The prompt word design and verification module supports access to multiple mainstream and domestic large language models, including but not limited to Xingchen and Qwen, and provides comparative testing of prompt word execution effects on domestic computing power platforms and NVIDIA computing power platforms; inter-model comparison testing allows users to easily switch between different models to verify the output effect of the same prompt word, intuitively evaluate the differences between solutions, and perform batch testing. After the prompt word is finalized, it supports large-scale batch testing to ensure stability and effectiveness.

6. The system according to claim 1, characterized in that, The API interface with access key generated by the prompt word publishing module can be securely called by external systems, and the published prompt words will be stored in the platform knowledge base as historical prompt word cases.

7. A method for intelligent design and verification of prompt words for large models supporting multi-user collaboration, characterized in that, Includes the following steps: Step 1: Manage projects through the prompt word project management module, create projects and assign collaboration permissions, members edit prompt words on their personal branches, and perform conflict detection and merging when submitting to the main branch, supporting version rollback; Step 2: In the prompt word design and verification module, designers select the model, computing card and model parameters, use the AI-assisted evaluation function to obtain optimization suggestions and generate improved versions, call the model for testing at any time, and use the test set to batch test and check the accuracy and latency after finalization. Step 3: Designers publish prompts through the prompt publishing module. After approval, an API interface with an access key is generated for external calls, and the prompts are stored in the knowledge base and displayed in the prompt square.

8. The method according to claim 7, characterized in that, In step 1, when members create personal branches for editing, the system records private historical versions and changes.

9. The method according to claim 7, characterized in that, In step 2, the AI-assisted evaluation function is implemented through the integrated DeepSeek analysis engine.

10. The method according to claim 7, characterized in that, In step 3, batch testing ensures the stability and effectiveness of the prompts under different models and computing power.

11. An iterative method for optimizing prompt words based on the DeepSeek auxiliary tool, characterized in that, Includes the following steps: Step 1: Designers submit initial prompts to the execution model, which then runs the prompts on the test set and obtains the LLM output. Step 2: The evaluation model receives and analyzes the LLM output results, and outputs an evaluation report containing quantitative indicators and defect localization. The quantitative indicators include task accuracy, response stability, and latency, while the defect localization includes logical contradictions, omission of key information, and ambiguous instructions. Step 3: The optimization model receives the evaluation report and automatically reconstructs the initial prompts based on the type of defect localization. The reconstruction methods include adding constraints, correcting ambiguous expressions, and generating multiple sets of optimization schemes. Step 4: Based on the number of iterations, a new optimization scheme is selected and input into the execution model for secondary verification. The execution model outputs the secondary verification results. Step 5: The evaluation model performs a secondary scoring on the secondary verification results. If the score meets the standard, the iteration is completed. If it does not meet the standard, steps 3 to 5 are repeated.

12. The method according to claim 11, characterized in that, In step three, the multiple optimization schemes generated by the optimization model have different reconstruction focuses to adapt to the needs of different scenarios.

13. The method according to claim 11, characterized in that, In step four, the number of iterations can be set or adjusted by the designer according to actual needs.

14. The method according to claim 11, characterized in that, In step five, the secondary scoring criteria for the evaluation model correspond to the quantitative indicators in step two, and the threshold for achieving the target can be customized.

15. The method according to claim 11, characterized in that, The execution model in step one is a large language model that supports LLM execution.

16. The method according to claim 11, characterized in that, In step two, the evaluation model uses natural language processing technology to analyze the LLM output results in order to calculate quantitative indicators and identify defects.

Images