An AI application prompt word management and testing method, platform, device and medium
The prompt word management platform, with its centralized storage and decoupled design, solves the problems of dispersion and security in prompt word management, enables parallel testing and scientific evaluation of multiple models, and improves the efficiency and quality of AI application development.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TIANFU JIANGXI LAB
- Filing Date
- 2026-03-04
- Publication Date
- 2026-06-23
AI Technical Summary
The existing methods for managing prompt words are fragmented and lack a unified management mechanism, which leads to difficulties in retrieval and maintenance, obstacles to team collaboration, unstable version control, lack of scientific evaluation methods, and risks of data leakage and interference in the development of multiple projects, and cannot meet the needs of parallel development of multiple models.
The system centrally stores prompt words and establishes independent workspaces based on applications. It adopts a decoupled design between draft and official versions to achieve parallel testing and scientific evaluation of multiple models. Combined with an authentication mechanism, it forms a complete prompt word management and testing system.
It achieves centralized management of prompt words, ensures version stability and data security, improves development efficiency and scientific rigor, forms a complete iterative optimization loop, and adapts to the development needs of multiple projects.
Smart Images
Figure CN122263901A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of AI application development technology, and in particular to a method, platform, device and medium for managing and testing AI application prompt words. Background Technology
[0002] In the current era of rapid development in artificial intelligence (AI) technology, prompt words serve as the core communication medium between AI applications and large-scale models. Their design and optimization directly impact the accuracy, efficiency, and adaptability of the large-scale model's output, making prompt word engineering a crucial aspect of AI application development. With the widespread adoption of AI technology across various fields, the scale of AI application development by enterprises and development teams continues to expand. The demand for parallel development of multiple projects and adaptation to multiple models is increasingly prominent, leading to a continuous increase in the number and types of prompt words. The shortcomings of existing prompt word management and testing methods are becoming increasingly apparent, failing to meet the actual needs of industry development.
[0003] Currently, the management of suggestion words is generally fragmented. A large number of suggestion words are stored in text form across different documents, code repositories, or local devices, lacking a unified management platform. This not only makes the retrieval, maintenance, and updating of suggestion words time-consuming but also creates obstacles to suggestion word sharing and synchronization in team collaboration, significantly reducing development efficiency. At the version control level, the existing model lacks an effective management mechanism, making it impossible to fully trace the modification history of suggestion words and ensuring the stability of suggestion word versions used by online applications. When problems arise after suggestion word modifications, it is impossible to quickly roll back to a stable version, which can easily pose risks to the normal operation of AI applications. The evaluation of suggestion word effectiveness lacks scientific methods and tools. Developers often rely on subjective judgment or experience to evaluate the quality of suggestion words, without objective data as a basis for decision-making. Furthermore, different large models show significant differences in the output effect of the same suggestion word, and existing evaluation methods cannot quantify these differences, making it difficult to make scientific optimization decisions. In addition, in scenarios of parallel development of multiple projects, the lack of effective isolation mechanisms for suggestion words across different applications can easily lead to data leaks or mutual interference, posing a significant threat to suggestion word data security.
[0004] Existing technologies also include some solutions for prompt word management. For example, invention patent CN117993382B discloses a prompt word management method, system, computer device, and computer program product based on a large language model. This prompt word management method based on a large language model includes: acquiring multiple target prompt word templates selected by the user; performing detection based on the large language model, wherein the detection based on the large language model includes: determining a prompt word detection instruction based on the multiple target prompt word templates, and sending the prompt word detection instruction to the large language model; wherein the prompt word detection instruction is used to instruct the large language model to detect the multiple target prompt word templates; and receiving the detection results returned by the large language model.
[0005] However, this prompt management method only focuses on the single step of detecting prompt templates through a large language model. It does not build a complete prompt management system, lacks a professional version control mechanism, fails to achieve parallel prompt testing of multiple models, and does not consider the need for prompt isolation under multi-project development. The management mode is too simplistic and cannot meet the needs of comprehensive management and scientific evaluation of prompts in actual development.
[0006] Invention patent application CN117494814A discloses a method, system, electronic device, and storage medium for full lifecycle management of prompt words. The method involves: acquiring original prompt words; iteratively optimizing the original prompt words using a model-based collaborative optimization method to obtain iteratively optimized prompt words; testing the iteratively optimized prompt words using a multi-turn dialogue mode with a large model to obtain valid prompt words; evaluating the valid prompt words according to evaluation metrics to obtain prompt words that meet the evaluation metrics; and finally, managing the online versions of prompt words that meet the evaluation metrics using a version comparison and replacement method to further complete the full lifecycle management of prompt words.
[0007] While the aforementioned method for managing the entire lifecycle of prompt words involves iterative optimization, testing, and online version management, the testing phase only employs a multi-turn dialogue mode with a single large model, failing to achieve parallel testing of multiple models. The evaluation dimensions are singular and lack comprehensiveness, making it impossible to quantify the adaptation differences of different models to the same prompt word. Furthermore, its version management lacks a decoupling design between draft and official versions, making it susceptible to impacting the stability of online versions due to modifications or deletions of draft versions. It also lacks independent workspaces based on applications and corresponding authentication mechanisms, resulting in insufficient data isolation and an inability to mitigate data leakage risks during parallel development of multiple projects. Additionally, the optimization history retention and analysis of this solution lacks systematicity, making it difficult to achieve data-driven targeted optimization and failing to form a complete closed loop for iterative optimization of prompt words.
[0008] In summary, existing prompt word management solutions all suffer from limited functionality and incomplete design, and there is an urgent need for a centralized, efficient, secure prompt word management and testing solution that can achieve scientific evaluation. Summary of the Invention
[0009] To address the aforementioned issues, this invention proposes an AI application prompt word management and testing method, platform, device, and medium, which can resolve related problems in the existing AI application prompt word management and testing process, and improve the development efficiency and output quality of AI applications.
[0010] The technical solution adopted in this invention is as follows: A method for managing and testing AI application prompt words, including: The prompts for AI applications are stored centrally, and independent workspaces are created for each application, with the prompts for different applications being independent of each other and accessible only with authorization. When editing and saving the prompt, a corresponding draft version is generated and version-related information is recorded. A specified draft version is selected and published as a read-only official version, decoupling the official version from the draft version. During prompt word testing, test parameters are set and prompt words and related parameters are sent to multiple preset artificial intelligence models for parallel testing. The test results of each model are displayed and the results are scored. All test records and scoring information are retained to form a prompt word optimization history.
[0011] Furthermore, the establishment of independent workspaces for each application at the application level, with the prompts for different applications being independent of each other and accessible only by authorization, includes: assigning a unique application identifier to each application; when accessing the official version of the prompt, the application identifier and corresponding key must be included in the request header to complete authentication; only requests that have passed authentication can obtain the official version content of the prompt for the corresponding application.
[0012] Furthermore, the decoupling of the official version from the draft version includes: subsequent modifications and deletions to the draft version will not affect the content of the published official version, nor will they affect the normal retrieval and use of the official version.
[0013] Furthermore, after the prompt word is edited and saved, it is possible to review all historical draft versions of the prompt word, and any historical draft version can be rolled back as a new draft version for subsequent editing and saving operations.
[0014] Furthermore, the test parameters set during the prompt word test include the prompt word variables to be filled, simulated dialogue history information, and system context information, and the types and number of artificial intelligence models participating in the parallel test can be selected autonomously.
[0015] Furthermore, the display of test results for each model and the scoring of the results include: the test results of each artificial intelligence model are displayed side by side on the same interface, and the displayed test results include the output content of the model in response to the prompt word, the response time of the model, and the token consumption data generated by the model in processing the prompt word.
[0016] Furthermore, after retaining all test records and scoring information to form a prompt word optimization history, the process also includes: retrieving and analyzing the prompt word optimization history, comparing the test performance of different versions of the same prompt word and the same version of the prompt word under different artificial intelligence models, and making targeted editing and optimization of the prompt word based on the comparative analysis results, thus forming an iterative optimization closed loop for the prompt word.
[0017] An AI application prompt word management and testing platform, comprising: The prompt word centralization and isolation module is configured to centrally store the prompt words of AI applications and establish independent workspaces for each application, with the prompt words of different applications being independent of each other and accessible only by authorization; The prompt word editing and saving module is configured to generate a corresponding draft version and record version-related information when the prompt word is edited and saved, select a specified draft version and publish it as a read-only official version, and decouple the official version from the draft version; The prompt word testing module is configured to set test parameters and send prompt words and related parameters to multiple preset artificial intelligence models for parallel testing during prompt word testing. It displays the test results of each model and scores the results, and retains all test records and scoring information to form a prompt word optimization history.
[0018] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the AI application prompt word management and testing method.
[0019] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the AI application prompt word management and testing method.
[0020] The beneficial effects of this invention are as follows: This invention forms a complete technical solution from multiple dimensions such as management, version control, evaluation and security, effectively solving various problems existing in the prior art, and greatly improving the efficiency, scientificity and security of AI application prompt word management and testing. At the same time, it realizes the standardized iterative optimization of prompt words, providing solid technical support for AI application development. The specific technical effects are as follows.
[0021] 1. At the prompt word management level, this invention centrally stores prompt words for all AI applications, while establishing independent workspaces for each application. Coupled with an authentication mechanism using a unique application identifier and key, it achieves an organic combination of centralized management and fine-grained isolation. This design not only eliminates the chaos of scattered storage and makes prompt word retrieval, viewing, and maintenance more convenient through a unified management interface, significantly shortening prompt word search time and improving management efficiency in team collaboration, but also ensures that prompt words for different applications are independent through a strict authorization access mechanism. Access to the official version is only possible after authentication, fundamentally preventing prompt word data leakage and mutual interference issues in multi-project parallel development, ensuring the security of prompt word data, and adapting to the actual needs of multi-project development.
[0022] 2. At the version control level, this invention achieves complete decoupling between draft versions and read-only official versions through a separate design. Subsequent modifications or deletions of draft versions will not affect the content or access of the released official version. Simultaneously, the system fully records relevant information for draft versions, supporting full rollback of historical draft versions and rollback of any version. This mechanism ensures both the flexibility of prompt word iteration updates, allowing developers to freely optimize and adjust draft versions, and maximizes the stability of prompt word versions used in online applications. When prompt word modifications encounter problems, developers can quickly trace the modification history and roll back to a stable version, effectively avoiding AI application malfunctions caused by prompt word version issues and ensuring business continuity.
[0023] 3. At the prompt word evaluation level, this invention constructs a scientific and complete integrated evaluation system. It supports the independent selection of multiple artificial intelligence models for parallel testing and allows flexible configuration of test parameters such as prompt word variables to be filled, simulated dialogue history, and system context, making the test scenarios highly similar to the actual application scenarios of the prompt words. Test results are displayed side by side on the same interface, covering multi-dimensional data such as output content, response time, and token consumption. Developers can directly score and add comments. All test records, scores, and comments are retained to form a prompt word optimization history. The optimization history can also be retrieved and analyzed to compare the test performance of different versions of the same prompt word and the same version under different models. Based on this, targeted optimizations can be made and tested again, forming a complete iterative optimization closed loop. This design frees prompt word evaluation from the limitations of subjective judgment, forming optimization decisions based on objective data. Parallel testing of multiple models can comprehensively quantify the adaptation differences of different models to the same prompt word, making prompt word optimization more targeted and continuously improving the quality of prompt words and multi-model adaptability.
[0024] 4. Compared to CN117993382B, this invention is not only designed for the single stage of prompt word detection, but also constructs a complete prompt word management and testing system, from centralized and isolated management and professional version control to multi-model parallel scientific evaluation and data-driven iterative optimization. It has more comprehensive functions and can meet the full-process management needs of prompt words in AI application development. At the same time, it solves the security isolation problem of multi-project development and is suitable for actual industry development scenarios.
[0025] 5. Compared to CN117494814A, this invention upgrades the testing process from single-model multi-turn dialogue to multi-model parallel testing, making the evaluation results more comprehensive and valuable. In version management, it decouples drafts from official versions, fundamentally ensuring the stability of online versions. It also adds an independent workspace and identity verification mechanism for application dimensions, strengthening the data isolation effect under multi-project development. At the same time, through systematic optimization of historical retention and analysis, it forms a complete closed loop for iterative optimization of prompt words, making the full lifecycle management of prompt words more scientific and efficient, enabling continuous optimization of prompt words, and further improving the output quality of AI applications. Attached Figure Description
[0026] Figure 1 This is a flowchart of an AI application prompt word management and testing method according to Embodiment 1 of the present invention.
[0027] Figure 2 This is a flowchart of the application management module in Embodiment 2 of the present invention.
[0028] Figure 3 This is one of the flowcharts of the prompt word management module in Embodiment 2 of the present invention.
[0029] Figure 4 This is the second flowchart of the prompt word management module in Embodiment 2 of the present invention.
[0030] Figure 5 This is a flowchart of the model management module in Embodiment 2 of the present invention.
[0031] Figure 6 This is a flowchart of the prompt word evaluation module in Embodiment 2 of the present invention. Detailed Implementation
[0032] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments are now described. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention; that is, the described embodiments are only a part of the embodiments of the invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0033] Example 1 like Figure 1 As shown, this embodiment provides a method for managing and testing AI application prompt words, including: The prompts for AI applications are stored centrally, and independent workspaces are created for each application, with the prompts for different applications being independent of each other and accessible only with authorization. When editing and saving the prompt, a corresponding draft version is generated and version-related information is recorded. A specified draft version is selected and published as a read-only official version, decoupling the official version from the draft version. During prompt word testing, test parameters are set and prompt words and related parameters are sent to multiple preset artificial intelligence models for parallel testing. The test results of each model are displayed and the results are scored. All test records and scoring information are retained to form a prompt word optimization history.
[0034] It should be noted that this method achieves centralized and isolated management of prompt words, standardizes the version iteration and release process of prompt words, ensures the stability of online versions, realizes scientific evaluation of prompt words through multi-model parallel testing, and provides data support for subsequent iterations of prompt words by retaining the optimization history, thus improving the efficiency and scientific nature of AI application prompt word management and testing as a whole.
[0035] Preferably, an independent workspace is established for each application, with the prompts for different applications being independent of each other and accessible only by authorized personnel. Specifically, each application is assigned a unique application identifier. When accessing the official version of the prompt, the application identifier and the corresponding key must be included in the request header to complete authentication. Only requests that have passed authentication can obtain the official version of the prompt content for the corresponding application.
[0036] Specifically, when staff create an AI application workspace, the platform automatically generates and assigns a unique application identifier to each application, and matches it with a corresponding access key. When staff or related systems retrieve the official version of the prompt word, they need to enter the application identifier and key in the request header of the access request. After receiving the request, the platform first verifies the identifier and key. If the verification passes, the corresponding content can be retrieved; if the verification fails, the access request is rejected.
[0037] It should be noted that this method achieves precise control over access to the official version of the prompt words through dual authentication of unique identifiers and keys, strengthens the isolation effect of prompt words between different applications, effectively avoids unauthorized access and data leakage issues, and ensures the security of prompt word data in multi-project parallel development scenarios.
[0038] Preferably, the decoupling of the official version from the draft version means that any subsequent modifications or deletions to the draft version will not affect the content of the published official version, nor will they affect the normal retrieval and use of the official version.
[0039] Specifically, when the platform publishes a draft version as the official version, it stores the content of the draft version independently, forming storage data separate from the original draft version. When staff subsequently modify or delete the content of the original draft version, the platform only updates or clears the storage data of the draft version and does not perform any operation on the independent storage data of the official version. Requests to retrieve the official version can still obtain the complete content normally.
[0040] It should be noted that this method achieves physical data separation between the draft version and the official version, avoiding interference from subsequent operations on the draft version with the official version used online, ensuring the stability and retrieval availability of the official version, and reducing the risk of AI application malfunctions due to improper operation of the prompt word version.
[0041] Preferably, after the prompt word is edited and saved, it is possible to review all historical draft versions of the prompt word, and any historical draft version can be rolled back as a new draft version for subsequent editing and saving operations.
[0042] Specifically, the platform will store all draft versions generated each time the prompt word is edited and saved. A version rollback view entry is set up on the platform interface, through which staff can view all historical draft versions of the prompt word and the corresponding version information. At the same time, the platform provides a version rollback function. When a staff selects any historical draft version and triggers the rollback operation, the platform will restore the content of that historical version to a new draft version, which the staff can then continue to edit and save.
[0043] It should be noted that this method enables full traceability of the draft version of the prompt, making it convenient for staff to view the iterative modification process of the prompt. The version rollback function allows staff to quickly restore to a historical stable version when they discover modification problems, thus improving the flexibility and fault tolerance of prompt iteration optimization.
[0044] Preferably, the test parameters set during the prompt word test include the prompt word variables to be filled, simulated dialogue history information, and system context information, and the types and number of artificial intelligence models participating in the parallel test can be selected autonomously.
[0045] Specifically, when staff initiate a prompt word test task on the platform, they fill in the reserved variables in the prompt words in the test parameter configuration interface, and at the same time enter the simulated dialogue history information and system context information. After completing the parameter configuration, staff can choose the types of models participating in parallel testing from the list of artificial intelligence models provided by the platform according to the test requirements. They can also adjust the number of models participating in the test according to the test accuracy requirements. After selecting and confirming, all settings can be completed.
[0046] It should be noted that the rich test parameter configuration makes the test scenarios closer to the actual application scenarios of the prompt words, which enhances the actual reference value of the test results. The ability to independently select the type and number of models makes the test more targeted, and the test plan can be flexibly adjusted according to different test needs, which improves the flexibility and scenario adaptability of prompt word testing.
[0047] Preferably, the test results of each model are displayed and scored. Specifically, the test results of each artificial intelligence model are displayed side by side on the same interface. The displayed test results include the output content of the model in response to the prompt word, the response time of the model, and the token consumption data generated by the model in processing the prompt word.
[0048] Specifically, after receiving the test results returned by each artificial intelligence model, the platform standardizes and organizes the results, and presents the test results of all models in a side-by-side layout on the test results display interface. In the display area of each model, the output content of the model, the response time from sending the request to receiving the result, and the token consumption data generated by the model in processing the prompt word are displayed simultaneously. Staff can directly score the test results of each model on this interface.
[0049] It should be noted that the side-by-side display on the same interface makes it easy for staff to intuitively compare the test results of different models. The presentation of multi-dimensional test data allows staff to comprehensively evaluate the effectiveness of the prompts from multiple perspectives, improving the comprehensiveness of the evaluation. At the same time, completing the scoring directly on the display interface simplifies the operation process and improves the overall efficiency of test scoring.
[0050] Preferably, after retaining all test records and scoring information to form a prompt word optimization history, the prompt word optimization history can be retrieved and analyzed to compare the test performance of different versions of the same prompt word and the same version of the prompt word under different artificial intelligence models. Based on the comparative analysis results, the prompt word can be edited and optimized in a targeted manner to form an iterative optimization closed loop for the prompt word.
[0051] Specifically, the platform stores all test records and scoring information in a structured manner and forms a history of prompt word optimization, providing functions for retrieving and analyzing the optimization history. Staff can use the platform's analysis tools to filter and view various test data and scores of different versions of the same prompt word, and can also compare the test performance of the same version of the prompt word under different artificial intelligence models. Based on the advantages and disadvantages of the prompt word obtained from the analysis, they can return to the editing interface to make targeted modifications and optimizations. The optimized prompt word can be tested again to complete a new round of evaluation and analysis.
[0052] It should be noted that by retrieving and analyzing historical optimization data, staff can make optimization decisions for prompt words based on actual data, avoiding optimization biases caused by subjective judgments. The resulting iterative optimization loop makes the prompt word optimization process more systematic, continuously improving the quality of prompt words and their adaptability to different artificial intelligence models.
[0053] Accordingly, this embodiment also provides an AI application prompt word management and testing platform, including a prompt word centralization and isolation module, a prompt word editing and saving module, and a prompt word testing module. The prompt word centralization and isolation module is configured to centrally store the prompt words of AI applications and establish independent workspaces for each application, with prompt words for different applications being independent of each other and accessible only with authorization. The prompt word editing and saving module is configured to generate a corresponding draft version and record version-related information when editing and saving prompt words, and select a specified draft version to publish as a read-only official version, decoupling the official version from the draft version. The prompt word testing module is configured to set test parameters and send the prompt words and related parameters to multiple preset artificial intelligence models for parallel testing during prompt word testing, display the test results of each model and score the results, and retain all test records and scoring information to form a prompt word optimization history.
[0054] Specifically, the platform establishes a unified storage architecture through centralized and isolated prompt word modules, centralizing the collection of prompt words for various AI applications. It also divides the workspaces by application and configures authorized access mechanisms. The prompt word editing and saving module provides staff with a prompt word editing interface, automatically generating a draft version and recording version information when a save operation is triggered. It also provides a version release function to convert the selected draft version into a read-only official version. The prompt word testing module provides a test parameter configuration interface and model selection function, sending the integrated prompt words and parameters to the selected model for parallel testing, displaying test results and supporting scoring, while storing test-related information to form an optimization history.
[0055] It should be noted that the modular architecture design breaks down the core functions of the platform, with each module performing its own function and working together to improve the platform's development and maintenance efficiency and operational stability. The modular function configuration makes the implementation of each link more targeted, ensuring the efficient implementation of various management and testing functions of the platform.
[0056] Example 2 This embodiment provides an AI application prompt word management and testing platform, including an application module, a prompt word management module, a model management module, and a prompt word evaluation module, which are described in detail below.
[0057] I. Application Management Module like Figure 2 As shown, the application management module can execute the following processes: Creating an Application: Users click the "Create Application" button on the platform interface, which will bring up a form for creating an application. In the form, users need to fill in basic information such as the application name and identifier. After the form is completed and submitted, the system (i.e., the platform) automatically generates a unique AppId for each application and stores the application information in the database. For example, if an AI education company creates an AI application for English learning, it fills in "English Learning AI Application" as the application name and a custom identifier; the system will generate an AppId such as "APP001".
[0058] Application List: The platform's main interface features an "Application List" area, displaying all applications the user has access to. Applications are presented in a list format, with each row showing key information such as the application name and AppId. A search box is also provided above the list, allowing users to search by application name or AppId. For example, if a user enters "English learning," the list will quickly locate "English Learning AI Application."
[0059] Application Details: When a user clicks on an application in the application list to enter the application details page, they can view and manage all prompts for that application. The page prominently displays the application's AppId and, if applicable, API access keys. Additionally, there is an entry point for prompt management; clicking it will take you to the application's prompt management interface.
[0060] Prompt retrieval endpoint: The platform provides a secure API endpoint based on the HTTP protocol, such as GET / api / v1 / prompts / {prompt_id} / release. When external programs call this endpoint, they must include authentication information, such as AppId and key, in the request header. After receiving the request, the platform first verifies the identity information in the request header. If the verification is successful, it retrieves the latest release version content of the corresponding prompt word from the database based on the prompt_id in the request and returns it to the external program in JSON format as the response body.
[0061] II. Prompt Word Management Module like Figure 3 and Figure 4 As shown, the prompt word management module can execute the following process: Creating Prompt Words: After entering the prompt word management interface of a specific application, users can click the "Create Prompt Word" button to bring up a form for creating prompt words. In the form, users need to fill in information such as the prompt word name, detailed content, and description. After the form is submitted, the system generates a unique PromptId for the prompt word and stores the relevant information in the database, establishing a connection with the corresponding application's AppId. For example, in the "English Learning AI Application," to create a prompt word for generating English essay writing prompts, the name could be "English Essay Prompt Word," the content "Please write an English essay based on the following topic: {topic}," and the description "Used to guide students in English essay writing."
[0062] Editing and Version History: When a user edits a created prompt, each time they click "Save," the system automatically generates a new draft version. The system records the version number (e.g., V1.0, V1.1, etc.), modification time, and author information for each draft version, and stores this version information in the database. The prompt details page includes a version history view; users can click to view all historical version records. For any historical version, users can choose to roll back and use that version as a new draft for further editing.
[0063] Release Prompt: From the list of draft versions of prompts, the user selects a draft version and clicks the "Publish" button. The system copies the content of that draft version and marks it as a read-only Release version, while recording information such as the release time. The Release version is decoupled from its source draft version; subsequent deletion of draft versions does not affect the already released Release version. When external programs retrieve prompts via the API endpoint, they always obtain the Release version content of the current prompt. If a new version is released, the new release version will overwrite the old Release version.
[0064] Deleting Prompts: In the prompt management interface, users can delete prompts. By default, the system performs a soft delete, marking the prompt as deleted and moving it to the recycle bin. Deletion does not affect currently released release versions; online applications can still access the content of that release version until a new version is released and overwrites it. In the recycle bin, users can choose to permanently delete the prompt or restore it.
[0065] III. Model Management Module like Figure 5 As shown, the model management module can execute the following processes: Adding a Model: Users with administrator or specific permissions can find the "Model Management" section in the platform settings interface and click the "Add Model" button to bring up a form for adding a model. The form requires configuring information such as the model name, provider, API Endpoint, and API Key. After the form is submitted, the system attempts to establish a connection with the configured API Endpoint and verify the validity of the API Key. If verification succeeds, the model information is stored in the database, and the model status is marked as "Available"; if verification fails, the user is prompted to check the configuration information, and the model status is marked as "Unavailable". For example, to add the OpenAI GPT-3 model, fill in the model name "GPT-3", provider "OpenAI", corresponding API Endpoint, and correct API Key.
[0066] Model List and Status: The "Model Management" page displays all configured models in a list format. Each row in the list shows the model name, provider, and connection status (e.g., "Available" or "Unavailable"). For models with a status of "Unavailable," a link or button is provided, allowing users to retry the connection or modify the configuration information.
[0067] IV. Prompt Word Evaluation Module like Figure 6As shown, the prompt word evaluation module can execute the following process: Creating a Test Task: Users select a prompt (draft or Release version) in the prompt management interface and click the "Test" button. The system redirects to the prompt evaluation page and automatically pre-selects the prompt. On the evaluation page, users can set test parameters. In the "Model Selection" area, multiple model checkboxes are provided, allowing users to select multiple models for parallel testing. In the "Input Parameters" area, a key-value pair form is provided, where users can fill in the corresponding values based on the variables in the prompt (e.g., {user_input}, {language}). In the "Context" area, a text area is provided where users can enter simulated dialogue history or system context. For example, for a prompt used for intelligent customer service, the user selects the GPT-3 model and the Wenxin Yiyan model, enters {user_input} as "How to check order status" in the input parameter form, and enters the previous dialogue record in the context area: "The user inquired about product functions, the customer service answered, and the user inquired about the order status."
[0068] Execution and Result Display: After setting the test parameters, the user clicks the "Start Test" button. The system will then send requests to all selected models according to their API requirements, using the combined prompts, parameters, and context. A progress bar will be displayed in real-time on the page. After each model returns its results, the system will display the results, execution time, and token consumption (if the API supports this) of all models side-by-side on a single interface. The results display area will present the output of each model, the time taken to generate the results, and the number of tokens consumed in a clear layout.
[0069] Results Scoring and Recording: Below the results display area for each model, there is a scoring area where users can rate the output of each model (e.g., 1-5 stars). A text box is also provided where users can add comments or notes, recording their evaluation of the result and the reasons for their feedback. After the user clicks the "Save" button, the system stores all test records, results, and scores in the database, establishing a link with the corresponding prompt words to form the optimization history of the prompt words.
[0070] Example 3 This embodiment is based on embodiment 1: This embodiment provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the AI application prompt word management and testing method of Embodiment 1. The computer program can be in the form of source code, object code, executable file, or some intermediate form.
[0071] Specifically, the computer device's memory pre-stores a computer program that implements the prompt word management and testing method. The program contains execution instructions for each step. After the staff starts the program, the processor retrieves and loads the computer program from the memory and executes the relevant steps in sequence according to the instructions in the program, such as centralized and isolated management of prompt words, version editing, saving and publishing, multi-model parallel testing and optimization history retention and analysis, to complete the entire prompt word management and testing process.
[0072] It should be noted that the method for managing and testing prompt words is implemented through a computer program. Relying on the computing power of computer equipment makes the execution of the method more efficient and standardized, avoiding errors caused by manual operation. At the same time, the portability of computer equipment allows the method to be applied in different AI application development environments, thus improving the practicality of the method.
[0073] Example 4 This embodiment is based on embodiment 1: This embodiment provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the AI application prompt word management and testing method of Embodiment 1. The computer program can be in the form of source code, object code, executable file, or some intermediate form. The storage medium includes: any entity or device capable of carrying computer program code, recording media, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.
[0074] Specifically, the computer-readable storage medium stores a computer program that implements the prompt word management and testing method in the form of digital storage, and the execution logic of the program matches the method. When the storage medium establishes a data connection with a device equipped with a processor, the processor can read and execute the computer program from the storage medium and complete all operations such as centralized management of prompt words, version control, parallel testing and optimization iteration according to the program instructions.
[0075] It should be noted that storing and distributing the method program through a computer-readable storage medium facilitates the copying, transfer, and deployment of the program across different computer devices, reducing the cost of implementing the method. At the same time, the stable storage characteristics of the storage medium ensure that the program data will not be easily lost, thus improving the reliability of the method implementation process.
[0076] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.
[0077] It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
Claims
1. A method for managing and testing AI application prompt words, characterized in that, include: The prompts for AI applications are stored centrally, and independent workspaces are created for each application, with the prompts for different applications being independent of each other and accessible only with authorization. When editing and saving the prompt, a corresponding draft version is generated and version-related information is recorded. A specified draft version is selected and published as a read-only official version, decoupling the official version from the draft version. During prompt word testing, test parameters are set and prompt words and related parameters are sent to multiple preset artificial intelligence models for parallel testing. The test results of each model are displayed and the results are scored. All test records and scoring information are retained to form a prompt word optimization history.
2. The AI application prompt word management and testing method according to claim 1, characterized in that, The method of establishing independent workspaces for each application based on the application dimension, with the prompts for different applications being independent of each other and accessible only by authorization, includes: assigning a unique application identifier to each application; when accessing the official version of the prompt, the application identifier and corresponding key must be included in the request header to complete authentication; only requests that have passed authentication can obtain the official version content of the prompt for the corresponding application.
3. The AI application prompt word management and testing method according to claim 1, characterized in that, The decoupling of the official version from the draft version means that any subsequent modifications or deletions to the draft version will not affect the content of the published official version, nor will they affect the normal retrieval and use of the official version.
4. The AI application prompt word management and testing method according to claim 1, characterized in that, After the prompt word is edited and saved, it is possible to review all historical draft versions of the prompt word, and any historical draft version can be rolled back as a new draft version for subsequent editing and saving operations.
5. The AI application prompt word management and testing method according to claim 1, characterized in that, The test parameters set during prompt testing include the prompt variable to be filled, simulated dialogue history information, and system context information, and the types and number of artificial intelligence models participating in parallel testing can be selected independently.
6. The AI application prompt word management and testing method according to claim 1, characterized in that, The process of displaying and scoring the test results of each model includes: displaying the test results of each artificial intelligence model side by side on the same interface, and displaying the test results content including the model's output content for the prompt word, the model's response time, and the token consumption data generated by the model in processing the prompt word.
7. The AI application prompt word management and testing method according to claim 1, characterized in that, After retaining all test records and scoring information to form a prompt word optimization history, the process also includes: retrieving and analyzing the prompt word optimization history, comparing the test performance of different versions of the same prompt word and the same version of the prompt word under different artificial intelligence models, and making targeted editing and optimization of the prompt word based on the comparative analysis results, thus forming an iterative optimization closed loop for the prompt word.
8. An AI application prompt word management and testing platform, characterized in that, include: The prompt word centralization and isolation module is configured to centrally store the prompt words of AI applications and establish independent workspaces for each application, with the prompt words of different applications being independent of each other and accessible only by authorization; The prompt word editing and saving module is configured to generate a corresponding draft version and record version-related information when the prompt word is edited and saved, select a specified draft version and publish it as a read-only official version, and decouple the official version from the draft version; The prompt word testing module is configured to set test parameters and send prompt words and related parameters to multiple preset artificial intelligence models for parallel testing during prompt word testing. It displays the test results of each model and scores the results, and retains all test records and scoring information to form a prompt word optimization history.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the AI application prompt word management and testing method according to any one of claims 1-7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the AI application prompt word management and testing method according to any one of claims 1-7.