Industrial mini-model prompt word optimization methods, devices, electronic equipment and media

By acquiring and analyzing the explanatory text of small models, and performing error attribution and regression testing, the problems of long repair cycles and poor results in traditional methods are solved, achieving efficient and accurate prompt optimization.

CN123114982APending Publication Date: 2026-11-17INVENTEC CHONGQING
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611504226.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-22
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Traditional industrial small model prompt word optimization relies on expert experience, resulting in long repair cycles and poor results. It cannot meet the needs of rapid changes in factory business logic and cannot clearly define the internal reasoning path of the model.

Method used

By acquiring initial prompt words and business test case sets, we use small models to generate explanatory analysis text, perform error attribution and correction, establish an automated regression testing process, and optimize prompt words until preset conditions are met.

Benefits of technology

This makes the internal reasoning path of the small model explicit, improves the accuracy and relevance of prompt word correction, shortens the optimization cycle, and reduces the cost of manual debugging.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123114982A_ABST
    Figure CN123114982A_ABST
Patent Text Reader

Abstract

This application relates to the field of artificial intelligence technology, and provides a method, device, electronic device, and medium for optimizing prompt words in an industrial small-scale model. The method includes: acquiring initial prompt words and a set of business test cases; conducting tests using the small-scale model based on the initial prompt words and the set of business test cases to obtain test results; if the test results do not reach a preset accuracy rate, attributing errors based on the explanatory text of the erroneous test cases in the test results, and correcting the initial prompt words based on the error attribution to obtain corrected prompt words; and conducting regression testing using the small-scale model based on the corrected prompt words until a preset termination condition is met to obtain the final optimized prompt words. This application achieves automated closed-loop optimization of prompt words while ensuring that they comply with industrial production safety standards and performance requirements, reducing optimization costs, shortening the optimization cycle, and adapting to the needs of continuous process updates in industrial scenarios, ensuring the long-term stable operation of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a method, apparatus, electronic device and medium for optimizing prompt words in industrial small models. Background Technology

[0002] In many core business scenarios of industrial manufacturing (such as appearance defect classification, standard operating procedure parsing, and automatic work order distribution), enterprises are increasingly inclined to use privately deployed small models. Small models typically refer to models with a small number of parameters, which can run on edge computing devices or local servers. Compared to cloud-based models with extremely large parameters, privately deployed small models can effectively protect the privacy and security of core factory data, while significantly reducing inference latency and controlling computing costs. On continuously operating production lines, millisecond-level response latency and absolute data security are fundamental to maintaining production order; therefore, the application of small models has irreplaceable value.

[0003] Cue words are instruction texts that guide small models to generate specific outputs. In smart manufacturing scenarios, factory business logic is complex and demands extremely high accuracy. Small models, limited by their parameter size, are highly sensitive to cue words; vague or generic instructions can easily lead to model illusions. This illusion can have serious consequences in practical applications, such as misjudgments in quality inspection (classifying qualified products as defective) or generating incorrect machine instructions in automated processes, thus affecting production safety and efficiency. Traditional cue word optimization and debugging rely on expert experience, and when the model outputs errors in business scenarios (such as misclassifying defect types or extracting incorrect screw specifications), developers can only see the final result and cannot know the internal reasoning path of the model. This results in long cue word repair cycles and poor cue word repair effects, failing to meet the rapid changes required by factory business logic. Summary of the Invention

[0004] In view of the shortcomings of the prior art, this application provides a method, device, electronic device and medium for optimizing prompt words in industrial small models, which at least solves the problem that the inference path inside the model cannot be known in the related technologies, resulting in a long prompt word repair cycle and poor prompt word repair effect, which cannot meet the needs of rapid changes in factory business logic.

[0005] Firstly, this application provides a method for optimizing prompt words in industrial small-scale models, including: Obtain the initial prompt words and business test case set; Based on the initial prompt words and business test case set, the small model is used to conduct tests and obtain test results. The test results include the explanation and analysis text of the small model during internal reasoning. The explanation and analysis text is generated by the small model based on the analysis fields of the preset structure in the tool definition before executing the tool call command. If the test results do not reach the preset accuracy rate, the error will be attributed based on the explanation and analysis text of the erroneous test cases in the test results, and the initial prompt words will be corrected based on the error attribution to obtain the corrected prompt words. Based on the corrected prompt words, regression testing is performed using a small model until the preset termination condition is met, thus obtaining the final optimized prompt words.

[0006] In one embodiment of this application, before obtaining the test results, the method further includes: The structure of the explanatory and analyzed text is validated, and the validation results are obtained. If the validation result indicates that the interpretation analysis text does not conform to the preset structure of the analysis field, the corresponding interpretation analysis text will be marked as an illegal response and / or the tool call instruction will be resent to the small model.

[0007] In one embodiment of this application, before obtaining the test results, the method further includes: Receive target configuration information, which is used to add analysis fields with preset structures to the tool call in the tool definition.

[0008] In one embodiment of this application, error attribution is performed based on the explanatory analysis text of erroneous test cases in the test results, including: The process involves analyzing the intent recognition content, parameter extraction basis, and confidence level in the explanatory text of the erroneous use case; determining the error attribution of the erroneous use case based on the intent recognition content, parameter extraction basis, and confidence level; and correcting the initial prompt word based on the error attribution to obtain a corrected prompt word, including adding at least one of the following to the initial prompt word based on the error attribution: rules, few-sample examples, or constraints.

[0009] In one embodiment of this application, after obtaining the correction prompt word, the process includes: The version number of the corrected prompt words, optimization direction labels, modified summaries, accuracy changes, and illusion rate changes are recorded in the change log; optimization direction labels include tool distribution labels to identify incorrect tool selection, parameter extraction labels to identify parameter extraction bias, illusion suppression labels to identify model fabrication, and boundary condition labels to identify boundary conditions.

[0010] In one embodiment of this application, the method further includes: If the accuracy of the test results does not improve after a preset number of consecutive tests, or if the preset intervention conditions are met, a target prompt message is triggered, which prompts for manual review. Confirmation information based on the manual review is received. In response to the confirmation information, the final optimized prompt words, business test case set, and tool definition are deployed to the production environment.

[0011] In one embodiment of this application, before obtaining the initial prompt words and business test case set, the method further includes: based on the original prompt words and business test case set, conducting tests through a small model to obtain the original accuracy rate and original illusion rate corresponding to the original prompt words; and using the original accuracy rate and original illusion rate as the starting point for iteration.

[0012] Secondly, this application also provides a prompt word optimization device for industrial miniature models, comprising: The data acquisition module is configured to acquire initial prompt words and business test case sets; The first test module is configured to perform tests through a small model based on the initial prompt words and business test case set, and obtain test results. The test results include the explanatory analysis text of the small model during internal reasoning. The explanatory analysis text is generated by the small model based on the analysis fields with a preset structure in the tool definition before executing the tool call command. The error correction module is configured to, if the test results do not reach the preset accuracy rate, perform error attribution based on the explanation and analysis text of the error cases in the test results, and correct the initial prompt words based on the error attribution to obtain the corrected prompt words; The second testing module is configured to perform regression testing using a small model based on the corrected prompt words until a preset termination condition is met, thus obtaining the final optimized prompt words.

[0013] Thirdly, this application also provides an electronic device, which includes: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the steps of the above-described industrial small model prompt word optimization method.

[0014] Fourthly, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, enables the computer to implement the steps of the above-described industrial small model prompt word optimization method.

[0015] This technical solution obtains initial prompt words and a set of business test cases; based on these, it conducts tests using a small model to obtain test results; if the test results do not reach a preset accuracy rate, it performs error attribution based on the explanatory text of the erroneous test cases in the test results, and corrects the initial prompt words based on the error attribution to obtain corrected prompt words; based on the corrected prompt words, it performs regression testing using a small model until a preset termination condition is met to obtain the final optimized prompt words. It has at least the following technical effects: First, by embedding pre-structured analysis fields into the tool call in the tool definition, the small model generates explanatory analysis text before executing the tool call instruction, transforming the originally invisible internal reasoning process into structured and parsable data. This breaks the opaque black box state of the decision-making process in traditional tool calls, making the model's internal reasoning path and decision basis explicit, and providing a data foundation for subsequent error analysis and attribution. Second, based on the generated explanatory analysis text, the error causes of the erroneous use cases are accurately attributed, and the initial prompt words are corrected accordingly. This can clarify the root cause of each type of error, avoid the blind trial and error that previously relied on experience or intuition, and thus improve the accuracy and pertinence of prompt word correction. To a certain extent, this solves the problems of unclear optimization direction caused by the lack of quantitative feedback loop in traditional methods, as well as the high iteration cost caused by over-reliance on expert experience. Third, by establishing an automated regression testing process, a continuous iterative closed loop was constructed, from executing tests, analyzing results, modifying prompt words to re-regressing and verifying. This enabled a data-driven automated prompt word optimization process, which not only reduced the manpower and time costs of repeated manual debugging, but also shortened the overall prompt word optimization cycle from problem discovery to resolution, and from the initial version to the stable version.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 This is a schematic flowchart illustrating a prompt word optimization method for an industrial small model, as shown in an exemplary embodiment of this application. Figure 2 This is a schematic diagram illustrating the structure of a prompt word optimization device for an industrial small model, as shown in an exemplary embodiment of this application; Figure 3 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0018] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0019] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the shape, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0020] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0021] Before providing a detailed description of the technical solution of this application, some technical terms and concepts involved in the embodiments of this application will be explained so that those skilled in the art can accurately understand the technical solution of this application.

[0022] Small models refer to language models with relatively small parameters that can run on edge computing devices or local servers in factories. They are used to perform complex logical judgments and tool invocation tasks in smart manufacturing scenarios.

[0023] Tool call instructions refer to instructions that a small model uses to call predefined external functions and output corresponding parameters based on input information, such as calling defect classification functions or robotic arm control functions.

[0024] Explanatory analysis text refers to the structured text generated by the small model based on the analysis fields with a preset structure in the tool definition before executing the tool call command. It is used to characterize the internal reasoning process of the model. Its content structure can include intent recognition content, parameter extraction basis, and confidence level.

[0025] Error attribution refers to the process of determining the specific cause type of a test error based on the explanatory analysis text corresponding to the error test case. Error cause types may include, for example, errors in understanding concepts, deviations in parameter extraction, and logical judgment errors.

[0026] Please see Figure 1 , Figure 1 This is a flowchart illustrating a prompt word optimization method for an industrial small-scale model, as shown in an exemplary embodiment of this application. Figure 1 As shown, in an exemplary embodiment, the prompt word optimization method for industrial mini-models includes steps S110 to S140, and each step is described in detail below.

[0027] S110, Obtain initial prompt words and business test case set; S120, based on the initial prompt words and business test case set, test through the small model to obtain test results, including the explanation and analysis text of the small model during internal reasoning. The explanation and analysis text is generated by the small model based on the analysis fields of the preset structure in the tool definition before executing the tool call command. S130, if the test results do not reach the preset accuracy rate, then the error is attributed based on the explanation and analysis text of the erroneous test cases in the test results, and the initial prompt words are corrected based on the error attribution to obtain the corrected prompt words; S140: Based on the corrected prompt words, a small model is used to perform regression testing until the preset termination condition is met, thus obtaining the final optimized prompt words.

[0028] The prompt word optimization method for the industrial mini-model in this embodiment can be executed by a computing device deployed on the factory's internal network, or by an artificial intelligence agent configured on the computing device.

[0029] Initial prompts, business test case sets, and tool definitions can be stored in the same project directory or separately in storage locations accessible to the test program. The small model can be deployed on the same computing device as the test program, or on another computing device within the factory's internal network, and receives test requests and returns tool call responses through the model's API.

[0030] The business test case set includes input data, expected tool names, and expected parameters. Input data can be text descriptions of quality inspection images, sensor logs, or operator voice commands, etc.; the expected tool name can be a defect classification tool (classify_defect) or a robotic arm control tool (control_robot_arm), etc.; expected parameters can include defect type, defect coordinates, screw specifications, torque values, robotic arm movement coordinates, or alarm types, etc. The business test case set covers core factory business scenarios: including normal production processes, abnormal operating conditions (such as sensor offline, blurred images), and boundary conditions (such as extremely small defects, critical thresholds), etc.

[0031] In some examples, the small model in this application can be any privately deployed model that supports tool calls; this application does not depend on a specific model architecture or a specific parameter scale. The business test case set (test_cases.json) can be stored in JavaScript Object Notation (JSON) format. Each record can contain a test case identifier, input data, expected tool name, expected parameters, and a scenario type field. The test case identifier is used to map the model response to the original test input; the scenario type field is used to distinguish between normal production processes, abnormal operating conditions, and boundary conditions. The business test case set can also be a table file or database record with corresponding fields, as long as the test program can maintain the mapping relationship between the input data and the expected results.

[0032] Once testing begins, the small model can be used to perform tests based on the initial prompts and the business test case set, and the test results can be obtained. During the testing process, before calling tools (such as determining whether a part is qualified or controlling the movement of a robotic arm), the small model needs to generate explanatory analysis text based on the pre-structured analysis fields, transforming the originally invisible internal reasoning of the small model into parsable text data. That is, when the small model receives input data from the business test case set, it must, according to the mandatory constraints defined by the tool, first explain its thought process in the explanatory analysis text, and then output the specific tool call parameters.

[0033] In other words, the analysis field in the tool definition and the tool's function parameters are in the same parameter structure of the tool call. Before filling in the function parameters, the small model fills in the analysis field first, thereby forcibly exposing the thought chain.

[0034] Test results can include accuracy, tool selection error rate, parameter matching rate, and illusion rate. When the test results reach the preset accuracy, the prompt can proceed to the termination condition judgment. When the test results do not reach the preset accuracy, erroneous test cases are extracted, and error attribution can be performed based on the explanatory analysis text of the erroneous test cases, rather than simply guessing the cause of the error based on the final tool name or parameters. Because the explanatory analysis text exposes the thought process of the small model, error attribution can clearly determine whether the error stems from confusion of industrial concepts, deviation of sensor value readings, or error in business logic judgment. Based on a clear error attribution, the initial prompt can be modified to obtain a corrected prompt.

[0035] It should be noted that the correction is not a random rewrite of the initial prompt words, but rather a targeted supplement to content related to the cause of the error. For example, when the conceptual boundaries are unclear, distinguishing rules are added; when similar defects are repeatedly confused, standard descriptions and comparisons are added; when the source of parameters is unclear, parameter extraction restrictions are added; when uncertain inputs are directly converted into production actions, manual re-inspection constraints are added.

[0036] Next, regression testing is conducted using a small model based on the corrected prompts. Regression testing involves retesting the entire set of business test cases to verify that the corrected prompts improve the performance of the target error test cases without introducing new errors into other test cases.

[0037] This process can be automatically and cyclically executed by the implementing entity, continuously running, analyzing, modifying, and regressing until a preset termination condition is met, ultimately obtaining the optimized keywords for the locked version. The preset termination condition could be, for example, the accuracy of the test results reaching a preset target accuracy, the number of iterations reaching a preset maximum, or triggering manual intervention.

[0038] In some embodiments, before obtaining the test results, the method further includes: Perform structural validation on the interpretation and analysis text and obtain the validation results; if the validation results indicate that the interpretation and analysis text does not conform to the preset structure of the analysis fields, then mark the corresponding interpretation and analysis text as an illegal response and / or resend the tool call command to the small model.

[0039] Understandably, in practical applications, small models, due to limitations in parameter size and instruction compliance, may ignore system instructions and attempt to bypass the analysis phase by directly outputting tool call results when facing complex industrial scenarios. If this occurs, the explanatory analysis text will be missing or formatted incorrectly, leading to a lack of data support for subsequent error attribution.

[0040] Before obtaining the test results, this embodiment performs structural verification on the explanatory analysis text generated by the small model, and obtains the verification results. This structural verification not only checks whether the explanatory analysis text exists, but also whether it meets the requirements of the preset structure of the analysis fields in the tool definition. For example, it checks whether the explanatory analysis text contains content structures corresponding to intent recognition, parameter extraction basis, and confidence level. If the verification result indicates that the explanatory analysis text does not conform to the preset structure of the analysis fields, interception measures are taken.

[0041] Interception measures can include marking the corresponding explanatory text as an illegal response, preventing the output of illegal responses from being included in normal accuracy statistics, and recording them as format failure errors for subsequent analysis; or resending the tool call command to the small model to force a retry until the output of explanatory text conforms to a preset structure or the preset retry limit is reached; or performing both operations simultaneously, first marking the current response as an illegal response, and then resending the tool call command for retry. All three methods can intercept non-compliant outputs, and those skilled in the art can choose the appropriate method based on the real-time and fault-tolerance requirements of the specific application scenario.

[0042] In some examples, structural validation can be achieved through programmatic field checks: the validation program reads the tool call parameters returned by the small model and checks whether the analysis fields exist, whether the field type is string, and whether the string content contains all the components required by the preset structure. Optionally, the validation program can also set a lower limit on the content length of the analysis fields to prevent the model from outputting only blank placeholder content.

[0043] According to the technical solution provided in this embodiment, the problem of unstable instruction compliance in small models can be improved through structural verification and forced retry mechanisms. The verification mechanism ensures that the internal reasoning process is forcibly exposed, preventing small models from bypassing the analysis phase and directly outputting results. This guarantees the reliability of subsequent error attribution data and improves the robustness and industrial-grade reliability of the entire automated iterative optimization system for prompt words.

[0044] In some embodiments, before obtaining the test results, the method further includes: Receive target configuration information, which is used to add analysis fields with preset structures to the tool call in the tool definition.

[0045] Target configuration information can be represented as a modification instruction to the configuration file in the tool definition. This instruction forces the addition of an analysis field to the function call within the tool definition. In some examples, this analysis field can be set to a string type, and its internal structure can be explicitly specified in its description, for example, requiring the model to output the intent recognition, parameter extraction basis, and confidence level sequentially in this field.

[0046] The target configuration information can be configured to uniformly add analysis fields to all tool calls in the tool definition, requiring the small model to output explanatory analysis text before executing any tool call, thereby achieving complete exposure of the thought process across all business scenarios. Optionally, for some auxiliary tools with extremely high real-time requirements but relatively low interpretability requirements, analysis fields can be added only to the core business tools. Those skilled in the art can flexibly configure this according to the classification of business security levels. This embodiment does not impose specific limitations on this.

[0047] In some examples, the target configuration information is distributed once by relevant experts during the system initialization phase through a configuration interface or configuration file; in other embodiments, the target configuration information can also be automatically generated and distributed by the agent when it detects a new tool in the tool definition that does not have an added analysis field. These two methods can be used individually or in combination.

[0048] According to the technical solution provided in this embodiment, adding a pre-structured analysis field to the tool call within the tool definition can ensure the stability and standardization of the reasoning thought chain exposed within the small model, which is particularly suitable for intelligent manufacturing scenarios that require strong interpretability.

[0049] In some embodiments, error attribution is performed based on the explanatory analysis text of erroneous test cases in the test results, including: Analyze the intent identification content, parameter extraction basis, and confidence level in the explanatory text of the erroneous use cases; determine the error attribution of the erroneous use cases based on the intent identification content, parameter extraction basis, and confidence level; Among them, the intent recognition content can reflect the small model's understanding of the task intent of the current input, such as whether the model understands the current working condition as requiring a cleaning action or an alarm action; the parameter extraction basis can reflect the source of the input information on which the small model's output parameters are based, such as which sensor channel's value the model reads; and the confidence level can reflect the model's degree of certainty in making this judgment.

[0050] Based on the intent recognition content, parameter extraction basis, and confidence level, the error attribution of erroneous use cases can be comprehensively determined. For example, if the intent recognition content shows that the model is attempting to perform a cleaning action, but the expectation of this use case in the business test case set is to trigger an alarm, it is attributed to a logical judgment error; if the parameter extraction basis shows that the model reads incorrect sensor channel data, it is attributed to parameter extraction deviation; if the confidence level is extremely low, it indicates that the model lacks background knowledge of the current operating condition, which can be attributed to insufficient conceptual understanding or knowledge gaps.

[0051] Based on the error attribution, the initial prompt words are corrected to obtain corrected prompt words, including: Based on the error attribution, add at least one of the following to the initial prompt: a rule, a few-sample example, or a constraint.

[0052] The three correction methods are illustrated below.

[0053] Adding rules: When the error attribution indicates that the model lacks a criterion for judging a certain type of industrial concept, explicit judgment rules can be added to the initial prompts. For example, when the error attribution is that the model confuses cracks with scratches, and the attribution analysis shows that the confusion is due to the failure to distinguish between length and depth features, feature judgment rules that differentiate between cracks and scratches can be added to the initial prompts.

[0054] Adding few-shot examples: Few-shot examples refer to a small number of sample inputs and expected outputs embedded in the prompt words, used to demonstrate the correct judgment method. For example, when the misattribution shows that the model is unclear about the conceptual boundaries between qualified and unqualified products, few-shot examples that provide a comparison of standard descriptions of qualified and unqualified products can be added to the initial prompt words to help the small model establish the correct conceptual boundaries.

[0055] Adding constraints: Constraints are restrictive rules imposed on output behavior. For example, when an error attribution shows that an erroneous instruction is still forcibly output even when the confidence level is low, a constraint can be added to stipulate that a manual review should be triggered when there is uncertainty. That is, if there is insufficient certainty about the judgment result, it is better to transfer the use case to manual processing than to output a potentially erroneous automatic instruction, reflecting the industrial safety principle of "better to have less than to have something bad".

[0056] The three correction methods described above can be used individually or in combination. For example, for the same misattribution, both a distinction rule and corresponding few-sample examples can be added to strengthen the model's understanding through the synergy of rules and examples. The choice of the three correction methods can be driven by the type of misattribution: for conceptual confusion attributions, rules and few-sample examples should be prioritized; for parameter extraction bias attributions, parameter source rules should be prioritized; and for logical judgment errors attributions, process constraints should be prioritized.

[0057] According to the technical solution provided in this embodiment, the correction strategy based on the type matching of error attribution can accurately locate the defects in the initial prompt words, improve the pertinence and efficiency of prompt word correction, reduce unnecessary prompt word redundancy, adapt to the limited context window capacity of industrial small models, and ensure the running efficiency of the optimized prompt words.

[0058] In some embodiments, after obtaining the corrected prompt word, the process includes: The version number of the corrected prompt words, optimization direction labels, modified summaries, accuracy changes, and illusion rate changes are recorded in the change log; optimization direction labels include tool distribution labels to identify incorrect tool selection, parameter extraction labels to identify parameter extraction bias, illusion suppression labels to identify model fabrication, and boundary condition labels to identify boundary conditions.

[0059] Among them, the version number is used to uniquely identify the version of the prompt word after each correction; the optimization direction label is used to identify the error type targeted by this correction; the modification summary is used to record the specific content of this correction in concise text, such as what rules or examples were added; the accuracy change is used to record the difference in test accuracy before and after this correction; and the illusion rate change is used to record the difference in the proportion of test cases that fabricate factual content in the model output before and after this correction.

[0060] Specifically, this embodiment establishes a standardized optimization direction label mapping system. This optimization direction label specifically includes: a tool distribution label to identify incorrect tool selection, such as calling a detection tool when a tightening tool should be called; a parameter extraction label to identify parameter extraction deviations, such as extracting an incorrect screw angle; a hallucination suppression label to identify model fabrication, such as fabricating non-existent sensor readings; and a boundary condition label to identify boundary situations, such as improper handling of default behavior when input data is empty.

[0061] Each change log entry can be bound to one or more of the above tags, allowing each modification to be clearly categorized in terms of direction. By binding each modification to the specific optimization direction tag and recording the resulting changes in accuracy and illusion rate, a keyword optimization gene map can be formed. This allows for quick identification of which optimization directions have been tried and whose effectiveness is diminishing when debugging keyword suggestions, enabling timely strategy adjustments and avoiding redundant patching.

[0062] In addition, each record in the change log can be associated with the commit records of the version control tool. That is, after completing a prompt word correction, the corrected prompt word file is submitted to the version control system and the change log is updated, thereby enabling one-click retrospection of any historical version and comparison of differences between versions.

[0063] According to the technical solution provided in this embodiment, by combining standardized label classification with versioned change log management, a complete, traceable, and reproducible record can be provided for the iterative optimization process of prompt words. This reduces the disorder of debugging actions during multiple rounds of optimization, helps R&D personnel quickly sort out the optimization path, and improves the overall efficiency of prompt word iteration in industrial small models.

[0064] In some embodiments, the method further includes: If the accuracy of the test results does not improve after a preset number of consecutive tests, or if the preset intervention conditions are met, a target prompt message is triggered, which prompts for manual review. Confirmation information based on the manual review is received. In response to the confirmation information, the final optimized prompt words, business test case set, and tool definition are deployed to the production environment.

[0065] The preset number of consecutive iterations can be set to three consecutive rounds. This means that manual intervention is triggered when the accuracy fails to improve after three consecutive iterations. Preset intervention conditions could include, for example, unauthorized actions by the executing entity attempting to modify the test script or the model itself. The significance of this unauthorized monitoring is that the test script and business test case set are the evaluation benchmark. If modifications to the benchmark are allowed, the accuracy value will lose its objectivity. Therefore, any attempt to modify the benchmark should be intercepted and referred to manual judgment.

[0066] Relevant experts can review the contribution of each optimization direction label in the change log, that is, the cumulative change in accuracy and false alarm rate brought about by the correction of each label direction, and comprehensively confirm whether the optimized prompt words have achieved the preset performance and safety goals. As a specific example, the criteria for confirming compliance may include an accuracy rate greater than or equal to 95% and a false alarm rate of less than 1%, and compliance with factory safety regulations. The system then receives confirmation information based on manual review.

[0067] In response to this confirmation, the final optimized prompts, business test case sets, and tool definitions from the tool definitions are packaged together and deployed to the actual production environment. Optionally, a corresponding version tag can be generated during packaging and deployment, associating this deployment with the version number in the change log, facilitating quick location of the specific prompt version in case of anomalies in the production environment.

[0068] After deployment, regression tests can be run periodically in the production environment to continuously verify the behavioral stability of the private small model. Furthermore, when a new process is introduced, a new round of iterative optimization can be automatically triggered. This involves using the current production environment's prompts as new initial prompts and the business test case set supplemented with new process test cases as the new evaluation benchmark, re-entering the closed loop of execution, analysis, modification, and regression. This ensures the long-term operation of the system in continuous production.

[0069] According to the technical solution provided in this embodiment, by adding a manual triggering intervention mechanism and a closed-loop iteration process, it is possible to terminate invalid iterations in a timely manner when automated optimization reaches a bottleneck, avoiding unnecessary consumption of computing power and time. It is also possible to ensure that the optimized prompt words always meet the safety specifications and performance requirements of industrial production, while adapting to the actual needs of continuous process updates in intelligent manufacturing scenarios, and realizing dynamic optimization management of the entire life cycle of prompt words.

[0070] In some embodiments, before obtaining the initial prompt words and business test case set, the method further includes: Based on the original prompt words and business test case set, tests are conducted through a small model to obtain the original accuracy rate and original illusion rate corresponding to the original prompt words; the original accuracy rate and original illusion rate are used as the starting point for iteration.

[0071] The initial prompt can be a very simple prompt, containing only the most basic task description without any optimization rules. The initial prompt is tested, and its initial accuracy and initial illusion rate are statistically calculated. Then, the calculated initial accuracy and initial illusion rate are used as the starting point for the entire automated iterative optimization process.

[0072] The raw accuracy reflects the correctness of the small model's task completion, while the raw illusion rate reflects the risk level of the model's output fabricating factual content. In subsequent iterations, the test results of the initial prompt words and the business test case set can be compared with this starting point. Only when the accuracy of the initial prompt words is higher than the baseline can the iteration be considered valid.

[0073] According to the technical solution provided in this embodiment, using the performance of the unoptimized original prompt words as the iteration benchmark can provide a comparative reference for the effect of the initial optimization, avoid blind iteration without a goal, ensure that the iterative optimization actions are all centered around performance improvement, and ensure that the entire optimization process proceeds in an effective direction.

[0074] To better understand this application, the overall implementation process is illustrated below with a complete practical application example: Phase 1: Engineers or system engineers construct a factory business test case set. The business test case set (test_cases.json) is defined in JSON format and includes input data (e.g., quality inspection image descriptions, sensor logs, operator voice commands), the name of the tool to be invoked (e.g., the defect classification tool `classify_defect`, the robotic arm control tool `control_robot_arm`), and the parameters to be extracted (e.g., defect type, coordinates, torque value). It covers three core scenarios: normal production processes, abnormal operating conditions (e.g., sensor offline, blurred images), and boundary conditions (e.g., extremely small defects, critical thresholds). Subsequently, an initial, minimally coded set of prompts is run, and the initial accuracy and initial illusion rate are calculated as the starting point for iteration.

[0075] The second phase involves receiving target configuration information and forcibly adding analysis parameters to all function calls in the tool definition file (tools.json). These analysis parameters are of type String, and their descriptions specify the structure as intent recognition, parameter extraction basis, and confidence level. Simultaneously, the configuration request ensures that before the small model calls a tool (e.g., to determine if a part is qualified or to control the movement of a robotic arm), it must output explanatory analysis text conforming to this structure; otherwise, it is considered an illegal response or a forced retry.

[0076] Phase Three: The AI ​​agent automatically executes an iterative loop, which involves reading the current prompt (system_prompt.md) and the business test case set (test_cases.json); executing the test script (run_cases.py), generating the current round's result file (results / vN.json), and calculating accuracy, tool selection error rate, and parameter matching rate. When the target is not met, the agent parses the explanatory analysis text in the erroneous test cases to pinpoint the error attribution (e.g., identifying that the model confuses cracks and scratches due to a failure to distinguish between length and depth features, or identifying that the model mistakenly triggers a cleaning process instead of an alarm). Based on the error attribution, the agent corrects the current prompt, including adding at least one of the following: rules, few-sample examples, and constraints; marking the optimization direction of this correction; submitting the version record; and updating the change log. When the target is met, the agent outputs the final report and locks the version.

[0077] Phase Four: The agent automatically maintains the change log (CHANGELOG.md). Each record is required to include a version number, optimization direction label, change summary, accuracy change, and illusion rate change. Optimization direction labels are divided according to a standardized mapping system: tool distribution label (tool_dispatch) is used to address tool selection errors; parameter extraction label (param_extract) is used to address missing or incorrect parameters; illusion suppression label (anti_hallucination) is used to address model fabrication of facts; and edge case label (edge_case) is used to address improper handling of boundary conditions. This forms a gene map of prompt word optimization, facilitating later review and migration to similar production lines, and meeting the traceability requirements of the quality system for change management.

[0078] Phase 5: Monitor the iteration process. If accuracy fails to improve for three consecutive rounds, or if the agent attempts to modify test scripts or models, trigger a target prompt message to request manual intervention. Check the contribution of each tag direction in the change log to confirm whether the preset targets have been met, such as accuracy greater than or equal to 95%, false positive rate less than 1%, and compliance with factory safety regulations. After confirming compliance, the system responds with a confirmation message, packages and deploys the final version of optimization prompts, business test case sets, and tool definitions to the production environment, and runs regression tests regularly in the production environment; when a new process is introduced, a new round of agent iteration optimization is automatically triggered.

[0079] It should be understood that the order of the steps in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the process of the embodiments of this application.

[0080] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0081] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0082] Figure 2 This is a schematic diagram illustrating the structure of a prompt word optimization device for an industrial small-scale model, as shown in an exemplary embodiment of this application. Figure 2 As shown, the prompt word optimization device for this exemplary industrial mini-model includes: Data acquisition module 210 is configured to acquire initial prompt words and business test case sets; The first test module 220 is configured to perform tests through a small model based on the initial prompt words and business test case set, and obtain test results. The test results include the explanatory analysis text of the small model during internal reasoning. The explanatory analysis text is generated by the small model based on the analysis fields of the preset structure in the tool definition before executing the tool call command. The error correction module 230 is configured to, if the test result does not reach the preset accuracy rate, perform error attribution based on the explanation and analysis text of the error cases in the test result, and correct the initial prompt words based on the error attribution to obtain the corrected prompt words; The second test module 240 is configured to perform regression testing using a small model based on the corrected prompt words until a preset termination condition is met, thereby obtaining the final optimized prompt words.

[0083] In some embodiments, the data acquisition module 210 is further configured to perform structural verification on the interpretation analysis text and obtain the verification result; if the verification result indicates that the interpretation analysis text does not conform to the analysis field of the preset structure, the corresponding interpretation analysis text is marked as an illegal response and / or the tool call instruction is resent to the small model.

[0084] In some embodiments, the data acquisition module 210 is further configured to receive target configuration information, wherein the target configuration information is used to add analysis fields with a preset structure to the tool call in the tool definition.

[0085] In some embodiments, the error correction module 230 is further configured to parse the intent identification content, parameter extraction basis, and confidence level in the explanatory analysis text of the error use case; determine the error attribution of the error use case based on the intent identification content, parameter extraction basis, and confidence level; and add at least one of the following to the initial prompt word based on the error attribution: rules, few-sample examples, or constraints.

[0086] In some embodiments, the error correction module 230 is further configured to record the version number of the correction prompt, the optimization direction label, the modification summary, the accuracy change, and the illusion rate change in the change log; the optimization direction label includes a tool distribution label for identifying tool selection errors, a parameter extraction label for identifying parameter extraction bias, an illusion suppression label for identifying model fraud, and a boundary condition label for identifying boundary conditions.

[0087] In some embodiments, the error correction module 230 is further configured to trigger a target prompt message if the accuracy of the test results for a preset number of consecutive tests does not improve, or if a preset intervention condition is met. The target prompt message is used to prompt for manual review. The module 230 receives confirmation information based on the manual review. In response to the confirmation information, the module 230 deploys the final optimization prompt words, business test case set, and tool definition to the production environment.

[0088] In some embodiments, the data acquisition module 210 is further configured to perform tests using a small model based on the original prompt words and the business test case set to obtain the original accuracy rate and the original illusion rate corresponding to the original prompt words; and to use the original accuracy rate and the original illusion rate as the starting point for iteration.

[0089] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the methods provided in the above embodiments.

[0090] Figure 3 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 3 The computer system 300 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0091] like Figure 3 As shown, the computer system 300 includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 302 or programs loaded from storage portion 308 into Random Access Memory (RAM) 303, such as executing the methods described in the above embodiments. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An Input / Output (I / O) interface 305 is also connected to the bus 304.

[0092] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0093] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs various functions defined in the system of this application.

[0094] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation that may be implemented in systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0096] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0097] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0098] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods described in the various embodiments above.

[0099] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the steps of this application.

Claims

1. A method for optimizing prompt words in a small industrial model, characterized in that, include: Obtain the initial prompt words and business test case set; Based on the initial prompt words and the business test case set, tests are conducted through a small model to obtain test results. The test results include the explanatory analysis text of the small model during internal reasoning. The explanatory analysis text is generated by the small model based on the analysis fields of the preset structure in the tool definition before executing the tool call instruction. If the test results do not reach the preset accuracy rate, then the error is attributed based on the explanation and analysis text of the erroneous test cases in the test results, and the initial prompt words are corrected based on the error attribution to obtain the corrected prompt words; Based on the corrected prompt words, regression testing is performed using the small model until a preset termination condition is met, thus obtaining the final optimized prompt words.

2. The method according to claim 1, characterized in that, Before obtaining the test results, the process also includes: The structure of the explanatory and analyzed text is validated, and the validation results are obtained. If the verification result indicates that the interpretation analysis text does not conform to the analysis field of the preset structure, then the corresponding interpretation analysis text is marked as an illegal response and / or the tool call instruction is resent to the small model.

3. The method according to claim 1, characterized in that, Before obtaining the test results, the process also includes: Receive target configuration information, wherein the target configuration information is used to add analysis fields with a preset structure to the tool call in the tool definition.

4. The method according to claim 1, characterized in that, The error attribution based on the explanatory analysis text of the erroneous test cases in the test results includes: The analysis of the explanatory text of the error test cases reveals the intent identification content, parameter extraction basis, and confidence level. Based on the intent recognition content, the parameter extraction criteria, and the confidence level, the error attribution of the erroneous use case is determined; The step of correcting the initial prompt word based on the error attribution to obtain the corrected prompt word includes: Based on the error attribution, add at least one of the following to the initial prompt word: a rule, a few-sample example, or a constraint.

5. The method according to claim 1, characterized in that, After obtaining the corrected prompt word, the process includes: Record the version number of the corrected prompt, the optimization direction label, the modified summary, the accuracy change, and the hallucination rate change in the change log; The optimization direction labels include tool distribution labels for identifying incorrect tool selection, parameter extraction labels for identifying parameter extraction deviations, illusion suppression labels for identifying model fabrication, and boundary condition labels for identifying boundary conditions.

6. The method according to claim 1, characterized in that, The method further includes: If the accuracy of the test results does not improve after a preset number of consecutive tests, or if the preset intervention conditions are met, a target prompt message is triggered, which is used to prompt for manual review. Receive confirmation information based on manual review; In response to the confirmation information, the final optimized prompts, business test case sets, and tool definitions are deployed to the production environment.

7. The method according to any one of claims 1-6, characterized in that, Before obtaining the initial prompt words and business test case set, the process also includes: Based on the original prompt words and the business test case set, tests were conducted using a small model to obtain the original accuracy and original illusion rate corresponding to the original prompt words. The original accuracy rate and the original illusion rate are used as the starting point for iteration.

8. A prompt word optimization device for a small industrial model, characterized in that, include: The data acquisition module is configured to acquire initial prompt words and business test case sets; The first test module is configured to perform tests through a small model based on the initial prompt words and the business test case set, and obtain test results. The test results include the explanatory analysis text of the small model during internal reasoning. The explanatory analysis text is generated by the small model based on the analysis fields with a preset structure in the tool definition before executing the tool call instruction. The error correction module is configured to, if the test result does not reach a preset accuracy rate, perform error attribution based on the explanation and analysis text of the error cases in the test result, and correct the initial prompt word based on the error attribution to obtain a corrected prompt word; The second testing module is configured to perform regression testing on the small model based on the corrected prompt words until a preset termination condition is met, thereby obtaining the final optimized prompt words.

9. An electronic device, characterized in that, include: One or more processors and a memory, the memory storing a computer program that, when executed by the one or more processors, causes the device to perform the steps of the prompt word optimization method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, It stores a computer program that, when executed by one or more processors, causes the device to perform the steps of the prompt word optimization method as described in any one of claims 1 to 7.