Business rule cue word generation method and device, electronic equipment, medium and program product

Abstract

The invention provides a business rule cue word generation method and device, electronic equipment, a medium and a program product, relates to application of a large model in a financial science and technology scene, and can be applied to the technical field of artificial intelligence and the technical field of big data. The method comprises the steps of obtaining business demand information submitted based on a target interactive interface, and converting the business demand information into structured demand data; generating an initial cue word based on the structured demand data; inputting the initial cue word and the test data into the large model to obtain a first output result; comparing the first output result with a corresponding expected label, identifying a difference sample, and generating a cue word variant based on the difference sample; and utilizing the cue word variant to drive the large model to carry out iterative verification on the test data until the verification performance meets a preset performance condition, and outputting the service rule cue word.

Description

Technical Field

[0001] This application relates to the application of large models in fintech scenarios, which can be applied to the fields of artificial intelligence, big data and fintech. More specifically, it relates to a method, apparatus, device, medium and program product for generating business rule prompts. Background Technology

[0002] With the development of artificial intelligence and large language model technology, more and more business scenarios are beginning to try to use models to automatically generate and execute rules in order to improve the efficiency of approval, risk control, and compliance processing. The advantage of this type of method is that it can reduce the cost of manual coding, allowing non-technical personnel to directly participate in rule building.

[0003] However, in practical applications, existing technologies still rely primarily on natural language prompts, small-sample testing, and error correction, and have many shortcomings overall: complex logic is difficult to express accurately in natural language, industry experience-based knowledge is difficult to translate explicitly, and error attribution is vague, making it difficult to distinguish between rule, model, or data problems; at the same time, business personnel generally lack structured logic and data sensitivity, are prone to setting unreasonable thresholds and are accustomed to vague expressions, resulting in risky rules or rules that machines cannot recognize. Summary of the Invention

[0004] In view of the above problems, this application provides a method, apparatus, device, medium and program product for generating business rule prompts.

[0005] According to a first aspect of this application, a method for generating business rule prompts is provided. The method includes: acquiring business requirement information submitted based on a target interactive interface; converting the business requirement information into structured requirement data, wherein the target interactive interface is used to support structured rule expressions in multiple input forms; generating initial prompts based on the structured requirement data; inputting the initial prompts and test data into a large model to obtain a first output result; comparing the first output result with the corresponding expected label, identifying difference samples, and generating prompt variants based on the difference samples; and using the prompt variants to drive the large model to perform iterative verification on the test data until the verification performance meets preset performance conditions, and outputting the business rule prompts.

[0006] According to an embodiment of this application, obtaining business requirement information submitted based on a target interactive interface includes at least one of the following: selecting target fields through a visual form and defining target parameters corresponding to business rules; uploading positive and negative example data through the visual form, wherein the positive and negative example data are used to indicate the boundary conditions of the business rules; and setting threshold rules in the visual form, wherein the threshold rules are used to constrain numerical judgment logic or conditional judgment logic.

[0007] According to an embodiment of this application, the step of converting the business requirement information into structured requirement data includes: parsing the format and mapping the fields of the business requirement information to generate a structured field set; and performing integrity and consistency checks on the structured field set, filling in missing fields or removing outliers to generate the structured requirement data.

[0008] According to an embodiment of this application, the step of comparing the first output result with the corresponding expected label, identifying difference samples, and generating prompt word variants based on the difference samples includes: marking difference points and calculating the corresponding difference rate based on the comparison result between the first output result and the expected label; and generating prompt word variants based on the difference points in response to the difference rate exceeding a first preset threshold.

[0009] According to embodiments of this application, the generation of the prompt word variant includes at least one of the following: adjusting the order of the business rules, supplementing the prompt word with counterexamples, and modifying the natural language expression of the business rules.

[0010] According to an embodiment of this application, the step of using the prompt word variant to drive the large model to perform iterative verification on the test data until the verification performance meets a preset condition, and then outputting the business rule prompt word, includes: after each round of prompt word variant optimization, using a reserved test set to verify the prompt word variant of the current round, and calculating the corresponding performance evaluation index; in response to the change in the performance evaluation index of the current round compared to the performance evaluation index of the previous round exceeding a second preset threshold, using the prompt word variant of the current round as a candidate prompt word; and in response to the performance evaluation index of the candidate prompt word meeting the preset performance condition, determining the candidate prompt word as the business rule prompt word.

[0011] According to an embodiment of this application, the method further includes: terminating iterative verification and generating verification feedback information in response to the fact that the change amplitude has not reached the second preset threshold for K consecutive rounds, wherein the verification feedback information includes at least a set of information on differential samples and insufficient coverage of business rules, and K is a positive integer not less than 2.

[0012] According to an embodiment of this application, the method further includes: terminating iterative verification in response to a preset resource constraint exceeding a corresponding boundary value, wherein the preset resource constraint includes at least one of the following: a maximum number of large model calls, a computational resource budget, and a verification latency threshold.

[0013] According to an embodiment of this application, the method further includes: generating explanatory feedback information associated with the business rule prompt word, the explanatory feedback information including at least one of the following: differential sample tracking information for characterizing the prompt word generation process, a performance evaluation index change curve for indicating iterative verification status, and a visual rule diagram for displaying rule logic.

[0014] A second aspect of this application provides a device for generating business rule prompts, the device comprising: a data acquisition module, configured to: acquire business requirement information submitted based on a target interactive interface, and convert the business requirement information into structured requirement data, wherein the target interactive interface is used to support structured rule expressions in multiple input forms; an initial prompt processing module, configured to: generate initial prompts based on the structured requirement data; input the initial prompts and test data into a large model to obtain a first output result; a prompt variant generation module, configured to: compare the first output result with the corresponding expected label, identify difference samples, and generate prompt variants based on the difference samples; and a business rule prompt generation module, configured to: use the prompt variants to drive the large model to perform iterative verification on the test data until the verification performance meets preset performance conditions, and output the business rule prompts.

[0015] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0016] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0017] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0018] According to embodiments of this application, a structured interactive interface supports multiple input formats, avoiding semantic ambiguity and logical risks associated with purely natural language descriptions. This allows business personnel to accurately express rule requirements in an intuitive and low-barrier manner, thereby improving user experience. Secondly, initial prompt words are generated based on structured requirement data, and the automatic comparison between the large model inference results and expected labels accurately identifies discrepancies and generates prompt word variants, effectively reducing manual debugging and trial-and-error steps and enabling rapid iterative optimization of prompt words. Furthermore, an iterative verification mechanism is used to evaluate the performance of prompt word variants, and the final prompt word is output when preset performance conditions are met. This not only ensures the accuracy and stability of the generated results but also avoids invalid calculations by introducing performance threshold control, thereby reducing computational resource consumption. Attached Figure Description

[0019] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0020] Figure 1 The illustrations depict application scenarios of a method, apparatus, device, medium, and program product for generating business rule prompts according to embodiments of this application.

[0021] Figure 2 A flowchart illustrating a method for generating business rule prompts according to an embodiment of this application is shown schematically;

[0022] Figure 3 The flowchart illustrates a method for generating cue word variants based on differential samples according to some exemplary embodiments of this application.

[0023] Figure 4 The flowchart schematically illustrates a method for iteratively verifying test data using cue word variations according to some exemplary embodiments of this application;

[0024] Figure 5 This schematically illustrates a structural block diagram of a business rule prompt generation apparatus according to an embodiment of this application; and

[0025] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a method for generating business rule prompts according to an embodiment of this application. Detailed Implementation

[0026] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0029] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0030] The following will explain and clarify the terms used in this article.

[0031] Prompts are textual information or instructions input to a large language model to guide the model in generating expected output. In this application, prompts are primarily used to describe the logical conditions and output format of business rules, and their quality directly affects the accuracy and controllability of the model's generated results.

[0032] Cue word variants are rewritten versions of existing cue words, obtained by adjusting the rule order, adding counterexamples, or modifying the wording. The generation of cue word variants aims to optimize for discrepancies in samples, thereby improving the accuracy of the model's output.

[0033] Reinforcement learning is a machine learning method that continuously adjusts a strategy to maximize cumulative rewards through a "state-action-reward" interaction process. In this application, reinforcement learning is used to guide the iterative optimization of cue words: the system generates cue word variants based on feedback from differential samples and evaluates the optimization effect through performance metrics (such as precision, recall, F1 score, etc.), thereby achieving the dynamic evolution of cue words.

[0034] Performance evaluation metrics are measures used to quantify the performance of prompt words on a test set, and may include precision, recall, F1 score, etc.

[0035] With the rapid development of artificial intelligence technology, large language models are gradually being applied in scenarios such as business rule construction, automated approval, and compliance management. The basic idea is to drive the model with prompt words to generate rule judgment results that conform to business logic, thereby reducing the cost of manual rule writing. However, the current implementation path is still relatively preliminary, mainly employing a cyclical approach of natural language rule description, small-sample testing, and error correction.

[0036] In some embodiments, business personnel first focus on the core decision logic and use natural language to write concise rule prompts. For example, in a credit approval scenario, business personnel might describe the logic as "reject the order when the order amount exceeds 100,000 yuan and the customer's credit score is below 600," and specify the task objective, required fields, and output format in the prompt, such as requiring the approval result to be output in the form of structured key-value pairs. In this way, business personnel can quickly build a basic rule framework without needing programming skills.

[0037] After rule construction is completed, existing solutions typically introduce a small-scale real-world dataset for testing. This dataset generally contains 20 to 30 typical business data points, covering normal, boundary, and abnormal scenarios, to observe the model's performance under different input conditions. The testing phase mainly focuses on three aspects: first, whether compliant samples are misjudged, such as a customer with a credit score of 599 being wrongly rejected when they should actually be approved; second, whether special scenarios are overlooked, such as the failure to correctly identify large customer whitelists; and third, whether the output results conform to the preset format, such as the model returning natural language text instead of a restricted structured format.

[0038] After obtaining the test results, business personnel also need to conduct attribution analysis on the error cases to try to determine the source of the errors. For example, error types may include: risky rule descriptions (e.g., not defining the handling method when the credit score is equal to 600), ambiguous semantic expressions (e.g., not specifying the specific province range for "high-risk areas"), and lack of data fallback logic (e.g., logic interruption caused by an empty industry field). Such attribution can provide direction for subsequent rule revisions.

[0039] Finally, existing solutions typically undergo targeted iterative improvements, refining the rules item by item to address identified issues. Specific measures include: supplementing rule details (e.g., "If a customer belongs to a membership group, the order amount threshold is increased to 200,000 yuan"), adding counterexamples (e.g., "When the industry field is empty, it is considered manufacturing by default"), strengthening format constraints (e.g., adding "No additional interpretation is allowed" at the end of the prompt), and providing typical input / output examples for model reference. Through multiple iterations, the stability and accuracy of rule execution can be improved to some extent.

[0040] While the aforementioned methods have value in rapidly building prototypes, they also have some drawbacks. First, there is an inherent bottleneck in rule expression. Complex business logic often involves multi-layered nested conditions, dynamic threshold calculations, and cross-field dependencies, which are difficult to describe precisely using only natural language, easily leading to ambiguity and causing the model's output to deviate from business expectations. Second, the tacit knowledge accumulated by business personnel over a long period cannot be fully and explicitly conveyed. For example, "key customers" often involves comprehensive judgments about industry background, customer relationships, and historical behavior. This type of knowledge is difficult to translate into machine-executable rules through simple descriptions, causing the model to misjudge when lacking industry common sense. Furthermore, the attribution of errors is ambiguous. The current process struggles to clearly distinguish whether an error stems from incomplete rule descriptions, insufficient model understanding, or inadequate training data coverage. This makes the correction process often rely on trial and error, lacking systematicity and traceability, resulting in low iteration efficiency.

[0041] At the practical level, the skill set of business personnel also limits the effectiveness of this solution. Most business personnel are accustomed to expressing requirements using experiential, natural language, lacking the ability to break down structured logic, which can easily lead to logical risks in the rules. For example, some boundary conditions may not be defined, or there may be conflicts between different rules. Furthermore, due to a lack of data analysis background, business personnel are prone to deviating from the actual data distribution when setting thresholds, even mistakenly treating extreme outliers as universal standards, resulting in unreasonable rules. Moreover, business personnel often use vague concepts such as "key customers" or "high-risk areas," failing to translate them into clear, machine-recognizable expressions, thus causing the model to fail to identify privileged customers or miss high-risk groups.

[0042] Based on this, embodiments of this application provide a method for generating business rule prompts. The method includes: acquiring business requirement information submitted based on a target interactive interface; converting the business requirement information into structured requirement data, wherein the target interactive interface is used to support structured rule expressions in multiple input forms; generating initial prompts based on the structured requirement data; inputting the initial prompts and test data into a large model to obtain a first output result; comparing the first output result with the corresponding expected label, identifying difference samples, and generating prompt variants based on the difference samples; and using the prompt variants to drive the large model to perform iterative verification on the test data until the verification performance meets preset performance conditions, and outputting the business rule prompts. According to embodiments of this application, a structured interactive interface supports multiple input formats, avoiding semantic ambiguity and logical risks associated with purely natural language descriptions. This allows business personnel to accurately express rule requirements in an intuitive and low-barrier manner, thereby improving user experience. Secondly, initial prompt words are generated based on structured requirement data, and the automatic comparison between the large model inference results and expected labels accurately identifies discrepancies and generates prompt word variants, effectively reducing manual debugging and trial-and-error steps and enabling rapid iterative optimization of prompt words. Furthermore, an iterative verification mechanism is used to evaluate the performance of prompt word variants, and the final prompt word is output when preset performance conditions are met. This not only ensures the accuracy and stability of the generated results but also avoids invalid calculations by introducing performance threshold control, thereby reducing computational resource consumption.

[0043] It should be noted that the methods, apparatus, devices, media, and program products for generating business rule prompts as defined in this application relate to the application of large-scale models in fintech scenarios. They can be used in the fields of artificial intelligence, big data, and fintech, and also in various other fields besides these. The application areas of the methods, apparatus, devices, media, and program products for generating business rule prompts provided in the embodiments of this application are not limited.

[0044] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0045] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this application all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0046] Figure 1 The illustration schematically depicts an application scenario diagram of a method, apparatus, device, medium, and program product for generating business rule prompts according to embodiments of this application.

[0047] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0048] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0049] In the embodiments of this application, the first terminal device 101 can be an example of the first device, and the second terminal device 102 and / or the third terminal device 103 can be an example of at least one second device. The first device and the second device can communicate collaboratively through an internal client mechanism to implement the data distribution and rendering logic described in the method for generating business rule prompts.

[0050] In some embodiments, the first device and at least one second device may be different display modules, windows or screens on the same computing terminal (such as a host), or multiple physical devices that work together through a network, such as different client instances deployed on a desktop computer, tablet terminal or mobile device respectively.

[0051] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smart mobile terminals, tablet computers, laptop computers, and desktop computers.

[0052] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0053] It should be noted that the method for generating business rule prompts provided in this application embodiment can generally be executed by server 105. Correspondingly, the device for generating business rule prompts provided in this application embodiment can generally be located in server 105. The method for generating business rule prompts provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the device for generating business rule prompts provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0054] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0055] The following will be based on Figure 1 The described scene, through Figures 2-4 The method for generating business rule prompts according to the disclosed embodiments is described in detail.

[0056] Figure 2 A flowchart illustrating a method for generating business rule prompts according to an embodiment of this application is shown.

[0057] like Figure 2 As shown, the business rule prompt word generation method 200 of this embodiment includes operations S210 to S240.

[0058] In operation S210, business requirement information submitted based on the target interactive interface is obtained, and the business requirement information is transformed into structured requirement data. The target interactive interface is used to support structured rule expression in multiple input forms.

[0059] In the embodiments of this application, the target interactive interface can be a web-based visual operation interface, a configuration panel integrated into an enterprise's internal management system, or an application programming interface (API) or command-line interactive tool for developers. Through this interactive interface, users can input or configure business rules in an intuitive way without relying on complex programming statements.

[0060] Through the target interactive interface, users can input business rule-related requirement information in various forms. This business requirement information can include key fields (such as order amount and customer credit score), sample data (such as positive and negative examples from historical transaction records), and threshold parameters (such as upper limit of amount and lower limit of credit score). After obtaining the above information, the system can use rule parsing to transform it into structured requirement data to ensure consistency and accuracy in subsequent processing.

[0061] In some embodiments, business requirement information can originate from natural language descriptions directly input by the user, such as "Reject the order when the order amount is greater than 100,000 and the customer's credit score is lower than 600". The target interface, through its built-in natural language parser, can automatically identify the condition fields, threshold parameters, and logical relationships in the description and convert them into standardized business requirement information. For example, "order amount greater than 100,000" is mapped to a comparison between the field "order amount" and the numerical threshold "100,000", and "customer's credit score lower than 600" is mapped to a judgment relationship between the field "credit score" and the numerical threshold "600".

[0062] In some embodiments, business requirement information can be submitted by uploading files, such as comma-separated values ​​(CSV), spreadsheets, or exported database files. Users can upload data files containing typical transaction samples. The target interface automatically extracts positive and negative examples from these files and combines them with threshold conditions set by the user on the target interface to form complete business requirement information. Furthermore, the system not only parses the input data for formatting but also performs field mapping and validates and processes missing or outlier values ​​to ensure the consistency and usability of the generated structured requirement data.

[0063] In some embodiments, the system supports collaborative input and version management. Different business personnel can submit their respective business requirements through a target interface; for example, one person might submit a monetary threshold, while another submits an industry classification field. The system can integrate, deduplicate, and perform consistency checks on this information, generating unified structured requirement data. Simultaneously, the system can generate corresponding version tags for different input versions to facilitate subsequent comparison and traceability.

[0064] In some embodiments, the system can also incorporate industry knowledge bases or historical rule bases to assist users in inputting business requirement information. For example, in a financial risk control scenario, fields such as "amount threshold," "credit score," and "high-risk area list" can be pre-set on the target interaction interface, allowing users to complete their input simply by checking or adjusting the selections. In an e-commerce promotion scenario, conditions such as "order volume," "discount amount," and "user level" can be pre-set on the target interaction interface, allowing users to quickly submit their business requirement information by modifying the default values.

[0065] It's important to note that although the business requirements submitted by business personnel may already be in structured or semi-structured formats such as forms, data tables, or threshold rules, the system still needs to perform standardized parsing and field mapping to ensure subsequent processing under a unified data model. Specifically, inputs from different sources may have inconsistent field names, format differences, or different numerical units. The system identifies semantic consistency through a standardized parsing process and maps these to an internally unified set of standard fields, thereby avoiding logical conflicts or rule ambiguities during subsequent prompt generation and large-scale model invocation.

[0066] In operation S220, an initial prompt word is generated based on the structured requirement data; the initial prompt word and test data are input into the large model to obtain the first output result.

[0067] In the embodiments of this application, after obtaining the structured requirement data, initial prompts can be generated first using a preset template. For example, the preset template can contain a general prompt framework, such as descriptions of input fields, threshold conditions, and output formats. The system can fill the fields and parameters from the structured requirement data into this framework to generate initial prompts. For example, when the structured requirement data contains the fields "order amount > 100,000" and "credit score < 600", the system can automatically generate prompts, explicitly requiring the large model to make judgments based on this logic and output results conforming to the structured format.

[0068] In some embodiments, the generation of initial prompts is not limited to fixed templates and can also be achieved through a rule combination engine. The system can dynamically combine conditional expressions based on the logical relationships between fields in the structured requirement data. For example, when business requirement information includes monetary thresholds, credit scores, and industry fields simultaneously, the system can generate prompts with multi-level logical nesting to cover more complex business rules. Thus, the generated prompts not only contain basic conditions but can also include format constraints or boundary handling instructions, enabling the model to execute rule logic more accurately when processing test data.

[0069] In some embodiments, the generation of initial prompt words can be enhanced by incorporating historical experience data or a knowledge base. While parsing structured requirement data, the system can retrieve historical prompt word case libraries, select prompt word fragments highly relevant to current business needs, and integrate them with newly input data. For example, for financial risk control, existing "high-risk area identification" prompt fragments from the knowledge base can be called and automatically appended to the newly generated prompt words, improving the comprehensiveness and stability of the prompt words.

[0070] In some embodiments, the system may also utilize machine learning or natural language generation models to assist in generating initial prompt words. Structured requirement data, as input features, is fed into a trained prompt word generation model, which can automatically generate natural language-described prompt words based on established patterns and historical data. For example, when a new field, "membership level," appears in an e-commerce business, the system can automatically learn and incorporate this field into the prompt word logic through the prompt word generation model, without requiring manual modification of the template.

[0071] In some embodiments, the system can employ different interaction methods when inputting initial prompts and test data into the large language model. The test data can be pre-defined standard samples or dynamic data sampled in real-time from the business system. The system encapsulates the initial prompts and this test data into an input sequence, calls the large language model for inference, and obtains the first output result. Furthermore, the system can set output constraints for the large language model, such as specifying that the output is in key-value pair form or a Boolean value judgment, to ensure that the result can be directly parsed and used by downstream modules.

[0072] In some embodiments, after generating initial prompts, the system can pre-validate them to ensure their quality. During this process, a small amount of test data can be input into the large language model to check if the output meets format requirements or logical expectations. If there are significant deviations in the output, the prompt word descriptions can be automatically adjusted or supplementary examples can be provided to improve the effectiveness of the prompts in formal batch validation.

[0073] In operation S230, the first output result is compared with the corresponding expected label to identify the difference sample, and a prompt word variant is generated based on the difference sample.

[0074] In the embodiments of this application, after obtaining the first output result generated by the large language model, the system can compare it with the pre-labeled test data labels one by one. The comparison results are used to identify the difference samples between the output and the labels. The difference samples may include instances of classification errors, instances of logical judgment deviations, and instances of output formats that do not meet expectations.

[0075] In other embodiments, the system not only identifies differing samples during the comparison process but also performs quantitative analysis of the differences. For example, the system can calculate the proportion of differing samples in the entire test data or calculate the false negative rate and false positive rate for a certain field. If a certain difference rate exceeds a set threshold, the optimization module can be triggered to generate cue word variants. During this process, cue word variants can be generated in various ways to better conform to the understanding habits of the large language model.

[0076] In some embodiments, the system can incorporate historical optimization experience or a knowledge base to assist in generating prompt word variants. Specifically, after detecting a discrepancy sample of a certain type, the system can retrieve similar optimization records from the historical case library and automatically embed the corresponding correction rules into the prompt words. For example, when the system identifies a discrepancy sample where "high-risk areas are not correctly identified," it can automatically introduce the standardized description of "high-risk area list" from the knowledge base and supplement it into the prompt words to form a new prompt word variant.

[0077] In some embodiments, the identification of discrepancies is not limited to comparing logical results, but may also include verifying the output structure and format. For example, when the expected label requires the model to output standardized JavaScript Object Notation (JSON), and the first output contains redundant explanatory text or lacks necessary key-value pairs, the system can also mark it as a discrepancy. The corresponding prompt word variant will add explicit format constraints to the end of the original prompt word, such as "Please strictly output results conforming to the JSON specification, and do not add extra explanations," to ensure that the model output meets the parsing requirements.

[0078] Furthermore, the system can utilize automated data augmentation techniques when generating prompt word variants. The system can synthesize new negative example data based on discrepancy samples and incorporate these negative example scenarios into the prompt words as examples. For instance, when a discrepancy sample is identified stating "error processing occurs when credit score equals the boundary value of 600," the system automatically generates positive and negative examples under this boundary condition and adds the instruction "pass should be determined when credit score equals 600" to the prompt word. In this way, prompt word variants can better cover boundary cases, improving the robustness of large language models.

[0079] In the embodiments of this application, reward signals can be generated for the optimization process based on the discrepancy samples. For example, a positive reward is given when the number of discrepancy samples decreases, and a negative reward is given when the number of discrepancy samples increases. Subsequently, a reinforcement learning (RL) mechanism can be used to dynamically adjust the cue word representation based on the reward signal, forming new cue word variants. Thus, the cue word optimization process no longer relies on manually set fixed rules, but gradually converges to the optimal representation that minimizes the number of discrepancy samples through policy iteration.

[0080] In another embodiment, after identifying discrepancy samples, the system can input them as part of the training state into a reinforcement learning agent. The reinforcement learning agent selects different optimization actions through a policy network, such as adjusting the rule order, adding counterexamples, or rewriting boundary condition statements. After each optimization action is executed, the system can again call the large language model to generate results and compare them with the expected labels. Changes in the discrepancy rate are returned to the policy network as reward feedback, thereby continuously updating the optimization policy. After multiple iterations, the reinforcement learning agent can automatically learn to select the most effective way to generate cue word variants, achieving continuous improvement in overall performance.

[0081] In operation S240, the large model is driven by the variant of the prompt word to perform iterative verification on the test data until the verification performance meets the preset performance conditions, and then the business rule prompt word is output.

[0082] In the embodiments of this application, after generating the prompt word variant, the system can re-input it into the large language model and verify it in conjunction with test data. During the verification process, the output results can be evaluated using the test data, and the optimization effect can be calculated based on performance evaluation metrics. When the performance metrics of the prompt word variant are detected to be significantly improved compared to the previous round of prompt words, it can be retained as a candidate version; if the candidate version reaches the preset performance conditions in subsequent iterations, such as the accuracy or F1 value exceeding a specified threshold, the prompt word is output as the final business rule prompt word.

[0083] In some embodiments, the system can introduce dynamic performance thresholds during iterative verification. Specifically, verification conditions can be automatically adjusted based on the distribution characteristics of the test set or the importance of the business scenario. For example, in scenarios with a large number of boundary samples, the weight of recall can be increased; while in high-risk transaction scenarios, the weight of precision can be increased.

[0084] In some embodiments, during each iteration of verification, the system not only verifies the performance of the prompt words on the overall test set but also performs segmented verification. Segmented verification involves dividing the test data into different subsets based on features, such as region, user category, or time period, and evaluating the performance of the prompt word variants on each subset separately. When the performance of certain subsets fails to meet the standards, new prompt word variants can be generated specifically for optimizing those subsets. The final output business rule prompt words ensure overall performance meets the standards while also taking into account the performance in local scenarios.

[0085] In some embodiments, the system uses a reinforcement learning (RL) mechanism to guide optimization during iterative validation. Specifically, performance improvements in each validation round can be used as a reward signal, rewarding cue word variants that reduce discrepancies while penalizing variants with poor performance. As the iteration process continues, the system's policy network gradually learns to select better cue word adjustment methods, thereby accelerating cue word convergence and ultimately outputting business rule cue words that meet performance requirements.

[0086] In some embodiments, when validating prompt word variants, the system can also combine adversarial data or automatically generated boundary samples to test the robustness of the prompt words under extreme conditions. For example, when the system identifies boundary conditions such as "credit score equals threshold" or "amount close to upper limit," it can automatically generate corresponding test data and input it into the verification process. If the performance of the prompt word variant on these boundary data meets preset conditions, it indicates that the prompt word has stronger generalization ability and can be used as the final output business rule prompt word.

[0087] According to embodiments of this application, various forms of business requirement information submitted by business personnel are obtained through the target interactive interface and transformed into unified structured requirement data. This effectively avoids semantic ambiguity and logical risks inherent in natural language expressions, ensuring the consistency and usability of input information. Initial prompt words are generated based on the structured requirement data, and combined with test data, they are input into a large language model to the first output result, realizing the automated translation of rule intent into executable prompt words, significantly reducing the workload of manual writing and debugging. By comparing the output result with the expected label, difference samples are identified, and prompt word variants are generated using the difference samples, thereby achieving automatic iterative optimization of the prompt words. The prompt word variants are iteratively verified until the performance evaluation indicators meet the preset conditions, at which point the final prompt word is output. This not only ensures the accuracy and stability of the optimization results but also improves the robustness and computational efficiency of the system by setting performance thresholds, resource constraints, and adversarial testing.

[0088] The method for generating business rule prompts according to the present application will be specifically described below by way of preferred embodiments.

[0089] In the embodiments of this application, the process of obtaining business requirement information submitted based on the target interactive interface can be implemented through a visual form. Specifically, business personnel can select target fields in the interface, such as "order amount", "credit score", "customer level", etc., and can further define corresponding business rule parameters, such as setting "order amount" to more than 100,000 yuan and "credit score" to less than 600.

[0090] In another embodiment, business personnel can upload positive and negative example data through a visual form to help the system better understand the boundary conditions of the rules. For example, in credit approval, positive example data could be historical customer samples that meet the approval conditions, while negative example data could be samples that do not meet the conditions. After receiving these samples, the system can automatically extract their feature information and map it into part of the structured requirement data.

[0091] In another embodiment, business personnel can directly set threshold rules in a visual form to constrain numerical or conditional logic. For example, users can set rules such as "additional review is required when the order amount is greater than 100,000" or "reject the order when the customer's credit score is below 600." They can also set conditional threshold logic, such as "default to manufacturing when the customer's industry field is empty." These threshold rules can be transformed into a unified structured expression in the system for subsequent automatic generation of prompts.

[0092] It should be noted that the above input methods can also be used in combination. Business personnel can describe the core rules by selecting fields and setting parameters, upload positive and negative examples to supplement the sample boundaries, and add threshold conditions to enhance logical control.

[0093] By employing the above methods, the risks of semantic ambiguity and logical problems are effectively reduced, significantly improving the accuracy and completeness of rule descriptions. Furthermore, multiple input methods can be used individually or in combination, enabling the system to uniformly parse and map information from different sources, thereby generating standardized, structured requirement data.

[0094] In the embodiments of this application, after receiving business requirement information submitted by business personnel through the target interactive interface, format parsing can be performed to identify the category and semantics of the input information. For example, when business personnel input data through a form, the system can read the form items and identify their corresponding business meanings.

[0095] During the generation of a structured field set, field mapping operations can be performed to unify fields from different sources and with different representations. For example, data from different departments may use different field names to represent the same indicator. Through the field mapping mechanism, the system can uniformly map these to standardized fields, ensuring consistency in subsequent processing.

[0096] After obtaining the structured field set, further integrity and consistency checks can be performed. Integrity checks include checking for missing required fields, such as order number, amount, or credit score; consistency checks include detecting contradictory values ​​for the same field. For example, if the same user's credit score is labeled as both 580 and 720 in the data, the system will automatically identify this as inconsistency and trigger the exception handling mechanism.

[0097] In some embodiments, the system can perform completion operations when a field is found to be missing, such as filling the missing field with default values, historical data interpolation, or external knowledge base reasoning; and can perform removal or correction operations when outliers are detected. For example, if a "credit score" is detected to be negative or outside a reasonable range, the data can be removed or replaced with a preset boundary value. After the above processing, the system can ensure that the structured requirement data is both complete and reasonable, providing reliable input for subsequent prompt word generation and large language model invocation.

[0098] Optionally, the system can also incorporate data standardization and version management mechanisms to enhance the robustness of data processing. Specifically, structured requirement data that has been parsed, mapped, validated, and corrected can be stored as a unified data model version, ensuring that subsequent operations are always performed based on the same version. When business personnel update requirements in the interactive interface, a new round of parsing and mapping processes can be automatically triggered, generating a new version number to maintain traceability and consistency during rule evolution.

[0099] In the embodiments of this application, after the large language model generates the first output result, it can be compared with the corresponding expected labels one by one to determine the differences between the output and the labels.

[0100] Figure 3 The flowchart illustrates a method for generating cue word variants based on differential samples according to some exemplary embodiments of this application.

[0101] like Figure 3 As shown, the method for generating cue word variants based on differential samples includes operations S310 to S320.

[0102] In operation S310, based on the comparison results of the first output result and the expected label, the difference points are marked and the corresponding difference rate is calculated.

[0103] In embodiments of this application, the comparison process may include determining logical correctness and verifying output format conformity. For example, when the expected output is JSON format but the actual output is plain text, the system will mark the result as a difference point. While marking these difference points, the system can further calculate the difference rate, for example, by dividing the number of difference samples by the total number of samples to obtain the overall difference ratio.

[0104] In some embodiments, the labeling of discrepancies can also relate to the accuracy of business logic. For example, in a credit approval scenario, if the expected label is "Approved" but the model outputs "Rejected," the sample can be marked as a misjudgment discrepancy. In a boundary condition scenario, such as when a customer's credit score equals 600 and should be judged as approved, but the model outputs "Rejected," the system can mark it as a boundary discrepancy. These discrepancies can be categorized and recorded, and used as the basis for generating prompt word variations.

[0105] When calculating the discrepancy rate, different statistical dimensions can be used for different business scenarios. For example, for transaction risk control scenarios, the system can calculate the false negative rate (i.e., the proportion of high-risk samples that are not identified); for recommendation system scenarios, the coverage rate (i.e., the proportion of prompts that can cover all user categories) can be calculated.

[0106] In operation S320, in response to the difference rate exceeding a first preset threshold, a prompt word variant is generated based on the difference point.

[0107] Specifically, the first preset threshold can be set by the system based on historical task experience, for example, 5%. This means that when the number of erroneous samples exceeds 5% of the total number of samples, the current prompt word is considered to have a significant problem and needs optimization. This avoids overreacting to random errors and ensures the stability of the optimization process.

[0108] In some embodiments, the first preset threshold can also employ a dynamic adjustment mechanism. During system operation, the threshold can be automatically adjusted based on the size of the test set and fluctuations in the difference rate. For example, when there are few test samples, the threshold can be automatically increased to avoid frequent optimizations triggered by small sample errors; while with a large test set, the threshold can be decreased to detect deviations in the prompt word logic earlier.

[0109] The above approach not only accurately labels discrepancies such as logical errors, misjudged boundary conditions, and inconsistent output formats, but also calculates the discrepancy rate and compares it with a preset threshold, automatically triggering an optimization process when the discrepancy rate exceeds the threshold. Simultaneously, through discrepancy sample-driven generation of prompt word variants, the system can quickly correct issues such as missing boundary rules, unreasonable order, or unclear expression, effectively reducing misjudgments and omissions, and improving the model's accuracy and stability in business scenarios.

[0110] In embodiments of this application, the generation of prompt word variants includes at least one of the following: adjusting the order of business rules, supplementing counterexamples in the prompt words, and modifying the natural language expression of the business rules.

[0111] Specifically, when generating prompt word variations, the model's output can be optimized by adjusting the order of business rules. For example, when a prompt word contains multiple judgment conditions, the model may misunderstand due to the order of the conditions. The system will rearrange the condition logic based on the errors exposed in the difference samples, such as placing "whether the customer is on the whitelist" before "credit score judgment," thereby enabling the model to prioritize key conditions and reduce the false positive rate.

[0112] Adding counterexamples to the prompts enhances the model's understanding of boundary or outlier conditions. For instance, if the system detects a boundary value handling error when the credit score is 600, a counterexample could be added to the prompt: "When the credit score is 600, it should be considered a pass," thus clarifying the boundary condition. In this way, the prompts can cover more potential input scenarios, reducing the risk of omissions.

[0113] By modifying the natural language representation of business rules to generate variant prompt words, the consistency between model output and business requirements can be improved. For example, when the system detects that the model does not understand the vague term "high-risk area," it will modify the representation to "high-risk areas (including specific areas such as Beijing and Shanghai)," thereby transforming the vague concept into a clear enumeration condition.

[0114] This approach provides multiple improvement paths during the prompt word optimization process, thereby enhancing the coverage and accuracy of prompt words.

[0115] In the embodiments of this application, after generating the prompt word variant, the system does not immediately output it as the final result. Instead, it uses an iterative verification mechanism to ensure the reliability and stability of the prompt word. Specifically, the system verifies and evaluates the performance of the prompt word variant in each iteration and decides whether to continue optimization based on the verification results until the preset performance conditions are met. This iterative verification process forms a closed-loop quality control mechanism, avoiding uncertainties and deviations that may exist when the prompt word is directly output.

[0116] Figure 4 The flowchart illustrates a method for iteratively verifying test data using cue word variations according to some exemplary embodiments of this application.

[0117] like Figure 4 As shown, the method for iteratively verifying test data using cue word variants includes operations S410 to S430.

[0118] When operating S410, after each round of prompt word variant optimization, the current round of prompt word variants are validated using a reserved test set, and the corresponding performance evaluation metrics are calculated.

[0119] The reserved test set is a portion of the original dataset that is not involved in the generation of prompt word variants; it is used solely for objectively evaluating the effectiveness of the prompt words. Performance evaluation metrics can include one or more combinations of metrics such as accuracy, recall, precision, F1 score, and area under the curve (AUC) to comprehensively measure the performance of prompt word variants across different dimensions. For example, in a credit approval scenario, both the "accuracy of rejection samples" and the "recall of approval samples" can be calculated simultaneously to address business needs across different risk dimensions.

[0120] In operation S420, in response to the change in the performance evaluation index of the current round compared with the performance evaluation index of the previous round exceeding the second preset threshold, the prompt word variant of the current round is used as a candidate prompt word.

[0121] When a performance improvement exceeding a second preset threshold is detected, the system can mark the current round's suggestion word variant as a candidate suggestion word. The second preset threshold can be a fixed value, such as a 2% accuracy improvement or a 0.05 F1 score improvement; or it can be a dynamic threshold, automatically adjusted by the system based on task complexity and test data distribution. For example, similar to the first preset threshold, in scenarios with small sample sizes, the threshold can be increased to avoid random fluctuations triggering candidate marking; while in scenarios with large sample sizes, the threshold can be decreased to capture subtle performance improvements. In this way, the system can effectively filter out insignificant optimization results, ensuring that candidate suggestion words truly have improvement value.

[0122] In operation S430, in response to the performance evaluation index of the candidate prompt word meeting the preset performance conditions, the candidate prompt word is determined to be the business rule prompt word.

[0123] Preset performance conditions can be configured by the system or set by business personnel according to their needs, such as "accuracy ≥ 95% and recall ≥ 90%" or "overall F1 score ≥ 0.9". When a candidate suggestion word meets the performance conditions, the system determines it as the final business rule suggestion word and outputs it for actual business execution.

[0124] By introducing a holdout test set for verification and performance evaluation metric calculation after each round of prompt word variant optimization, the embodiments of this application can objectively evaluate the effectiveness of prompt words on independent datasets, avoiding result distortion caused by overfitting training samples. Furthermore, by setting a second preset threshold, the system only considers prompt word variants as candidates when performance improvement reaches a significant level, thereby effectively filtering out versions with insufficient improvement and improving the efficiency and stability of iterative verification. Finally, through the judgment of preset performance conditions, the system can ensure that the output business rule prompt words not only perform well statistically but also meet the needs of actual business scenarios. Compared with the traditional method relying on manual judgment, the embodiments of this application automate the prompt word verification and convergence process, improving the accuracy and reliability of prompt word generation while significantly shortening the verification cycle and computational overhead, thereby enhancing the system's business adaptability and engineering practicality.

[0125] In the embodiments of this application, the system continuously monitors the changes in performance evaluation indicators in each round during the iterative verification of prompt word variants. When it is detected that the performance improvement in K consecutive iterations has not reached the second preset threshold, it can automatically determine that the current optimization has entered a stagnant state and terminate the subsequent iterative verification process. This effectively avoids consuming computing resources without substantial improvement, thereby improving the overall operating efficiency of the system.

[0126] In this embodiment, the system can generate verification feedback information upon terminating iterative verification. This feedback information records key deficiencies in the prompt word optimization process. The feedback information includes at least discrepancy sample information, such as "a sample with a credit score of 600 was incorrectly rejected," and a set of information regarding insufficient coverage of business rules, such as "cross-border orders do not include tariff rules." Through this feedback information, the system can help users quickly identify defects in prompt words in specific scenarios and provide a basis for subsequent manual review or further optimization.

[0127] In some embodiments, the value of K can be flexibly set according to task requirements, and is typically a positive integer not less than 2. For example, in high-precision scenarios, K can be set to 3 or 5 to ensure that the system stops iterating only after confirming that performance has indeed stalled; while in resource-constrained scenarios, K can be set to 2 to quickly identify ineffective optimization in fewer rounds. In this way, the system can achieve a balance between efficiency and performance under different business and computing conditions.

[0128] Optionally, the validation feedback information can not only record discrepancies and insufficient coverage, but also include information on the changing trends of statistical indicators, such as the convergence of accuracy curves or F1 scores. Therefore, the feedback information can not only reflect the weaknesses of the prompts, but also visually demonstrate the trajectory of performance evolution during the optimization process, thereby improving the interpretability and traceability of the system results.

[0129] By introducing a stagnation detection mechanism during the iterative verification process where performance improvement fails to meet targets for K consecutive rounds, the embodiments of this application can promptly terminate ineffective optimizations, avoiding the waste of resources and computational overhead caused by continuously calling large models when improvements are insufficient. Simultaneously, by automatically generating verification feedback information, the system can clearly record discrepancies in samples and insufficient coverage of business rules, and further supplement trend information of performance evaluation metrics, thus providing a clear reference for subsequent manual review or a new round of optimization.

[0130] In the embodiments of this application, the system can also monitor preset resource constraints in real time during the iterative verification of prompt word variants. When a constraint is detected to exceed its corresponding boundary value, the current iterative verification can be terminated to avoid unnecessary resource consumption. These resource constraints may include an upper limit on the number of large model calls, for example, setting the number of calls to no more than 50 in a single optimization task. When the number of calls reaches the upper limit and the prompt word performance has not yet converged, the system automatically stops the iteration and saves the currently best-performing prompt word as the output result.

[0131] In another embodiment, resource constraints can be embodied in a computational resource budget, such as setting a fixed budget for Graphics Processing Unit (GPU) usage time, Central Processing Unit (CPU) utilization, or storage usage. When the system detects that the cumulative consumption is approaching the budget limit, it will proactively terminate the iteration to ensure the overall system stability and scalability. For example, in a shared GPU server environment, a 2-hour upper limit can be set for a single verification task. When the iteration verification duration reaches this threshold, the system will terminate the task and output the current optimal prompt.

[0132] In another embodiment, resource constraints may include a verification latency threshold to ensure system availability in scenarios with high real-time requirements. For example, in e-commerce promotions or financial risk control scenarios, business rule prompts need to be optimized and deployed within minutes. The system can set a maximum latency threshold for each round of verification iterations, such as 30 seconds. When the verification time for a certain prompt variant exceeds this threshold, the system will terminate the subsequent verification process and directly output the best-performing prompt from the previous round as the result.

[0133] In another optional embodiment, the system can simultaneously combine multiple resource constraints for joint control. For example, both an upper limit on the number of large model calls and a verification delay threshold can be set. When either condition is triggered, the system will terminate the iteration process and generate verification feedback information. In this way, the system can flexibly balance computational performance, resource costs, and response speed while ensuring optimization effects, thereby improving overall operating efficiency and engineering practicality.

[0134] In the embodiments of this application, after determining the final business rule prompt word, the system can also simultaneously generate explanatory feedback information associated with the prompt word, so that users can understand the formation process and verification effect of the prompt word, thereby improving the transparency and traceability of the prompt word optimization process. The explanatory feedback information may at least include difference sample tracking information, used to characterize the trajectory of difference changes between the model output and the expected label during the prompt word generation process.

[0135] In some embodiments, the interpretation of feedback information may further include performance evaluation metric change curves to indicate the progress of iterative verification. Specifically, the system records performance metrics after each round of verification and generates curves or bar charts to reflect their changing trends.

[0136] In some embodiments, the explanation feedback information may also include a visualized rule logic diagram to display the business rule logic structure corresponding to the prompt words. The system maps structured requirement data and final prompt words into a logic diagram or flowchart, for example, using nodes to represent condition fields and lines to represent logical relationships, thereby intuitively displaying complex rule chains.

[0137] Optionally, the feedback information can also support interactive displays, allowing users to click on the difference samples to view the corresponding prompt word versions and output results, or select prompt word variants from a specific round in the performance curve for comparative analysis. Through this interactive approach, users can not only obtain macro-level trends but also delve into micro-level details, thereby improving the transparency and controllability of the prompt word optimization process.

[0138] According to embodiments of this application, an automated generation and optimization process for business rule prompts is constructed, realizing a closed-loop mechanism from structured parsing of multi-source inputs, initial prompt generation, iterative optimization driven by differential samples, to performance convergence and feedback output. This scheme effectively avoids semantic ambiguity and logical risks in natural language descriptions, improving the accuracy and stability of prompt generation; by introducing threshold control, reinforcement learning, and resource constraints, computational overhead is reduced and convergence is accelerated; simultaneously, explanatory feedback information is generated, enhancing the transparency and traceability of the results.

[0139] Corresponding to the above-described method for generating business rule prompts, embodiments of this application also provide a device for generating business rule prompts.

[0140] Figure 5 The diagram illustrates the structure of a device for generating business rule prompts according to an embodiment of this application.

[0141] like Figure 5 As shown, the business rule prompt word generation device 500 of this embodiment includes a data acquisition module 510, an initial prompt word processing module 520, a prompt word variant generation module 530, and a business rule prompt word generation module 540.

[0142] The data acquisition module 510 can be used to acquire business requirement information submitted based on the target interactive interface, and convert the business requirement information into structured requirement data. The target interactive interface is used to support structured rule expressions in multiple input formats. In one embodiment, the data acquisition module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0143] The initial prompt word processing module 520 can be used to generate initial prompt words based on the structured requirement data; input the initial prompt words and test data into the large model to obtain a first output result. In one embodiment, the initial prompt word processing module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0144] The prompt word variant generation module 530 can be used to compare the first output result with the corresponding expected label, identify the difference samples, and generate prompt word variants based on the difference samples. In one embodiment, the prompt word variant generation module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0145] The business rule prompt generation module 540 can be used to drive the large model to perform iterative verification on the test data using the prompt variants until the verification performance meets the preset performance conditions, and then output the business rule prompt. In one embodiment, the business rule prompt generation module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0146] According to an embodiment of this application, the data acquisition module 510 can also be used to select target fields through a visual form and define target parameters corresponding to business rules; upload positive and negative example data through the visual form, wherein the positive and negative example data are used to indicate the boundary conditions of the business rules; or set threshold rules in the visual form, wherein the threshold rules are used to constrain numerical judgment logic or conditional judgment logic.

[0147] According to embodiments of this application, the prompt word variant generation module 530 can also be used to annotate differences and calculate corresponding difference rates based on the comparison results of the first output result and the expected label; and in response to the difference rate exceeding a first preset threshold, generate prompt word variants based on the differences. The prompt word variants can be generated in at least one of the following ways: adjusting the order of the business rules, supplementing counterexamples in the prompt words, and modifying the natural language expression of the business rules.

[0148] According to an embodiment of this application, the business rule prompt word generation module 540 can also be used to verify the prompt word variant of the current round using a reserved test set after each round of prompt word variant optimization, and calculate the corresponding performance evaluation index; in response to the change of the performance evaluation index of the current round compared with the performance evaluation index of the previous round exceeding a second preset threshold, the prompt word variant of the current round is used as a candidate prompt word; and in response to the performance evaluation index of the candidate prompt word meeting the preset performance condition, the candidate prompt word is determined as the business rule prompt word.

[0149] According to an embodiment of this application, the business rule prompt generation module 540 can also be used to terminate the iterative verification and generate verification feedback information in response to the fact that the change magnitude has not reached the second preset threshold for K consecutive rounds. The verification feedback information includes at least the difference sample information and the information set of insufficient business rule coverage, where K is a positive integer not less than 2.

[0150] According to an embodiment of this application, the business rule prompt generation module 540 can also be used to terminate iterative verification in response to a preset resource constraint condition exceeding the corresponding boundary value, wherein the preset resource constraint condition includes at least one of the following: a maximum number of large model calls, a computational resource budget, and a verification delay threshold.

[0151] According to an embodiment of this application, the business rule prompt word generation module 540 can also be used to generate explanatory feedback information associated with the business rule prompt word, the explanatory feedback information including at least one of the following: differential sample tracking information for characterizing the prompt word generation process, a performance evaluation index change curve for indicating iterative verification, and a visual rule diagram for displaying the rule logic.

[0152] According to embodiments of this application, any multiple modules among the data acquisition module 510, initial prompt word processing module 520, prompt word variant generation module 530, and business rule prompt word generation module 540 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the data acquisition module 510, initial prompt word processing module 520, prompt word variant generation module 530, and business rule prompt word generation module 540 can be at least partially implemented as hardware circuitry, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the data acquisition module 510, the initial prompt word processing module 520, the prompt word variant generation module 530, and the business rule prompt word generation module 540 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0153] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a method for generating business rule prompts according to an embodiment of this application.

[0154] like Figure 6 As shown, an electronic device 600 according to an embodiment of this application includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0155] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0156] According to embodiments of this application, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0157] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0158] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 described above.

[0159] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the business rule prompt word generation method provided in the embodiments of this application.

[0160] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0161] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0162] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0163] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0164] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0165] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0166] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A method for generating business rule prompt words, characterized in that, The method includes: Obtain business requirement information submitted based on the target interactive interface, and transform the business requirement information into structured requirement data. The target interactive interface is used to support structured rule expression in multiple input forms. Initial prompt words are generated based on the structured requirement data; the initial prompt words and test data are input into the large model to obtain the first output result; The first output result is compared with the corresponding expected label to identify discrepancies, and cue word variants are generated based on the discrepancies; and The large model is driven by the variant of the prompt word to perform iterative verification on the test data until the verification performance meets the preset performance conditions, and then the business rule prompt word is output.

2. The method according to claim 1, characterized in that, The acquisition of business requirement information submitted based on the target interactive interface includes at least one of the following: Select target fields using a visual form and define the target parameters corresponding to the business rules; Upload positive and negative example data through the visual form; the positive and negative example data are used to indicate the boundary conditions of the business rules. In the visual form, a threshold rule is set, which is used to constrain numerical judgment logic or conditional judgment logic.

3. The method according to claim 1 or 2, characterized in that, The process of converting the business requirement information into structured requirement data includes: The business requirement information is parsed and its fields mapped to generate a structured field set; and The structured field set is subjected to integrity and consistency checks, missing fields are filled in or outliers are removed, and the structured requirement data is generated.

4. The method according to claim 1, characterized in that, The step of comparing the first output result with the corresponding expected label, identifying difference samples, and generating prompt word variants based on the difference samples includes: Based on the comparison results between the first output result and the expected label, the differences are marked and the corresponding difference rates are calculated; and In response to the difference rate exceeding a first preset threshold, a prompt word variant is generated based on the difference point.

5. The method according to claim 4, characterized in that, The generation of the prompt word variant includes at least one of the following: adjusting the order of the business rules, adding counterexamples to the prompt word, and modifying the natural language expression of the business rules.

6. The method according to claim 1, characterized in that, The process of using the variant of the prompt word to drive the large model to iteratively verify on the test data until the verification performance meets the preset conditions, and then outputting the business rule prompt word, includes: After each round of prompt word variant optimization, the current round of prompt word variants are validated using a reserved test set, and the corresponding performance evaluation metrics are calculated. If the change in the performance evaluation metric of the current round compared to the performance evaluation metric of the previous round exceeds a second preset threshold, then the variant of the prompt word in the current round is selected as a candidate prompt word; and If the performance evaluation index of the candidate suggestion word meets the preset performance condition, the candidate suggestion word is determined as the business rule suggestion word.

7. The method according to claim 6, characterized in that, The method further includes: If the change magnitude fails to reach the second preset threshold for K consecutive rounds, the iterative verification is terminated and verification feedback information is generated. The verification feedback information includes at least the difference sample information and the information set of insufficient business rule coverage, where K is a positive integer not less than 2.

8. The method according to claim 6, characterized in that, The method further includes: The iterative verification is terminated when the preset resource constraints exceed the corresponding boundary values. The preset resource constraints include at least one of the following: a maximum number of large model calls, a computational resource budget, and a verification latency threshold.

9. The method according to claim 1, characterized in that, The method further includes: Generate explanatory feedback information associated with the business rule prompt words, the explanatory feedback information including at least one of the following: difference sample tracking information for characterizing the prompt word generation process, performance evaluation index change curve for indicating iterative verification status, and a visual rule diagram for displaying the rule logic.

10. A device for generating business rule prompts, characterized in that, The device includes: The data acquisition module is used to: acquire business requirement information submitted based on the target interactive interface, and convert the business requirement information into structured requirement data. The target interactive interface is used to support structured rule expressions in multiple input forms. The initial prompt word processing module is used to: generate initial prompt words based on the structured requirement data; input the initial prompt words and test data into the large model to obtain a first output result; The prompt word variant generation module is configured to: compare the first output result with the corresponding expected label, identify difference samples, and generate prompt word variants based on the difference samples; and The business rule prompt word generation module is used to: drive the large model to perform iterative verification on the test data using the prompt word variant until the verification performance meets the preset performance conditions, and output the business rule prompt word.

11. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 9.

Images