A method and system for real-time evaluation of business rules based on a large language model

Abstract

This application relates to a method, system, device, and medium for real-time evaluation of business rules based on a large language model. The method includes: invoking a large language model to convert natural language business rules into structured description data containing entity identifiers, attribute identifiers, and rule constraints; mapping entity identifiers and attribute identifiers to the location information of corresponding program elements in a domain model based on a pre-stored mapping library, generating detailed mapping data; generating source code files implementing a preset evaluator interface based on the structured description data and the detailed mapping data; converting the source code files into an executable form and executing them in an isolated execution environment; invoking the evaluator interface, evaluating the execution results using generated test cases, and generating a verification report. This application achieves fully automated conversion of business rules to executable code, reducing maintenance costs and improving the efficiency of rule deployment.

Description

Technical Field

[0001] This application relates to the field of intelligent software engineering technology, and in particular to a method, system, computer device and storage medium for real-time evaluation of business rules based on a large language model. Background Technology

[0002] With the widespread application of Domain-Driven Design (DDD) in microservice architectures, business rules are advocated to be encapsulated within the domain model, forming a rich, well-cohesive model with low coupling. In existing business rule evaluation technologies, traditional rule engines externalize the configuration of business rules, forcing the domain model to degenerate into anemic objects that merely carry data, thus compromising the encapsulation of the domain model. Furthermore, rule changes require maintaining both external rule files and the internal domain model simultaneously, resulting in inefficiency and potential logical inconsistencies. While manual coding can ensure consistency with the domain model, it heavily relies on specialized personnel, with the cycle from rule proposal to code deployment lasting several days or even weeks, failing to meet agile iteration requirements. Fine-tuning or training schemes based on large language models suffer from high computational and data costs far exceeding the benefits of rule changes, as rule evaluation itself involves lightweight logical judgments. Moreover, trained models quickly become invalid when the domain model structure is updated, and the time cost of retraining cannot meet business timeliness requirements. Therefore, existing technologies struggle to achieve a rapid, low-cost, and consistent conversion of business rules into executable evaluation code while maintaining the encapsulation of the domain model. Summary of the Invention

[0003] Therefore, it is necessary to address the aforementioned technical issues by providing a method, system, computer device, and storage medium for real-time evaluation of business rules based on a large language model, which can achieve rapid, low-cost, and consistent conversion of business rules into executable evaluation code while maintaining the encapsulation of the domain model.

[0004] Firstly, this application provides a method for real-time evaluation of business rules based on a large language model. The method includes: The large language model is invoked to convert natural language business rules into structured description data, which includes entity identifiers, attribute identifiers, and rule constraints. Based on the pre-stored mapping library, the entity identifier and the attribute identifier are mapped to the location information of the corresponding program element in the domain model, generating detailed mapping data; Based on the structured description data and the mapping details data, the large language model is invoked to generate source code files that implement the preset evaluator interface; The source code file is converted into an executable form and executed in an isolated execution environment; The evaluator interface is invoked to evaluate the execution results of the source code file using the generated test cases, and a verification report is generated.

[0005] In one embodiment, the conversion of natural language business rules into structured descriptive data includes: The large language model is invoked to extract business entities and attributes from business documents and generate a general language vocabulary; the domain model source code is parsed to generate a structured description of the model. Based on the general language lexicon, the large language model is invoked to normalize the terms in the natural language business rules into entity identifiers and attribute identifiers in the structured description data.

[0006] In one embodiment, mapping entity identifiers and attribute identifiers to location information of corresponding program elements in the domain model includes: Map entity identifiers to the fully qualified names of domain model classes, and map attribute identifiers to attribute access method signatures.

[0007] In one embodiment, converting the source code file into an executable form includes: The source code file is compiled directly in runtime memory using a memory compiler, without writing intermediate files to disk during compilation, and the resulting executable is stored in a cache.

[0008] In one embodiment, the execution in an isolated execution environment includes at least one of the following: Method 1: Create a custom class loader that only allows loading classes from a pre-defined whitelist; use bytecode enhancement technology to intercept illegal instructions in the loaded code; Method 2: Build a containerized image, set resource limits and run the container as a non-privileged user; trigger a pipeline task to mount relevant data into the container, perform compilation and evaluation within the container, and determine whether to allow integration or deployment based on quality access thresholds.

[0009] In one embodiment, the process of generating the test samples includes: Generate pattern description data based on the structured description of the domain model; Based on the pattern description data, and according to the rule constraints in the structured description data, the large language model is invoked to generate a positive test case set and a negative test case set, wherein each negative test case in the negative test case set violates at least one of the rule constraints.

[0010] In one embodiment, the method further includes: When the pass rate in the verification report is lower than a preset threshold, the error information is input as feedback to the large language model, and the steps of generating the source code file, executing in an isolated execution environment, and evaluating the execution result are re-executed. Write the general language vocabulary, the mapping library, the code generation description information corresponding to the source code file, and the verification report into the audit log; The general language vocabulary and mapping library are updated based on the unmapped tags generated during the mapping process of the location information or the verification report.

[0011] Secondly, this application also provides a real-time business rule evaluation system based on a large language model. The system includes: The rule conversion module is used to call the large language model to convert natural language business rules into structured description data, which includes entity identifiers, attribute identifiers, and rule constraints. The mapping module is used to map the entity identifier and the attribute identifier to the location information of the corresponding program element in the domain model based on the pre-stored mapping library, and generate detailed mapping data. The code generation module is used to call the large language model to generate source code files that implement the preset evaluator interface based on the structured description data and the mapping details data. An execution module is used to convert the source code file into an executable form and execute it in an isolated execution environment; The evaluation module is used to call the evaluator interface, evaluate the execution results of the source code file using the generated test cases, and generate a verification report.

[0012] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the aforementioned real-time business rule evaluation method based on a large language model.

[0013] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the aforementioned real-time evaluation method for business rules based on a large language model.

[0014] The aforementioned method, system, computer equipment, and storage medium for real-time evaluation of business rules based on a large language model convert natural language business rules into structured descriptive data containing entity identifiers, attribute identifiers, and rule constraints by calling the large language model. Based on a pre-stored mapping library, the entity identifiers and attribute identifiers are mapped to the location information of corresponding program elements in the domain model, generating detailed mapping data. This allows natural language rules to accurately align with the domain model without manual mapping coding. The large language model is then called based on the structured descriptive data and detailed mapping data to generate source code files implementing a preset evaluator interface, eliminating the need for model fine-tuning or training and reducing computational and data costs. The source code files are converted into executable form and executed in an isolated execution environment, ensuring the security of the generated code and preventing impact on the host system. Finally, the evaluator interface is called to evaluate the execution results using generated test cases and generate a verification report, achieving real-time automated verification of the generated code. This method achieves fully automated conversion from natural language business rules to executable evaluation code while maintaining the encapsulation of the domain model. This shortens the rule deployment time, reduces the maintenance costs of human and computing resources, and ensures the consistency between the rule implementation and the domain model without human intervention. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating the application environment of a business rule real-time evaluation method based on a large language model in one embodiment. Figure 2 This is a flowchart illustrating a real-time evaluation method for business rules based on a large language model in one embodiment. Figure 3 This is a schematic diagram illustrating the process of generating a test sample set in one embodiment; Figure 4 This is a block diagram of a business rule real-time evaluation system based on a large language model in one embodiment. Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. First, to facilitate understanding of the technical solutions provided by the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0017] With the widespread adoption of microservice architectures, Domain-Driven Design (DDD) has become a core methodology for building complex business systems due to its ability to achieve high cohesion and low coupling of business logic. DDD emphasizes encapsulating business rules within aggregate roots, entities, and value objects, forming a rich, behavior-rich model centered on the domain model. However, traditional rule engines are inherently designed with a transaction script pattern in mind, extracting business decision logic from the code and managing it through externalized rule-action statements. Forcing such rule engines into DDD systems creates a profound paradigm conflict: on the one hand, business rules are placed outside the domain model, forcing the originally cohesive business logic to degenerate into anemic objects that merely carry data, thus compromising the encapsulation integrity of the domain model; on the other hand, business experts and developers need to continuously perform manual mapping between rule syntax and domain model code, leading to a semantic gap between business language and technical implementation. More critically, a change to a business rule often requires simultaneous modifications to both the external rule file and the internal domain model state. The lack of mandatory linkage between these two modifications easily leads to logical inconsistencies, and verifying such consistency requires far more work than a single-model-driven system.

[0018] Therefore, in order to achieve rapid response, low-cost maintenance, and full-process auditability of business rules while maintaining the encapsulation of the domain model, this embodiment provides a method, system, computer equipment, and storage medium for real-time evaluation of business rules based on a large language model.

[0019] The real-time evaluation method for business rules based on a large language model provided in this application can be applied to, for example... Figure 1 In the application environment shown, input terminal 102 communicates with server 104 via a network. A data storage system stores a business general language vocabulary, mapping library, and audit logs. This data storage system can be integrated on server 104 or deployed in the cloud or other network storage nodes. Input terminal 102 acquires business documents, natural language-described business rules, and domain model source code, and sends this data to server 104. After receiving the data, server 104 sequentially performs business semantic extraction, rule standardization, mapping parsing, code generation, memory compilation and security sandbox loading, positive and negative sample verification, and audit log recording and mapping library update operations, ultimately generating rule evaluator code that can be called in the production environment. Input terminal 102 can be a desktop computer, laptop, industry-specific terminal device used by business personnel, or an enterprise-level application client integrated with rule submission functionality. Server 104 can be a standalone physical server, a server cluster consisting of multiple physical servers, or a cloud server node deploying a large language model service.

[0020] Firstly, in real-world business scenarios, business experts typically describe business rules using natural language, while developers need to manually convert these rules into evaluator code that conforms to Domain-Driven Design (DDD) specifications. Traditionally, this conversion process relies on manual understanding and coding, which is inefficient and prone to errors. Therefore, in one embodiment, such as... Figure 2 As shown, a method for real-time evaluation of business rules based on a large language model is provided, which can be applied to... Figure 1 Taking server 104 as an example, the following steps are included: Step S1: Call the large language model to convert natural language business rules into structured description data. The structured description data includes entity identifiers, attribute identifiers, and rule constraints.

[0021] In detail, the process involves acquiring natural language business rules input by business experts, such as "The registered capital of an enterprise shall not be less than 1 million RMB". A large language model (such as the open-source Qwen-14B model or the commercial GPT-4 model, accessed via API) is invoked to convert the rule into structured descriptive data using preset prompts. The output can be in JSON format, containing entity identifiers (e.g., "enterprise"), attribute identifiers (e.g., "registered capital"), and rule constraints (e.g., "registered capital is greater than or equal to 1,000,000"). Optionally, to further improve the semantic understanding and alignment of entities and attributes, in some feasible implementations, the structured descriptive data may also include entity semantic descriptions, attribute semantic descriptions, etc., which add easily understandable explanations to the entity and attribute identifiers.

[0022] Step S2: Based on the pre-stored mapping library, map entity identifiers and attribute identifiers to the location information of the corresponding program elements in the domain model, and generate detailed mapping data.

[0023] In detail, the pre-stored mapping library contains the correspondence between business entities and domain model classes, as well as attribute correspondences. For example, the entity "Enterprise" maps to the domain model class com.example.domain.Company, and the attribute "Registered Capital" maps to the getter method getRegisteredCapital(). Entity identifiers and attribute identifiers are mapped to fully qualified names and method signatures, generating detailed mapping data. If an identifier does not exist in the mapping library, it is marked as "unmapped," and a pending task is generated.

[0024] Step S3: Based on the structured description data and mapping details, call the large language model to generate source code files that implement the preset evaluator interface.

[0025] In detail, based on the structured description data and mapping details, code is constructed to generate prompt words (the specific composition of the prompt words will be detailed in subsequent embodiments), and the large language model is called to generate source code files that implement the preset evaluator interface. This evaluator interface is defined as Evaluator. <t>Where T is the type of the domain model aggregate root, and the interface contains an evaluate(T aggregate, List<T>) function. <string>The `errorMessages` method is used to perform rule constraint judgment on the domain model class instance. If the constraint is satisfied, it returns true; otherwise, it returns false and stores the error message in the `errorMessages` list.

[0026] Step S4: Convert the source code file into an executable form and execute it in an isolated execution environment.

[0027] In detail, after obtaining the source code files, they are converted into an executable form (e.g., compiled into bytecode) in the runtime environment. During the conversion, a memory compiler is used to compile directly in runtime memory space, without writing intermediate files to disk, and the compiled executable form is stored in a cache for reuse. Then, the executable form is loaded into an isolated execution environment for execution. The isolated execution environment can employ any existing technology; the appropriate method can be selected based on the deployment environment to ensure secure isolation during code execution.

[0028] Step S5: Call the evaluator interface to evaluate the execution results of the source code file using the generated test cases and generate a verification report.

[0029] In detail, test cases (including positive and negative cases) are automatically generated based on the structured description of the domain model. The loaded evaluator implementation class is called to evaluate each test case, and the evaluation results are collected to generate a structured verification report. The report includes information such as compilation status, positive case pass rate, negative case constraint coverage, and average execution latency.

[0030] Through the steps described above, this embodiment achieves a fully automated conversion from natural language business rules to executable evaluator code. The entire process requires no manual coding or fine-tuning or training of large language models; code generation is completed with a single prompt word call. Combined with just-in-time compilation and automated testing, rules can be deployed within seconds. This method shortens rule deployment time and reduces human maintenance costs while maintaining the encapsulation of the domain model, and avoids paradigm conflicts between traditional rule engines and domain-driven design.

[0031] In the process of converting natural language business rules into structured descriptive data, in order for the large language model to accurately understand the specific terms in the business documents and the structure of the domain model, it is necessary to pre-establish a general language vocabulary and a structured description of the model. Traditionally, the mapping between business terms and code models relies entirely on manual intervention, which is inefficient and prone to ambiguity. Therefore, in one embodiment, such as... Figure 3 As shown, converting natural language business rules into structured descriptive data includes: The system calls a large language model to extract business entities and attributes from business documents and generate a general language vocabulary; it also parses the source code of the domain model to generate a structured description of the model. Based on the general language lexicon, the large language model is invoked to normalize the terms in the natural language business rules into entity identifiers and attribute identifiers in the structured description data.

[0032] Specifically, business documents can be text files in PDF, Word, or HTML formats. The Apache Tika tool is used to preprocess the business documents, converting them into UTF-8 encoded plain text. Then, a large language model is invoked, taking the plain text content as input and using preset prompts to extract the business entities (e.g., "enterprise," "loan application") and the business attributes (e.g., "registered capital," "amount") contained within each entity. The extraction results are saved in structured JSON format, forming a general business language vocabulary. Similarly, to facilitate semantic understanding and alignment of entities and attributes, the terminology can be further standardized into structured descriptions of entity semantics and attribute semantics within the data.

[0033] In the process of generating a general language vocabulary, the Sentence-BERT model, an existing technology, can be used to calculate the semantic similarity between extracted business entities and domain model classes. When the semantic similarity exceeds a preset threshold (e.g., 0.85), candidate mapping relationships are generated for subsequent manual calibration. Sentence-BERT is a sentence embedding model based on BERT that can map text to a high-dimensional vector space and measure semantic similarity by calculating the cosine similarity between vectors. This step can help business experts quickly establish initial mappings and improve the accuracy of semantic extraction.

[0034] Simultaneously, the source code of the domain model is parsed using a syntax parsing tool to extract package names, class names, attribute definitions, method definitions, and code comments, generating structured descriptive data that includes these elements. Specifically, the JavaParser open-source library is used to parse the source code file into an abstract syntax tree. After traversing the tree, package names, class names, attribute definitions (attribute names, types, modifiers), method definitions (method names, return types, parameter lists, modifiers), and Javadoc comments are obtained and organized into a JSON object for storage. This structured descriptive data is used for subsequent mapping of business entities to the domain model and for generating test cases.

[0035] When natural language business rules are subsequently received, prompt words are constructed based on a general language lexicon. The large language model is then required to normalize the terms in the rules into entity and attribute identifiers already registered in the lexicon. If a word not registered in the general language lexicon appears in the rule, a prompt window pops up, awaiting confirmation from business experts and the addition of that word to the lexicon. Specifically, the rule text is segmented and part-of-speech tagged (e.g., using the HanLP Chinese word segmentation tool), and the segmentation results are matched against all words in the general language lexicon (including entity names, attribute names, and synonyms). If a segmentation result does not exist in the lexicon, it is marked as an unregistered word, and a prompt window pops up in the rule standardization interface for user confirmation.

[0036] In this way, this embodiment ensures the semantic accuracy and scalability of the rule standardization process. The general language vocabulary and the structured description of the domain model together build a bridge between business language and technical implementation, enabling the large language model to accurately understand the business context. At the same time, the unregistered vocabulary prompting mechanism enables the continuous evolution of the vocabulary, eliminating the terminology barrier between business experts and the system.

[0037] When mapping entity identifiers and attribute identifiers to the location information of corresponding program elements in the domain model, it is necessary to precisely specify the target class and attribute access method so that the attributes and methods of the domain model can be correctly called during subsequent code generation. Traditional mapping methods rely on precise string matching, which is difficult to handle practical situations such as synonyms and spelling differences. Therefore, in one embodiment, mapping entity identifiers and attribute identifiers to the location information of corresponding program elements in the domain model includes: Map entity identifiers to the fully qualified names of domain model classes, and map attribute identifiers to attribute access method signatures.

[0038] Specifically, the mapping library can use the Neo4j graph database, where node types include business entity nodes, domain model class nodes, attribute nodes, and method nodes, and the relationship type is "mapped to". For example, the business entity "Company" maps to the fully qualified name of the domain model class com.example.domain.Company, and the business entity attribute "Registered Capital" maps to the attribute access method signature getRegisteredCapital(). After obtaining the entity identifier "Company" and the attribute identifier "Registered Capital" from the structured description data, the mapping library is queried to obtain the corresponding fully qualified name and method signature, generating detailed mapping data.

[0039] During the mapping process, when an entity or attribute identifier does not have an exact match in the mapping database, the Levenshtein distance algorithm can be used to calculate the text similarity between the entity or attribute identifier and existing entries in the mapping database, and then combined with a thesaurus for fuzzy matching. The Levenshtein distance refers to the minimum number of editing operations required to transform one string into another, including insertion, deletion, and replacement. The calculated distance is converted into a similarity score (similarity = 1 - distance / maximum length). When the similarity exceeds a preset threshold (e.g., 0.8), the matched entry is recommended as a candidate mapping for manual confirmation. The thesaurus records common synonyms for business terms, such as "registered capital" and "registered funds," "capital," etc., expanding the query scope during matching. This fuzzy matching mechanism improves the robustness of the mapping and reduces the generation of unmapped tags.

[0040] If no corresponding relationship is found after both exact and fuzzy matching, the identifier is marked as "unmapped" and a mapping task is generated. The task record includes the task ID, the name of the unmapped identifier, the identifier type (entity or attribute), the original rule text, and the generation time. This task is stored in a pending task table with the status "pending." The front-end interface displays the number of pending tasks in real time. Business experts or operations personnel can click to view task details and manually select or enter the correct mapping target. After manual mapping calibration, the newly added mapping relationship is written to the mapping database, and the task status is updated to "processed." Simultaneously, the mapping resolution step is retried, using the updated mapping database to process the previously unmapped identifier again. An automatic retry mechanism can be set, such as scanning pending tasks every 5 minutes, and escalating an alarm if a task remains unprocessed for more than 24 hours.

[0041] Based on the above, this embodiment achieves a precise and configurable mapping from natural language terms to specific code elements. Levenshtein distance fuzzy matching and thesaurus expansion greatly improve the fault tolerance of the mapping, while the pending task mechanism ensures the integrity of the mapping process, avoiding process interruptions due to missing mappings. At the same time, manual calibration closed-loop ensures the continuous accuracy of the mapping library.

[0042] To improve code compilation efficiency and avoid disk I / O overhead, the traditional method saves the generated source code as a disk file and then calls an external compiler, resulting in slow compilation speed and the generation of temporary files that require additional cleanup. To achieve second-level deployment of rules, one embodiment converts the source code file into an executable form, including: The memory compiler compiles source code files directly in runtime memory space without writing intermediate files to disk during compilation, and stores the compiled executable in a cache.

[0043] Specifically, after obtaining the source code string returned by the large language model, it is not written to any disk file, but directly passed to the memory compiler. The memory compiler can use the JavaCompiler interface provided by the JDK, constructing a custom JavaFileObject object and taking the source code string as input. When JavaCompiler calls the getTask method, a custom DiagnosticCollector is passed in to collect compilation error information. During compilation, all intermediate artifacts (such as .class files) are kept in memory and not written to disk. After successful compilation, a bytecode array (byte[]) is obtained from the output of JavaCompiler, and then the bytecode is stored in a cache. The cache can use an LRU (Least Recently Used) strategy, with a maximum capacity of, for example, 100 entries. When the cache is full, the least recently used compilation artifact is automatically evicted. The cache key can be a hash value of the rule content or a rule identifier, so that when the same rule is triggered again, the bytecode can be retrieved directly from the cache without recompiling. In practical applications, the cache capacity can be adjusted according to the system memory size and business concurrency. A similar function can also be achieved using the third-party memory compilation library Janino. This embodiment does not limit this.

[0044] By employing in-memory compilation and caching mechanisms, this embodiment avoids disk I / O overhead, reducing compilation time from seconds to milliseconds. Combined with caching, it further reduces redundant compilation, thus achieving a second-level response time from business rule submission to execution, meeting the requirements of real-time business scenarios. Simultaneously, the absence of any temporary files simplifies cleanup and maintenance.

[0045] To ensure the secure execution of code generated by a large language model in a production environment and prevent malicious code (such as file access, network connection, or system command execution) or unexpected errors from compromising system stability, the generated code needs to be run in an isolated environment. Traditionally, directly loading and executing externally generated code via reflection poses serious security risks. Therefore, in one embodiment, execution is performed in an isolated execution environment, including at least one of the following methods: Method 1: Create a custom class loader that only allows loading classes from a pre-defined whitelist; use bytecode enhancement technology to intercept illegal instructions in the loaded code; Method 2: Build a containerized image, set resource limits and run the container as a non-privileged user; trigger a pipeline task to mount the relevant data into the container, perform compilation and evaluation inside the container, and decide whether to allow integration or deployment based on the quality access threshold.

[0046] Specifically, for Method 1 (Security Sandbox), the implementation involves creating a custom class loader that inherits from ClassLoader and overriding the findClass and loadClass methods. In the loadClass method, it first checks if the requested class name is in a predefined whitelist. The whitelist includes two categories: first, classes from JDK base libraries, such as those in java.lang, java.util, and java.math packages; and second, packages containing predefined domain model classes, such as com.example.domain. If the requested class name is not in the whitelist, the class loader throws a ClassNotFoundException. For the generated bytecode, the class loader defines it in the JVM using the defineClass method. At the security sandbox execution level, bytecode enhancement techniques are used. For example, before loading the bytecode, the ASM library analyzes the bytecode instruction sequence, replacing or throwing exceptions when the following instructions are detected: file access instructions (such as opcodes related to FileInputStream and FileWriter), network access instructions (such as opcodes related to Socket and URLConnection), and system call instructions (such as opcodes related to Runtime.exec and ProcessBuilder). You can also configure a security manager (using SecurityManager in versions below JDK 17, and bytecode enhancement instead in later versions) to restrict code permissions. For intercepted instructions, a security log is logged and a SecurityException is thrown to prevent the instruction from continuing execution. In practical applications, bytecode enhancement can be implemented using the Byte Buddy framework, which performs the transformation during class loading via the Agent pattern.

[0047] For method two (containerization), the implementation first involves building a multi-stage Docker image. The first stage (build stage) includes the JDK compilation environment and domain model JAR packages, used to compile the generated source code files. The second stage (runtime stage) only includes the minimum dependencies required for runtime (such as the JRE base image and verification framework libraries), ensuring a small image size and low attack surface. When starting the container, CPU resource limits (e.g., --cpus=2) and memory resource limits (e.g., --memory=512m) are set via Docker command-line parameters, and the container is run as a non-privileged user, prohibiting processes within the container from executing with root privileges. Once the source code files are generated, a CI / CD pipeline task is automatically triggered (e.g., configuring a webhook via Jenkins or GitLab CI). The pipeline task pulls the pre-built multi-stage Docker image, mounting the source code files, JSON schema description data, and mapping library files to a specified path within the container (e.g., mounting volumes using the -v parameter). Before the pipeline execution, positive and negative test case data are generated in batches based on the JSON schema description data and a large language model. These test cases are then stored in a versioned test case library, the version of which is bound to the version of the domain model (e.g., managed using Git tags). Inside the container, a pre-packaged verification program is invoked. This program compiles the mounted source code files, reads test case data from the versioned test case library, evaluates each case, and collects the results. All file operations and computations are confined within the container and do not affect the host environment. Finally, quality thresholds are set, such as a 100% pass rate threshold for positive test cases and a 100% constraint coverage threshold for negative test cases. The CI / CD pipeline is marked as successful only when the pass rate and constraint coverage in the verification report both reach or exceed the thresholds, allowing subsequent integration or deployment operations. If the thresholds are not reached, code merging is blocked, verification is marked as failed, and failure details are output to the pipeline log.

[0048] By employing at least one of the two methods described above, this embodiment ensures that the code generated by the large language model can only run in a restricted environment, unable to access the file system, network, and underlying operating system (Method 1) or completely isolated from the host machine (Method 2), thereby avoiding potential security risks associated with the generated code and enabling the method to be securely deployed in a production environment. Furthermore, Method 1 is suitable for lightweight, low-overhead scenarios, while Method 2 is suitable for production scenarios requiring strict resource control and environmental consistency. Users can flexibly choose according to their actual deployment environment.

[0049] To automate the verification of whether the generated evaluator code correctly implements all rule constraints and avoid the tediousness and omissions of manually writing test cases, a method is needed to automatically generate test cases covering all rule boundaries. Therefore, in one embodiment, the test case generation process includes: Generate pattern description data based on the structured description of the domain model; Based on the pattern description data, and according to the rule constraints in the structured description data, the large language model is invoked to generate a positive test case set and a negative test case set, wherein each negative test case in the negative test case set violates at least one rule constraint.

[0050] Specifically, first, the JSON schema description data of the domain model is obtained. This data defines the legal data structure of the domain model class instances (e.g., the type, value range, and nullability of each attribute). Then, a list of rule constraints is obtained from the structured description data. Assume this list contains three rules: Rule A "Registered capital is greater than or equal to 1 million yuan", Rule B "Establishment date is no later than the current date", and Rule C "Enterprise status is normal". The large language model is then invoked, with explicit requirements in the prompts: when generating positive test cases, each positive case must simultaneously satisfy all constraints of Rules A, B, and C; when generating negative test cases, for Rule A, a test case with a registered capital of 990,000 yuan is generated (violating Rule A but satisfying Rules B and C); for Rule B, a test case with an establishment date of tomorrow is generated (violating Rule B but satisfying Rules A and C); and for Rule C, a test case with an enterprise status of "cancelled" is generated (violating Rule C but satisfying Rules A and B). The large language model automatically fills in the valid values ​​of other fields based on the field types and value ranges in the JSON schema description data, generating complete domain model instance data. The generation strategy can be set to generate at least one corresponding negative example for each rule constraint, with no fewer than 100 positive examples and no fewer than 100 negative examples. In practical applications, the number of generated examples can be adjusted according to business complexity and testing requirements; this embodiment does not impose such limitations.

[0051] In this embodiment, positive test cases can fully verify whether the evaluator correctly identifies instances that satisfy all rules, while negative test cases can verify one by one whether the evaluator can correctly detect violations of each rule, thereby ensuring that the rule constraint coverage implemented by the evaluator reaches 100% and avoiding omissions. This method transforms the generation of test cases from manual writing to automated generation, reducing testing costs and improving the reliability of rule verification.

[0052] To create a continuous learning loop, enabling the system to self-correct and evolve based on verification results, while ensuring the auditability of the entire rule implementation process, in one embodiment, this method further includes: When the pass rate in the verification report is lower than the preset threshold, the error information is fed back into the large language model, and the steps of generating the source code file, executing it in an isolated execution environment, and evaluating the execution result are re-executed. Write the general language vocabulary, mapping library, code generation description information and verification report corresponding to the source code file into the audit log; Update the general language vocabulary and mapping library based on the unmapped tags or verification reports generated during the mapping process of location information.

[0053] Specifically, the preset threshold can be set to 100%, requiring all positive examples to pass and all negative examples to correctly detect violations. When the pass rate in the verification report is below 100%, the compilation error messages, error messages indicating failed positive example evaluations, and prompts indicating no violations were detected in negative examples are organized into feedback text. The large language model is re-invoked, and the new prompt text includes the original business background summary, structured description data, mapping details, error message templates, and coding standard constraints, as well as the error messages and failed source code generated in the previous round. The prompt text explicitly requires the large language model to correct the code based on the error messages. Subsequently, the code generation, in-memory compilation, security sandbox loading, and evaluation steps are re-executed. The above loop repeats a maximum of the preset number of retries, such as 3 times. If the pass rate threshold is reached within the number of retries, the currently generated code is accepted; if the maximum number of retries is reached and the code still fails, the loop is terminated and an alert is sent to the administrator, awaiting manual intervention.

[0054] Simultaneously, the general language vocabulary, mapping library, code generation description information (i.e., the business background summary, structured description data, mapping details, error message templates, and the concatenation result of coding standard constraints used when constructing prompt words), and verification report generated during this process will be written to the audit log. The audit log can be stored using a Kafka message queue or Elasticsearch. Each log record includes a timestamp, operation type, and data content hash value, for subsequent compliance tracing and problem investigation.

[0055] Furthermore, based on unmapped tags generated during the mapping process (i.e., pending tasks generated when entity identifiers or attribute identifiers cannot find corresponding relationships in the mapping library) or failure information in the verification report, the general language vocabulary and mapping library are updated. Each modification is version-marked, recording the modification time and content; the updated version is loaded into subsequent rule standardization and mapping resolution steps. Specifically, each time an addition, deletion, or modification operation is performed on the general language vocabulary or mapping library, the version marking process is automatically triggered. An independent version number is maintained for each vocabulary and mapping library, in the format "vYYYYMMDD-serial number," and the modification timestamp, modifier, modification type, and differences in content before and after the modification are stored in a version history table in JSON format. During subsequent rule standardization and mapping resolution, the latest version of the vocabulary and mapping library is always loaded.

[0056] Additionally, the entity and attribute identifiers corresponding to unmapped tags are retrieved from the audit logs, a mapping calibration report is generated, and output to the front-end interface for manual confirmation. The manually confirmed mapping relationships are then used to update the mapping library. The audit logs are scanned periodically (e.g., daily at midnight or weekly) to extract recorded "unmapped tag" events, which are then compiled into a mapping calibration report. This report lists all entities and attributes to be confirmed in tabular form, along with their context. This report is pushed to the front-end interface, where business experts or operations personnel can view it and manually select or enter the correct fully qualified name of the domain model class and the attribute access method signature for each unmapped item. After user confirmation, the confirmed mapping relationships are updated in the mapping library, and the corresponding unmapped tags are cleared.

[0057] Through the closed-loop evolution steps of this embodiment, the system can continuously improve itself. The automatic correction mechanism improves the one-time success rate of code generation and reduces manual intervention; the audit log provides a complete record for compliance traceability; and the continuous updates of the mapping library and thesaurus gradually improve the automation and accuracy of subsequent rule processing, realizing a fully automated process from rule input to qualified evaluator output.

[0058] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0059] Secondly, based on the same inventive concept, this application also provides a system for implementing the aforementioned method for real-time evaluation of business rules based on a large language model. The solution provided by this system is similar to the implementation described in the above method; therefore, the specific limitations in one or more system embodiments provided below can be found in the above-described limitations of the method for real-time evaluation of business rules based on a large language model, and will not be repeated here.

[0060] In one embodiment, such as Figure 4 As shown, the system includes: a rule conversion module, a mapping module, a code generation module, an execution module, and an evaluation module. Among them: The rule conversion module is used to call the large language model to convert natural language business rules into structured description data, which includes entity identifiers, attribute identifiers, and rule constraints. The mapping module is used to map entity identifiers and attribute identifiers to the location information of corresponding program elements in the domain model based on a pre-stored mapping library, and generate detailed mapping data. The code generation module is used to generate source code files that implement the preset evaluator interface by calling the large language model based on structured description data and mapping details data. The execution module is used to convert source code files into executable form and execute them in an isolated execution environment; The evaluation module is used to call the evaluator interface, evaluate the execution results of the source code file using the generated test cases, and generate a verification report.

[0061] In one embodiment, the rule transformation module includes a descriptive data transformation unit.

[0062] The descriptive data conversion unit is used to extract business entities and attributes from business documents by calling a large language model, generating a general language vocabulary; and to parse the domain model source code to generate a structured description of the model. Based on the general language vocabulary, the descriptive data conversion unit calls a large language model to standardize the terms in natural language business rules into entity and attribute identifiers in structured descriptive data.

[0063] In one embodiment, the mapping module includes a detailed data generation unit.

[0064] The detailed data generation unit is used to map entity identifiers to fully qualified names of domain model classes and attribute identifiers to attribute access method signatures.

[0065] In one embodiment, the execution module includes a code conversion unit.

[0066] The code conversion unit is used to compile source code files directly in runtime memory space using a memory compiler, without writing intermediate files to disk during compilation, and storing the compiled executable in a cache.

[0067] In one feasible embodiment, the execution module includes a secure execution unit.

[0068] The secure execution unit executes the executable form of the source code file in at least one of the following ways, and the specific methods are as follows: Method 1: Create a custom class loader that only allows loading classes from a pre-defined whitelist; use bytecode enhancement technology to intercept illegal instructions in the loaded code; Method 2: Build a containerized image, set resource limits and run the container as a non-privileged user; trigger a pipeline task to mount the relevant data into the container, perform compilation and evaluation inside the container, and decide whether to allow integration or deployment based on the quality access threshold.

[0069] In one embodiment, the evaluation module includes a sample generation unit.

[0070] Among them, the sample generation unit is used to generate pattern description data based on the structured description of the domain model; Based on the pattern description data, and according to the rule constraints in the structured description data, the large language model is invoked to generate a positive test case set and a negative test case set, wherein each negative test case in the negative test case set violates at least one rule constraint.

[0071] In one embodiment, the system also includes an audit update module.

[0072] The audit update module is used to input error information as feedback into the large language model when the pass rate in the verification report is lower than a preset threshold, and to re-execute the steps of generating source code files, executing in an isolated execution environment, and evaluating the execution results; the audit update module is used to write the general language vocabulary, mapping library, code generation description information corresponding to the source code files, and verification report into the audit log; the audit update module is used to update the general language vocabulary and mapping library based on the unmapped tags or verification reports generated during the mapping process of the location information.

[0073] The modules in the aforementioned real-time business rule evaluation system based on a large language model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can invoke and execute the corresponding operations of each module.

[0074] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores all data related to system operation. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a real-time evaluation method for business rules based on a large language model.

[0075] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0076] Thirdly, in one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described in the above method embodiments.

[0077] Fourthly, in one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described in the above method embodiments.

[0078] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0079] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0080] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.< / string> < / t>

Claims

1. A method for real-time evaluation of business rules based on a large language model, characterized in that, The method includes: The large language model is invoked to convert natural language business rules into structured description data, which includes entity identifiers, attribute identifiers, and rule constraints. Based on the pre-stored mapping library, the entity identifier and the attribute identifier are mapped to the location information of the corresponding program element in the domain model, generating detailed mapping data; Based on the structured description data and the mapping details data, the large language model is invoked to generate source code files that implement the preset evaluator interface; The source code file is converted into an executable form and executed in an isolated execution environment; The evaluator interface is invoked to evaluate the execution results of the source code file using the generated test cases, and a verification report is generated.

2. The method according to claim 1, characterized in that, The process of converting natural language business rules into structured descriptive data includes: The large language model is invoked to extract business entities and attributes from business documents and generate a general language vocabulary; the domain model source code is parsed to generate a structured description of the model. Based on the general language lexicon, the large language model is invoked to normalize the terms in the natural language business rules into entity identifiers and attribute identifiers in the structured description data.

3. The method according to claim 1, characterized in that, The mapping of entity identifiers and attribute identifiers to the location information of corresponding program elements in the domain model includes: Map entity identifiers to the fully qualified names of domain model classes, and map attribute identifiers to attribute access method signatures.

4. The method according to claim 1, characterized in that, The process of converting source code files into executable form includes: The source code file is compiled directly in runtime memory using a memory compiler, without writing intermediate files to disk during compilation, and the resulting executable is stored in a cache.

5. The method according to claim 1, characterized in that, The execution in an isolated execution environment includes at least one of the following methods: Method 1: Create a custom class loader that only allows loading classes from a pre-defined whitelist; use bytecode enhancement technology to intercept illegal instructions in the loaded code; Method 2: Build a containerized image, set resource limits and run the container as a non-privileged user; trigger a pipeline task to mount relevant data into the container, perform compilation and evaluation within the container, and determine whether to allow integration or deployment based on quality access thresholds.

6. The method according to claim 1, characterized in that, The process of generating the test cases includes: Generate pattern description data based on the structured description of the domain model; Based on the pattern description data, and according to the rule constraints in the structured description data, the large language model is invoked to generate a positive test case set and a negative test case set, wherein each negative test case in the negative test case set violates at least one of the rule constraints.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: When the pass rate in the verification report is lower than a preset threshold, the error information is input as feedback to the large language model, and the steps of generating the source code file, executing in an isolated execution environment, and evaluating the execution result are re-executed. Write the general language vocabulary, the mapping library, the code generation description information corresponding to the source code file, and the verification report into the audit log; The general language vocabulary and mapping library are updated based on the unmapped tags generated during the mapping process of the location information or the verification report.

8. A real-time evaluation system for business rules based on a large language model, characterized in that, The system includes: The rule conversion module is used to call the large language model to convert natural language business rules into structured description data, which includes entity identifiers, attribute identifiers, and rule constraints. The mapping module is used to map the entity identifier and the attribute identifier to the location information of the corresponding program element in the domain model based on the pre-stored mapping library, and generate detailed mapping data. The code generation module is used to call the large language model to generate source code files that implement the preset evaluator interface based on the structured description data and the mapping details data. An execution module is used to convert the source code file into an executable form and execute it in an isolated execution environment; The evaluation module is used to call the evaluator interface, evaluate the execution results of the source code file using the generated test cases, and generate a verification report.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the real-time evaluation method for business rules based on a large language model as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the real-time evaluation method for business rules based on a large language model as described in any one of claims 1 to 7.

Images