Intelligent data processing method based on large model atomization packaging and batch processing arrangement

Abstract

The invention relates to the technical field of computers, in particular to an intelligent data processing method based on large model atomization packaging and batch processing arrangement, which comprises the steps of constraint set construction, constraint treatment and verification, target constraint set determination, flow description generation and batch processing arrangement and controlled output. According to the method, the executable constraint set is generated by inputting the constraint configuration submitted by the user and is included in the configuration management, so that the constraint is promoted from a single session prompt to a reusable configuration object, the operation cost caused by cross-session and cross-task repeated setting of the constraint is reduced, and the sustainable use capability of the constraint in a platform processing task is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an intelligent data processing method based on large model atomic encapsulation and batch processing orchestration. Background Technology

[0002] As enterprise application platforms introduce large-scale model capabilities, users initiate processing requests for data objects such as documents, knowledge base content, structured data, and external tool interfaces within the platform. The platform outputs results for dialogue display, business analysis, report generation, ticket generation, or external service deployment. To meet the standardized requirements of different business scenarios, users typically expect the platform output to adhere to specific expression standards, template structures, field definitions, terminology usage, symbol rules, or compliance requirements. In existing platform practices, common approaches include attaching user preferences or constraints to the context of a single session as natural language prompts, or hard-coding a few scenarios using fixed templates.

[0003] The existing methods described above have the following problems: First, constraints are mostly implemented through session memory or prompts, which have weak reusability across sessions or tasks. Users need to repeatedly input constraints in different tasks, which can easily lead to output deviations and increased maintenance costs. Second, constraints usually lack structured expression and verifiable mechanisms. The platform finds it difficult to verify the executability of constraints before output and to provide interpretable handling for content that does not meet constraints after output. This forces users to repeatedly revise prompts through trial and error to approximate the expected results. Finally, when faced with multiple types of processing task requests and multiple types of input data objects, the common practice is either to apply the same constraint to all outputs or to rely on manual configuration switching. This can easily lead to over-constraints, incorrect constraint application, or constraint conflicts, affecting output quality and task execution efficiency, especially in batch processing orchestration scenarios.

[0004] To address this, an intelligent data processing method based on large model atomization encapsulation and batch processing orchestration is proposed. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides an intelligent data processing method based on large model atomization encapsulation and batch processing orchestration.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: an intelligent data processing method based on large model atomic encapsulation and batch processing orchestration, comprising the following steps: S1. Receiving constraint configuration input submitted by the user at the interactive interface of the application platform, generating a constraint set for constraining the platform's output based on the constraint configuration input, and incorporating the constraint set into the platform's configuration management; S2. Performing verification on the constraint set through a pre-verification mechanism and outputting verification information. If the verification information does not meet expectations, receiving revision input from the user and updating the constraint set based on the revision input; S3. When the application platform receives a processing task request, parsing the processing task request to obtain the task intent and acquiring the input data object associated with the processing task request, and determining the target constraint set based on the matching results of the task intent features, input data object features, and constraint set; S4. Encapsulating the data processing capabilities oriented towards the input data object into composable large model operator units, and generating a processing flow description based on the large model operator units and the target constraint set; S5. Performing batch processing orchestration on the input data object according to the processing flow description to obtain candidate outputs, and implementing constraint processing corresponding to the target constraint set on the candidate outputs before outputting the results.

[0007] As a preferred embodiment of the present invention, the constraint configuration input comes from natural language constraint text, constraint template, and example alignment input, and the constraint configuration input consists of one or more of the above input forms; generating a constraint set based on the constraint configuration input includes: performing atomic decomposition on the constraint configuration input to obtain multiple constraint fragments, performing normalization processing on the multiple constraint fragments to obtain structured constraint items, and the structured constraint items include constraint type, constraint object, and constraint parameters; generating the constraint set includes performing compilation processing on the structured constraint items to generate validator representations and repairer representations for the structured constraint items respectively.

[0008] As a preferred technical solution of the present invention, performing verification on the constraint set includes: constructing a verification object based on a preset verification sample, or constructing a verification object based on an online sampling object of a processing task; performing verification processing corresponding to the constraint set on the verification object and outputting verification information; the verification information includes at least one or more of the following: pass mark, failure reason, list of hit constraint items, and revision suggestion; receiving revision input from the user based on the verification information includes: receiving one or more of the following: rewriting input for constraint configuration input, parameter adjustment input, and example supplement input; updating the constraint set based on the revision input; generating a new version and recording change records associated with the verification information when updating the constraint set.

[0009] As a preferred technical solution of the present invention, determining the target constraint set based on the matching relationship between the processing task request and the constraint set includes: parsing the processing task request to obtain the task intent and acquiring the input data object; extracting the task intent features and the input data object features; performing matching calculations on the constraint items in the constraint set based on the task intent features and the input data object features to obtain a matching score; and determining the target constraint set based on the matching score and a screening threshold. Determining the target constraint set includes performing conflict processing on the target constraint set. Conflict processing is used to identify constraint items with incompatible constraint parameters for the same constraint object, and retaining constraint items that satisfy the adjudication rules based on constraint priority and constraint strength, thereby obtaining the target constraint set after conflict resolution.

[0010] As a preferred technical solution of the present invention, the constraint processing corresponding to the target constraint set of the candidate output includes: preferentially performing deterministic repair processing on the candidate output based on the repairer representation; triggering rewrite repair processing when the target constraint set is still not satisfied after deterministic repair processing; setting an upper limit for the number of retries and an upper limit for the resource budget for rewrite repair processing; and recording the constraint set version information and constraint processing trajectory information associated with the processing task request.

[0011] Compared with the prior art, the beneficial effects that this invention can achieve are: 1. This invention generates an executable set of constraints from the constraint configuration input submitted by the user and incorporates it into the configuration management, thereby transforming constraints from single-session prompts into reusable configuration objects. This reduces the operational costs caused by repeatedly setting constraints across sessions and tasks, and enhances the sustainable use of constraints in platform task processing.

[0012] 2. This invention provides a verification mechanism for constraint sets and outputs verification information, allowing users to revise constraint configuration inputs and update constraint sets based on verification information. This enables the constraint formation process to have an interpretable verification feedback loop, reducing the time cost incurred by users in repeatedly adjusting prompts through trial and error, and improving the controllability and maintainability of constraint implementation.

[0013] 3. When a task request arrives, this invention determines the target constraint set based on the matching relationship between the task intent and the input data object and the constraint set. It also combines the atomic encapsulation of large model operator units with batch processing orchestration to achieve self-adaptation and controlled output of constraints for different task scenarios, reducing the impact of incorrect constraint application and constraint conflicts on the platform's output results and batch processing efficiency. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0015] To make the technical means, creative features, objectives, and effects of this invention easier to understand, the invention is further described below with reference to specific embodiments. However, the following embodiments are merely preferred embodiments of this invention and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments described herein without creative effort are all within the protection scope of this invention.

[0016] Example: Figure 1 As shown, an intelligent data processing method based on large model atomic encapsulation and batch processing orchestration includes the following steps: S1. Receive constraint configuration input submitted by the user in the interactive interface of the application platform, generate a constraint set for constraining the output of the platform based on the constraint configuration input, and incorporate the constraint set into the platform's configuration management.

[0017] S2. Perform validation on the constraint set through a pre-validation mechanism and output validation information. If the validation information does not meet expectations, receive the user's revision input and update the constraint set according to the revision input.

[0018] S3. When the application platform receives a task processing request, it parses the task processing request to obtain the task intent and retrieves the input data object associated with the task processing request. Based on the matching results of the task intent features, input data object features and constraint set, the target constraint set is determined.

[0019] S4. Encapsulate the data processing capabilities for input data objects into composable large model operator units, and generate a processing flow description based on the large model operator units and the target constraint set.

[0020] S5. Based on the processing flow description, perform batch processing and arrangement on the input data objects to obtain candidate outputs, and then perform constraint processing on the candidate outputs corresponding to the target constraint set before outputting the results.

[0021] It should be noted that the typical use of the application platform for enterprise users revolves around processing tasks. These tasks can include generating structured summaries from documents, generating citation-based conclusions from knowledge base content, generating query statements and explanations from structured data, and generating reports or work orders in conjunction with external tools. Different processing tasks often have explicit constraints on the platform's output results. For example, the platform's output results must conform to fixed templates, field definitions, symbol specifications, brand terminology, and content compliance requirements. To enable these constraints to be reused across sessions and tasks, the application platform provides a constraint configuration input entry in the interactive interface, allowing users to submit constraint configuration inputs. The application platform generates an executable constraint set based on the constraint configuration inputs and incorporates the constraint set as a configuration object into configuration management. This transforms constraints from being merely a one-time interactive prompt into a long-term reusable configuration asset for the application platform.

[0022] In a preferred embodiment, the constraint configuration input comes from natural language constraint text, constraint templates, and example alignment input. The constraint configuration input can be composed of one of the above input forms or a combination of the above input forms. The natural language constraint text is used to express constraint requirements, such as the platform output results not including specific characters, the platform output results needing to be organized according to a specific paragraph structure, and the platform output results needing to include a field list and provide citation information. The constraint template is used to consolidate and reuse rules, such as patent specification paragraph templates, question caliber templates, brand language templates, and citation format templates. The example alignment input is used to reduce ambiguity and includes expected output examples, prohibited output examples, and historical satisfactory output examples. After receiving the constraint configuration input, the application platform adds task context information to the constraint configuration input. The task context information includes project space identifier, processing task type, input data object type, and capability module type, so that the constraint set generation process has contextual guidance and reduces the probability of constraints being incorrectly applied in subsequent processing tasks.

[0023] Natural language constraint texts often suffer from diverse expressions and ambiguous parameter boundaries. Application platforms perform atomic decomposition and normalization on constraint configuration inputs. Atomic decomposition breaks down a single constraint configuration input into multiple constraint fragments, each describing a single constraint intent and a single set of parameters. Normalization merges different expressions into a unified constraint semantic type; for example, preventing a certain character from appearing is normalized to a disabled item, replacing it with a blank is normalized to a replacement item, and requiring a table field to be output is normalized to a structural requirement. The result of decomposition and normalization is represented as a set of constraint fragments. ,in, For a set of constrained fragments, For the first A constraint segment, To constrain the number of fragments, the application platform transforms the constraint configuration input from free text into a structured modelable object through splitting and normalization. This lays the foundation for the executable nature of subsequent constraint sets and reduces the probability that the constraint configuration input cannot be implemented.

[0024] In addition, when the application platform generates a constraint set, the focus is on transforming the constraints into structured objects that can be executed and verified by the program. The application platform can use a two-stage generation process: the first stage is constraint extraction, and the second stage is constraint compilation.

[0025] Specifically, in the constraint extraction phase, the application platform will set the constraint fragments. The constraint candidate set is converted into a structured set, which includes three types of fields: constraint type, constraint target, and constraint parameters. in, To constrain the candidate set, For the first Constraint candidates For constraint type, For the object of constraint, For constraint parameters, To constrain the number of candidates; constraint types should at least include character specification type, terminology specification type, structure template type, field definition type, reference specification type, and tool call restriction type; constraint application object is used to clarify the location where the constraint is applied. The constraint application object should at least include the platform output result as a whole, the platform output result field, the platform output result reference fragment, the platform output result query statement, and the tool call parameters corresponding to the platform output result. With the help of constraint application object, the application platform can avoid execution instability caused by unclear constraint description and constraint application location.

[0026] During the constraint compilation phase, the application platform will set the constraint candidate set. The code is compiled into a constraint set, and verifier and repairer representations are generated for the constraint terms in the constraint set, making the constraint terms verifiable and remediable. ,in, For a constraint set, For the first One constraint item, For the verifier representation, For repairer representation, The number of constraint terms; the validator is used to determine whether the platform output meets the constraint parameters, and the repairer is used to perform deterministic repair processing or trigger rewrite repair processing when the platform output does not meet the constraint parameters; through the introduction of the validator and repairer representations, the effectiveness of the constraint set does not depend on the memory of a single session context, but on the repeated execution of the constraint set in each output link, so that the constraints can be stably applied to subsequent processing tasks.

[0027] Preferably, during the constraint set generation stage, the application platform can evaluate the usability of constraint fragments to indicate whether there are problems such as ambiguous descriptions, missing parameters, or unclear objects in the constraint configuration input; the application platform can denote the semantic feature vector of the constraint fragment as... And calculate the usability score: ,in, For the first Availability score for each constraint fragment, For the Sigmoid function, For parameter vectors, Represents the parameter vector transpose, The usability score is used as a bias factor to support subsequent pre-validation demonstrations, enabling users to intuitively see the executability of the constraint set after submitting constraint configuration inputs, and adjust the constraint configuration inputs accordingly, thereby reducing the deviation of constraint execution from expectations from the source.

[0028] Furthermore, when the application platform incorporates constraint sets into configuration management, it registers the constraint sets as indexable configuration objects, enabling them to be reused across sessions and subject to controlled changes. Configuration management includes at least assigning configuration identifiers to constraint sets, recording version information of constraint sets, recording the applicable scope of constraint sets, and establishing the association between constraint sets and user space or project space. Through configuration management, the application platform can directly reference existing constraint sets in subsequent processing tasks, avoiding users repeatedly submitting constraint configuration inputs in different sessions, reducing redundant communication costs, and reducing the probability of platform output results deviating between different tasks. Combined with batch processing orchestration execution scenarios, the application platform uniformly references the same constraint set for multiple input data objects, making it easier to maintain stable output constraints for tasks such as batch summarization, batch field extraction, and batch report generation, and reducing the burden of manual verification.

[0029] It should be further explained that after the constraint set is formed, it needs to enter a governable state, so that users can enable, disable, and adjust the constraint set at any time during normal use, and obtain interpretable verification information before and after the adjustment. The application platform provides a management control entry point for the constraint set in configuration management. The management control entry point is connected to the interactive interface controls, project space configuration entry point, and process orchestration configuration entry point, so that the constraint set can be switched and maintained in different usage paths. Compared with temporary prompts that only rely on the dialog context, the management control of the constraint set gives the constraints traceable, rollbackable, and reusable engineering attributes, thereby supporting long-term use and multi-person collaboration in enterprise scenarios.

[0030] In a preferred embodiment, the application platform's management and control of constraint sets includes enabling, disabling, and modifying operations. Enabling operations add the constraint set to the current effective scope, disabling operations remove the constraint set from the current effective scope, and modifying operations add, delete, merge, or adjust parameters of the constraint items within the constraint set. Management control can be triggered by controls in the interactive interface, project space settings, or process orchestration configuration. By incorporating the management control entry point into configuration management, the application platform avoids the maintenance difficulties caused by constraints only being effective within a single session and allows users to reuse the same constraint set across different processing tasks. When management control is implemented, the application platform can maintain status and version markers for the constraint set. Status markers distinguish whether the constraint set is enabled in the current scope, and version markers distinguish different modified versions of the constraint set. When a modifying operation occurs, the application platform generates a new version and retains historical versions for subsequent backtracking verification and rollback operations. The direct effect of versioning is that users do not need to worry about irreversible changes to the constraint set after modifying the constraint configuration input; the constraint set can be switched between multiple versions, facilitating iteration to achieve satisfactory results.

[0031] Furthermore, the constraint set verification process answers two core questions: Is the constraint set executable, and what impact does the constraint set have on the platform's output? To this end, the application platform performs a pre-verification mechanism on the constraint set and outputs verification information. The pre-verification mechanism does not rely on the forced execution of real business data. Instead, it obtains verification information through the construction of verification objects and the execution of the verifier. The verification objects come from preset verification samples or online sampling objects. The preset verification samples can be provided by the constraint template, and the online sampling objects can come from the input data objects of recently processed tasks and the sampling of the platform's output results.

[0032] The application platform executes a validator representation of the constraint set on the verification object to obtain verification information. The verification information includes at least one or more of the following: pass / fail flags, failure reasons, a list of hit constraints, and repair suggestions. To make the verification information interpretable, the application platform can calculate the hit rate for each constraint, indicating the strength of the constraint's impact on the verification object. The hit rate can be calculated based on the matching score; for example, the feature vector of the verification object can be denoted as... Let the eigenvectors corresponding to the constraint terms be denoted as Constraints The score of the shot is recorded as : ,in, Representing vectors transpose, For vector norm; calculate vector with vector The cosine similarity, the result The larger the value, the more likely it is to be a large number. and The more similar they are, the smaller the size indicates the less similar they are or the opposite direction; molecules It's a dot product, the denominator uses... Normalizing the dot product eliminates the influence of vector length on similarity; the hit score is used to show the degree of influence of constraints on the verification object in the verification information, and to guide users to locate the constraints that need to be adjusted.

[0033] To reduce the user's understanding cost, the application platform can organize the verification information into directly actionable feedback, such as listing the constraint item identifiers corresponding to the failure reasons, listing the parameter ranges corresponding to the failure reasons, and listing the trigger fragments or trigger field positions that caused the failure, so that users can revise the constraint configuration input with minimal modification cost.

[0034] Furthermore, when the verification information fails to meet user expectations, the application platform receives revised input submitted by the user based on the verification information. The revised input can be a rewriting of the natural language constraint text, an adjustment of the constraint template parameters, or a supplement to the example alignment input. The application platform treats the revised input as a new constraint configuration input increment, and performs the aforementioned normalization, extraction, and compilation process on the revised input to form a new version of the constraint set or to perform an incremental update on the constraint set. By guiding the closed loop of the revised input through the verification information, the application platform transforms the iteration of constraint configuration input from subjective trial and error into interpretable verification-driven iteration, enabling users to converge to a constraint set that meets their needs more quickly.

[0035] Finally, after the constraint set is verified and included in the configuration management, the constraint set can be referenced by the batch orchestration process. Pre-verification before batch orchestration can reduce the risk of rework caused by discovering incompatibility of constraints after the batch job has run. Referencing the same constraint set during batch orchestration can reduce fluctuations in batch output in terms of format, fields, and references, and reduce the cost of manual verification. This is suitable for scenarios such as batch processing of documents, batch processing of data records, and batch report generation.

[0036] It's important to note that after the constraint set is constructed and governed, the application platform needs to address a key issue: during normal use, the platform continuously receives different types of processing task requests. These requests have varying task intentions, different input data object types, and the constraint set contains a large number of constraints covering different scenario boundaries. If all constraints are forcibly executed for each task request, the platform's output may suffer from over-constraint leading to content loss, or the platform's output may experience increased repair costs due to mutual constraints. Therefore, the application platform automatically determines the target constraint set when a task request occurs. The target constraint set is a subset of the constraint set and is used to limit the scope of constraints that the current task request must follow, upgrading the constraint set from applying it all at once to adapting it to the task.

[0037] In a preferred embodiment, after receiving a task request, the application platform parses the task request to obtain the task intent. The task intent characterizes the target type of the task request and can be represented as a task category label and a parameter set. The task category label includes document comprehension, knowledge question answering, question analysis, content generation, tool linkage, and workflow arrangement. The parameter set expresses additional targets such as output structure, output granularity, and output language. The application platform can obtain the task intent representation through an intent recognition model or rule parsing. The task intent is encoded into a semantic feature vector. Semantic feature vector This can be obtained from the embedding model: ,in, This represents a semantic embedding function.

[0038] Processing task requests is typically associated with input data objects, which can be user-uploaded documents, knowledge base entries, database query results, or data returned by tool interfaces. After obtaining the input data object, the application platform extracts its object features to characterize its structural and content attributes. Object features may include data object type identifiers, field structure summaries, text length ranges, language type, and whether tables or images are included. The application platform then encodes these object features into object feature vectors. Object feature vector It can be obtained by combining a structure encoder and a text encoder: ,in, Represents the input data object. Represents the structure encoding function. This represents a text encoding function. This indicates a vector concatenation operation.

[0039] Furthermore, after the constraint set enters the configuration management, the application platform maintains a matching representation for each constraint item in the constraint set, facilitating rapid selection of the target constraint set at runtime. The matching representation includes at least the applicable task label, applicable object label, constraint priority, and constraint strength. The applicable task label indicates the task boundary corresponding to the constraint item; for example, enabling SQL output constraints only for question analysis tasks and citation format constraints only for knowledge question answering tasks. The applicable object label indicates the data object boundary corresponding to the constraint item; for example, enabling field whitelist constraints only for structured data objects and paragraph template constraints only for document data objects. The constraint priority is used for the decision-making order when constraint items conflict. The constraint strength distinguishes between mandatory and optional constraints. The application platform will assign each constraint item... The matchable representation is encoded as a constraint feature vector, which can be obtained by combining label embedding and parameter embedding: ,in, For constraint type, For the object of constraint, These are constraint parameters.

[0040] In addition, the application platform is based on the semantic feature vector of task intent. Calculate the matching score between the input data object feature vector and the constraint feature vector. This is used to determine whether a constraint term enters the target constraint set; the matching score can be expressed as a weighted similarity score. ,in, For constraint terms Match score, and These are the weighting coefficients. For similarity functions; the application platform can use cosine similarity: , and Let represent two non-zero vectors with the same dimension, respectively, representing the two vectors whose similarity is to be calculated. Representing vectors with vector The similarity score usually falls within [ Within the interval [1, 1], the closer to 1, the closer the direction; the closer to 0, the closer the direction, and the closer to -1, the opposite direction. Representing vectors The transpose of is used to transform a column vector into a row vector so that it can be used with . Perform dot product operations; the application platform selects the target constraint set based on the matching score and the filtering threshold. ,in, For the target constraint set, The filtering threshold can be preset by the application platform in the configuration management or adjusted by the statistical results of the verification information, so as to achieve a balance between the coverage and false trigger rate of the target constraint set.

[0041] Therefore, by matching scores to drive selection, the target constraint set can dynamically change with the processing task request. Document comprehension tasks tend to choose structural template constraints and terminology specification constraints, knowledge question answering tasks tend to choose citation specification constraints and expression specification constraints, question number analysis tasks tend to choose SQL output constraints and caliber constraints, and tool linkage tasks tend to choose tool call restriction constraints and output field constraints, thereby reducing the burden on users to manually switch constraints.

[0042] Furthermore, after the target constraint set is selected, the application platform performs conflict handling and simplification. Conflict handling is used to avoid incompatible constraints in the target constraint set. For example, a structural template constraint requires the output to be a structured object, while a character specification constraint prohibits the use of symbols required by structured objects. The application platform makes a decision based on constraint priority and constraint strength, retaining constraints with higher priority or stronger strength. Simplification is used to delete redundant constraints. Simplification can be based on inclusion relationships. For example, if a terminology constraint already covers multiple single-replacement constraints, the terminology constraint is retained and the single-replacement constraints are removed, thereby reducing the execution overhead of subsequent constraint gating.

[0043] Furthermore, the verification information obtained by the application platform can be used to optimize matching relationships. The verification information includes the hit status of constraint items and the distribution of failure reasons. The application platform can use the verification information as the basis for updating the applicable boundaries of constraint items. For example, the statistical results of a constraint item frequently failing on a certain type of input data object can be used to reduce the corresponding matching score weight, so as to avoid the same constraint item being selected into the target constraint set in unsuitable processing task requests. The above linkage enables the selection of the target constraint set to be upgraded from static configuration to a matching strategy with feedback, thereby improving the availability of the constraint set in real business tasks.

[0044] It should be noted that processing task requests in the application platform typically requires invoking multiple capabilities, including large model generation capabilities, as well as retrieval, parsing, structured transformation, verification and repair, and tool invocation capabilities. If all capabilities are directly solidified into a single prompt word or a single chain of calls, the processing flow is difficult to reuse, difficult to dynamically adjust according to the target constraint set, and difficult to control cost and quality in batch processing orchestration scenarios. To address this, the application platform encapsulates data processing capabilities into composable large model operator units and generates a processing flow description when a task request occurs. The processing flow description is used to uniformly represent the operator invocation order, operator input and output, operator dependencies, and operator parameter configuration, upgrading the processing flow from prompt word-driven to orchestratable execution.

[0045] In a preferred embodiment, the application platform atomically encapsulates the data processing capabilities, so that each large model operator unit corresponds to a single functional objective and has clear input and output boundaries; the large model operator unit includes at least a generation operator unit and a constraint gating operator unit. The generation operator unit is used to generate candidate outputs based on the input data object, and the constraint gating operator unit is used to perform verification and repair on the candidate outputs based on the target constraint set.

[0046] To support combinatorial arrangement, each large model operator unit can be abstracted as a quintuple representation: ,in, Indicates the operator name, Indicates input data structure constraints. This indicates the constraints of the output data structure. Indicates the operator's operating parameters. This represents resource metadata; resource metadata is used to characterize the cost and risk attributes of the operator during runtime. Cost attributes include token budget, latency budget, and cost budget, while risk attributes include whether external tools are allowed to call the operator and whether cross-data domain references are allowed. This encapsulation enables the application platform to reuse operator units in different processing task requests and make selections at the orchestration layer based on resource and risk requirements.

[0047] For constraint-gated operator units, the operator operating parameters include the verifier representation and repairer representation corresponding to the target constraint set, enabling the constraint-gated operator unit to be parameterizable; the output of the constraint-gated operator unit can be either the platform output result after verification or an intermediate result including the reason for failure, which is used to drive subsequent branches.

[0048] Furthermore, the application platform selects a process skeleton based on the task intent and input data object type of the task request. The process skeleton describes the typical processing stages of the task request, such as the input parsing stage, generation stage, gating stage, post-processing stage, and tool linkage stage. The process skeleton can be represented as an operator sequence or an operator graph structure. To accommodate batch processing and parallel execution, the application platform can use a directed acyclic graph to represent the processing flow description. ,in, Represents the set of operator nodes. This represents the set of operator dependencies; dependencies represent data dependencies. The existence of dependencies ensures the correct execution order of operators, while the absence of dependencies allows operators to execute in parallel. The introduction of the process skeleton enables the application platform to reuse the overall structure between different processing task requests, and to adjust only the local node parameters or local branches in the process skeleton when the target constraint set changes.

[0049] Furthermore, once the target constraint set is determined, the application platform maps the target constraint set to the constraint parameters in the processing flow description. The target constraint set has at least three types of impact on the processing flow description.

[0050] The first type of impact is the parameter assembly of the constraint gating operator unit. The application platform assembles the compiled product of the constraint terms in the target constraint set into the constraint gating operator unit, enabling the constraint gating operator unit to perform verification and repair according to the target constraint set.

[0051] The second type of impact is the injection of generation constraints into the generation operator unit. The application platform converts the structural template constraints, field caliber constraints, and citation specification constraints in the target constraint set into hint constraints or output structure constraints for the generation operator unit. This ensures that the generation operator unit generates candidate outputs that are as close as possible to the target constraint set, reducing subsequent repair overhead. Generation constraint injection can use constraint summary vectors. This is represented and passed as part of the generation parameters to the generation operator unit: ,in, Represents the target constraint set. Represents the eigenvector of the constraint term. The aggregation function can be weighted summation or attention aggregation; the constraint summary vector is used to compress the information of multiple constraint terms into a constraint representation that can be used by the generating operator unit, thereby avoiding the cost increase caused by repeatedly stacking long text in the prompt.

[0052] The third type of impact is the configuration of process branch conditions. The application platform configures branch conditions based on the constraint strength and risk attributes in the target constraint set. For example, when the target constraint set includes high-risk constraints, a more stringent verification branch is enabled; when the target constraint set includes structured output constraints, a structured formatting branch is enabled; and when the target constraint set includes tool call restriction constraints, a tool linkage branch is disabled. The above impacts make the process description have strategic characteristics, and the process description can change with the target constraint set.

[0053] In addition, the application platform considers batch processing orchestration and execution when generating the processing flow description. For scenarios where the input data objects are in the form of sets, the application platform marks parallelizable nodes and batchizable nodes in the processing flow description, so that subsequent batch processing orchestration and execution can merge and schedule the same operators on multiple input data objects. The application platform can also mark shared resource nodes in the processing flow description, such as shared retrieval nodes, shared parsing nodes, and shared constraint compilation nodes, so that the output of shared nodes can be reused during batch processing execution, reducing redundant calculations.

[0054] It should be noted that after the processing flow description is generated, the application platform needs to achieve two types of objectives in the real operating environment. One objective is to achieve scalable batch processing throughput for the set of input data objects, and the other objective is to ensure that the platform output results meet the target constraint set in terms of rules, templates, definitions, and references. To this end, the application platform performs batch processing orchestration on the input data objects according to the processing flow description. The batch processing orchestration is responsible for mapping multiple input data objects to operator execution plans and implementing parallel scheduling, batch merging, and failure isolation during execution. After the application platform generates the operator unit and obtains the candidate output, it performs verification and repair on the candidate output through the constraint gating operator unit, and outputs the platform output results that meet the target constraint set, thereby incorporating the large model generation results into the controllable output link.

[0055] In a preferred embodiment, input data objects often appear in the platform as collections, such as batch documents, batch knowledge entries, batch data records, and batch query requests; the application platform represents the collection of input data objects as follows: ,in, For the input data object collection, For the first One input data object, The number of input data objects is specified. The application platform generates an execution plan based on the processing flow description. The execution plan describes the scheduling method of each operator on the set of input data objects, including serial nodes, parallel nodes, and batch nodes. Batch nodes are used to merge and schedule the same operator on multiple input data objects, reducing the number of model calls and network overhead. Parallel nodes are used to run independent operators in parallel, improving throughput.

[0056] Furthermore, the batch node merging strategy can be driven by resource budget constraints, including token budget, latency budget, and cost budget; the application platform estimates the operator call cost for each input data object and determines the batch size. A feasible way to determine the batch size is: ,in, This represents the token cost estimate for a single input data object. Indicates the upper limit of the token budget. This indicates a candidate value for batch size. Through budget-driven batch size control, the application platform can maintain a controllable balance between throughput and cost, and avoid excessively large batches that could lead to truncation of the generator operator output or increased failure retries.

[0057] Furthermore, during batch processing and orchestration, the generation operator unit performs generation processing on the input data object to obtain a candidate output set: ,in, For the candidate output set, For input data objects The candidate outputs are used to enter the verification and repair phase of the constraint gating operator unit. To support subsequent traceability and debugging, the application platform internally saves the candidate outputs and related running information, including the processing task request identifier, input data object identifier, target constraint set identifier, operator version identifier, and timestamp information, so that the cause can be located when unexpected outputs occur.

[0058] Furthermore, after the candidate outputs are generated, the application platform applies constraint processing corresponding to the target constraint set to the candidate outputs. This constraint processing is executed by a constraint gating operator unit, which loads a set of verifier representations of the target constraint set. This set of verifier representations is used to determine whether the candidate outputs satisfy the target constraint set. The verification result can be represented as a constraint satisfaction vector. ,in, For the target constraint set, Indicates candidate output For constraint terms Whether it passes; when it exists At that time, the constraint gating operator unit outputs failure reason information, which is used to drive the selection of repair strategy and for subsequent verification information statistics.

[0059] Furthermore, to avoid over-repairing candidate outputs and causing content drift, the application platform prioritizes deterministic repair processing. This deterministic repair processing, based on the repairer representation, performs operations such as string replacement, symbol normalization, field completion, field sorting, and reference formatting, ensuring that candidate outputs satisfy the target constraint set without altering their primary semantic content. Deterministic repair processing can be represented as a function: ,in, This represents a deterministic repair function. This represents the intermediate output after deterministic repair.

[0060] When deterministic repair processing still fails to make the candidate output satisfy the target constraint set, the application platform triggers rewrite repair processing. Rewrite repair processing calls the generation operator unit or a dedicated repair operator unit to perform minimal modification rewriting of the candidate output under the constraints of the failure cause information, avoiding drift caused by rewriting irrelevant content. Rewrite repair processing can be represented as: in, This indicates that the repair function should be rewritten. This indicates the failure reason information corresponding to the candidate output. This indicates the output after rewriting and repair; the application platform can limit the number of retries for rewriting and repair. To avoid uncontrolled resource consumption due to non-convergence during the repair process, the budget limit should be set.

[0061] The set of platform output results that satisfy the target constraint set from the output of the constraint-gated operator unit: ,in, This indicates the final platform output result; the platform output result is used for display in the interactive interface, for external deployment and calling by the application, and for writing to downstream systems.

[0062] Finally, after the batch processing orchestration and execution are completed, the application platform summarizes the statistical information generated during the execution process. The statistical information includes the verification pass rate, the distribution of failure reasons, the number of deterministic repairs, the number of rewrite repairs, and the average cost consumption. The statistical information is used to support the updating of verification information, the adjustment of matching weights, and the continuous iteration of the constraint set in real business scenarios. Through batch-level feedback accumulation, the application platform forms a closed-loop governance path from constraint configuration input to constraint set, then to target constraint set, and finally to controlled output results. This ensures that the platform's output results remain controllable in different processing task requests and facilitates continuous optimization.

[0063] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. An intelligent data processing method based on large model atomization encapsulation and batch processing orchestration, characterized in that, Includes the following steps: S1. Receive constraint configuration input submitted by the user in the interactive interface of the application platform, generate a constraint set for constraining the output of the platform based on the constraint configuration input, and incorporate the constraint set into the platform's configuration management. S2. Perform verification on the constraint set through the pre-verification mechanism and output verification information. If the verification information does not meet expectations, receive the user's revision input and update the constraint set according to the revision input. S3. When the application platform receives a task processing request, it parses the task processing request to obtain the task intent and retrieves the input data object associated with the task processing request. Based on the matching results of the task intent features, input data object features and constraint set, it determines the target constraint set. S4. Encapsulate the data processing capabilities for input data objects into composable large model operator units, and generate a processing flow description based on the large model operator units and the target constraint set; S5. Based on the processing flow description, perform batch processing and arrangement on the input data objects to obtain candidate outputs, and then perform constraint processing on the candidate outputs corresponding to the target constraint set before outputting the results.

2. The intelligent data processing method based on large model atomic encapsulation and batch processing orchestration according to claim 1, characterized in that, The constraint configuration input comes from natural language constraint text, constraint template, and example alignment input, and the constraint configuration input consists of one or more of the above input forms.

3. The intelligent data processing method based on large model atomic encapsulation and batch processing orchestration according to claim 2, characterized in that, Generating a constraint set based on the constraint configuration input includes: performing atomic decomposition on the constraint configuration input to obtain multiple constraint fragments, and performing normalization processing on the multiple constraint fragments to obtain structured constraint items. The structured constraint items include constraint type, constraint object, and constraint parameters.

4. The intelligent data processing method based on large model atomic encapsulation and batch processing orchestration according to claim 3, characterized in that, Generating the constraint set includes performing a compilation process on the structured constraint items, generating a validator representation and a repairer representation for each structured constraint item.

5. The intelligent data processing method based on large model atomization encapsulation and batch processing orchestration according to claim 4, characterized in that, Including the constraint set in configuration management includes: assigning a configuration identifier to the constraint set and recording version information, configuring the applicable scope for the constraint set, and retaining the historical versions of the constraint set.

6. The intelligent data processing method based on large model atomic encapsulation and batch processing orchestration according to claim 5, characterized in that, Performing verification on the constraint set includes: constructing a verification object based on a preset verification sample, or constructing a verification object based on an online sampling object of a processing task; performing verification processing on the verification object corresponding to the constraint set; and outputting verification information, which includes at least one or more of the following: pass flag, failure reason, list of hit constraint items, and revision suggestions.

7. The intelligent data processing method based on large model atomic encapsulation and batch processing orchestration according to claim 6, characterized in that, The process of receiving user revision input based on verification information includes receiving one or more of the following: rewriting input for constraint configuration input, parameter adjustment input, and example supplement input. The process also involves updating the constraint set based on the revision input, generating a new version and recording the change record associated with the verification information when updating the constraint set.

8. The intelligent data processing method based on large model atomic encapsulation and batch processing orchestration according to claim 7, characterized in that, Determining the target constraint set based on the matching relationship between the processing task request and the constraint set includes: parsing the processing task request to obtain the task intent and acquiring the input data object; extracting the task intent features and input data object features; performing matching calculations on the constraint items in the constraint set based on the task intent features and input data object features to obtain a matching score; and determining the target constraint set based on the matching score and a filtering threshold.

9. The intelligent data processing method based on large model atomic encapsulation and batch processing orchestration according to claim 8, characterized in that, The determination of the target constraint set includes performing conflict processing on the target constraint set. Conflict processing is used to identify constraint items that have incompatible constraint parameters for the same constraint object, and retains the constraint items that satisfy the adjudication rules based on constraint priority and constraint strength, so as to obtain the target constraint set after conflict resolution.

10. The intelligent data processing method based on large model atomic encapsulation and batch processing orchestration according to claim 9, characterized in that, The constraint processing corresponding to the target constraint set for the candidate output includes: performing deterministic repair processing on the candidate output based on the repairer representation first; triggering rewrite repair processing when the target constraint set is still not satisfied after deterministic repair processing; setting an upper limit for the number of retries and an upper limit for the resource budget for rewrite repair processing; and recording the constraint set version information and constraint processing trajectory information associated with the processing task request.

Images