A multi-tenant agent policy learning method

Abstract

This invention provides a multi-tenant agent policy learning method, relating to the fields of machine learning and agent policy optimization. The method includes generating task constraint fingerprints and knowledge call context snapshots based on historical task execution records; constructing counterfactual comparison samples containing basic policy paths and revised policy paths under an attribution control constraint set; obtaining the net feedback gain and tenant isolation reward value of the revised action through attribution separation processing; determining the migration eligibility of the revised action based on reward direction consistency, task constraint fingerprint matching degree, and risk rule conflict degree; updating the shared policy model and the tenant private policy adapter respectively; and generating agent output based on the updated model. This invention can improve the attribution reliability between manual revision results and business feedback under multi-tenant constraints, enabling cross-tenant experience adaptation and differentiated policy updates.

Description

Technical Field

[0001] This invention relates to the field of machine learning and agent policy optimization technology, and in particular to a multi-tenant agent policy learning method. Background Technology

[0002] Multi-tenant architecture is a core supporting technology for enterprise software services (SaaS). With the maturity of large-scale language models and intelligent agent technology, more and more enterprise applications are introducing intelligent agents to complete tasks such as knowledge retrieval, business decision-making, and content generation. In a multi-tenant scenario, different tenants can be different enterprises or different departments or business units within the same enterprise. Each tenant typically has relatively independent business rules, risk requirements, knowledge content, and task processing habits. To balance system operating efficiency and business adaptation needs, enterprise-level intelligent agents usually need to reuse some common capabilities across multiple tenants and adapt to the specific task requirements of different tenants.

[0003] Existing enterprise-level AI agents typically iteratively optimize models or related strategies by collecting feedback from business personnel regarding modifications made to the agent's output, as well as feedback information such as adoption rate, conversion rate, and completion rate after task execution. However, the effectiveness of AI agent output is often influenced by factors such as task conditions, invoked knowledge content, business stage, external environment, and user status. Manual revisions and corresponding feedback cannot directly reflect the actual effect of a specific revision action. When task conditions or knowledge content change, the feedback may be mixed with the impact of environmental changes, thus affecting the accuracy of subsequent strategy optimization.

[0004] Meanwhile, different tenants may have different business rules and risk requirements for the same type of task. Content adjustment methods or task processing experience applicable in one tenant may lead to adaptation discrepancies or risk issues when applied to other tenants. On the other hand, feedback from enterprise-level tasks typically has a certain lag, and the amount of effective feedback available under a single tenant is limited, making it difficult for existing systems to reliably determine the applicability of manual revision experience under different tenant and task conditions. Therefore, improving the reliability of the correlation judgment between manual revision results and business feedback in multi-tenant intelligent agents, and enhancing the adaptability of strategy experience across different tenant tasks, have become technical problems that need to be solved. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, the technical problem to be solved by this invention is to propose a multi-tenant agent policy learning method, which adopts the following technical solution: A multi-tenant agent policy learning method includes the following steps: S1: Obtain historical task execution records including task constraint data, knowledge call records, original output, manual revision results, and business feedback results; based on the task constraint data and knowledge call records, generate task constraint fingerprints and knowledge call context snapshots respectively, and determine the revision actions corresponding to the manual revision results and their attribution relationships; S2: Construct an attribution control constraint set based on the task constraint fingerprint and knowledge call context snapshot. Under the condition of satisfying the attribution control constraint set, generate the original policy output based on the basic policy path and the revised policy output based on the revised policy path containing the above revision actions, forming a counterfactual comparison sample. S3: Determine the business feedback baseline value based on the historical task execution records matched with the counterfactual control sample, and perform attribution separation processing on the business feedback results based on the business feedback baseline value, external environment characteristics and task constraint fingerprints to obtain the net feedback gain and tenant isolation reward value corresponding to the revision action; S4: Based on the consistency of the reward direction of the tenant isolation reward value among multiple tenants and the matching relationship between the task constraint fingerprints, determine the migration eligibility corresponding to the revision action, and based on the migration eligibility, use the revision action with shared update eligibility to update the shared policy model, and use the revision action with tenant private update eligibility to update the private policy adapter of the corresponding tenant. S5: Obtain the task to be processed, generate the current task constraint fingerprint and the current knowledge call context snapshot; determine the knowledge call strategy and output generation strategy based on the shared policy model and the corresponding private policy adapter, and generate the agent output results.

[0006] As a further improvement, in step S2, the attribution control constraint set includes the tenant identifier, target platform, business stage, knowledge version, and risk rule in the task constraint fingerprint, as well as the retrieval path and candidate knowledge object set in the knowledge invocation context snapshot; the original policy output and the revised policy output are generated based on the same task input, knowledge invocation context snapshot, and candidate knowledge object set.

[0007] As a further improvement, in step S3, historical task execution records are filtered according to the field correspondence between the task constraint fingerprint and the knowledge retrieval context snapshot. Historical task execution records that match the constraint fields in the task constraint fingerprint with the attribution control constraint set and match the retrieval path in the knowledge retrieval context snapshot with the candidate knowledge object set are determined as historical task samples.

[0008] As a further improvement, in step S4, the determination of migration eligibility corresponding to the revision action includes: Obtain the predefined risk rule set corresponding to each tenant; Based on the knowledge objects and output fragments associated with the above revision actions, determine the number and risk level of risk rules triggered in each predefined risk rule set, and calculate the degree of conflict between the above revision actions and each tenant's risk rules accordingly. Based on the above-mentioned consistency in reward direction, matching relationship, and degree of conflict, the migration eligibility corresponding to the above-mentioned revision actions is determined.

[0009] As a further improvement, when the tenant isolation reward value of the revision action in multiple tenants is greater than the preset shared reward threshold, the matching degree between the corresponding task constraint fingerprints is greater than the preset matching threshold, and the conflict degree between the revision action and the risk rules of each tenant is less than the preset conflict threshold, the revision action is determined to be eligible for shared update. When the tenant isolation reward value of the revision action in the current tenant is greater than the preset private reward threshold, and the matching degree between the revision action and the task constraint fingerprint corresponding to the current tenant is greater than the preset private matching threshold, and the conflict degree between the revision action and other tenant risk rules is greater than the preset private conflict threshold, the revision action is determined to have tenant private update qualification.

[0010] As a further improvement, in step S3, based on the task constraint fingerprint of the counterfactual comparison sample, historical task execution records that meet the preset similarity threshold are selected from the above historical task samples, and the business feedback results of each historical task execution record are weighted according to the task constraint similarity, knowledge version matching degree, time decay coefficient and feedback credibility to obtain the business feedback baseline value.

[0011] To further improve the process, the initial reward value is determined based on the gain of the corresponding business feedback result output by the revision strategy relative to the business feedback baseline value. The confidence level of the initial reward value is adjusted based on the feedback delay duration and the number of historical task samples. The risk penalty item is determined based on the risk interception result to obtain the aforementioned tenant isolation reward value. Revision actions with a confidence level less than a preset threshold are stored in the experience set to be verified, and isolation verification processing is performed based on the experience set to be verified.

[0012] As a further improvement, in step S1, the original output and the manually revised result are aligned to identify the corresponding difference positions and semantic change types; the knowledge objects associated with each difference position are determined according to the knowledge call records, and the revision action is determined based on the above difference positions, semantic change types and associated knowledge objects, and the attribution relationship between the revision action and the output segment, associated knowledge object and associated risk rule is established.

[0013] As a further improvement, in step S5, the private policy adapter outputs a residual policy vector based on the intermediate features of the current task constraint fingerprint and the shared policy model, and fuses the residual policy vector with the basic policy vector output by the shared policy model to obtain the above-mentioned output generation policy.

[0014] Compared with the prior art, the beneficial effects of the present invention are: Firstly, this invention generates task constraint fingerprints and knowledge call context snapshots by acquiring task constraint data, knowledge call records, original outputs, manual revision results, and business feedback results from historical task execution records. Under the constraints of the attribution control constraint set, it constructs counterfactual comparison samples based on the basic policy path and the revision policy path containing revision actions, enabling manual revision results to be compared under similar task and knowledge call conditions. Combining the business feedback baseline value, external environment characteristics, and task constraint fingerprints, the business feedback results are subjected to attribution separation processing to obtain the net feedback gain and tenant isolation reward value corresponding to the revision action. This reduces the interference of changes in task conditions, knowledge content, and external environment on the policy learning results, avoids misjudging environmental noise as the effectiveness of revision actions, and improves the reliability of feedback signals corresponding to manual revision experience.

[0015] Secondly, this invention determines the migration eligibility corresponding to the revision action based on the consistency of the reward direction among multiple tenants and the matching relationship between the tenant isolation reward value and the task constraint fingerprint. Revision actions with shared update eligibility are used to update the shared policy model, and revision actions with tenant private update eligibility are used to update the private policy adapter of the corresponding tenant. This enables revision experience in different tenants to participate in policy updates according to their applicable scope, reduces the probability of local tenant experience causing adaptation bias or risk bias in other tenants, and improves the adaptability and stability of the multi-tenant agent output results.

[0016] Third, this invention selects historical task samples from historical task execution records based on the task constraint fingerprints of counterfactual comparison samples, and determines the business feedback baseline value by combining task constraint similarity, knowledge version matching degree, time decay coefficient and feedback credibility. This can improve the representativeness of the business feedback baseline value when there is a delay in business feedback, a limited number of samples or fluctuations in the external environment, and provide a more reliable reference for calculating the net feedback gain of revision actions and updating strategies. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the steps of the present invention. Detailed Implementation

[0019] To facilitate understanding by those skilled in the art, the present invention will now be described in further detail with reference to the accompanying drawings: This technical solution provides a multi-tenant intelligent agent policy learning method, applied to an enterprise-level intelligent agent service platform. This platform provides intelligent services such as knowledge retrieval, content generation, business decision support, process suggestions, and business Q&A to multiple tenants. Different tenants can correspond to different enterprises, or to departments, product lines, and business units within the same enterprise that have independent business rules, product information, or risk requirements.

[0020] The platform can maintain task data, knowledge objects, risk rules, historical output records, manual revision records, and business feedback records separately for different tenants. Knowledge objects can include business rule documents, product information, process descriptions, content templates, case summaries, Q&A records, operating procedures, and review requirements. Each knowledge object can store information such as tenant affiliation, version status, applicable platform, applicable business stage, permission scope, validity period, summary content, overview content, and detailed content.

[0021] The summary content outlines the core theme and scope of application of the knowledge object, the overview content records business rules, processing conditions, and key requirements, and the detailed content records complete business descriptions, specific cases, and operational requirements. When an agent executes a task, it can first read the summary content related to the task, and then read the overview content or detailed content in conjunction with the task complexity, resource budget, and risk requirements, in order to control the context length and ensure the relevance of knowledge retrieval.

[0022] The business feedback results in this technical solution can be determined based on the actual application scenario. In content operation scenarios, business feedback results may include content adoption rate, click-through rate, conversion rate, interaction rate, dwell time, and risk review results. In enterprise Q&A scenarios, business feedback results may include problem resolution rate, proportion of referrals to human agents, user satisfaction, and response adoption rate. In process support scenarios, business feedback results may include task completion rate, processing time, number of human rollbacks, and execution success rate. Business feedback results can be provided by content delivery systems, customer service systems, task management systems, review systems, business databases, or human evaluation terminals.

[0023] like Figure 1 As shown, the method includes the following steps.

[0024] Step S1: Obtain historical task execution records including task constraint data, knowledge call records, original output, manual revision results, and business feedback results; generate task constraint fingerprints and knowledge call context snapshots based on task constraint data and knowledge call records, and determine the revision actions corresponding to the manual revision results and their attribution relationships.

[0025] Historical task execution records correspond to task instances that the agent has already processed. Each task instance can be assigned a unique task identifier to associate data from task input, knowledge retrieval, content generation, manual revision, business execution, and feedback collection processes.

[0026] Task constraint data is used to characterize the business boundaries and processing conditions of the current task, and may include tenant identifier, product identifier, target platform, business stage, task type, target user group, permission scope, content risk level, resource budget, and knowledge version requirements.

[0027] For example, for product promotion content generation tasks, task constraint data may include the brand owner corresponding to the current tenant, the content community platform corresponding to the target platform, the new product pre-launch stage corresponding to the business stage, the skincare product category corresponding to the product category, the young consumer group corresponding to the target user group, and the general commercial content level corresponding to the risk level. For internal Q&A tasks, task constraint data may include the department, question topic, permission level, applicable process version, and question urgency.

[0028] Task constraint fingerprints are used to convert task constraint data into a comparable data representation. Task constraint fingerprints can be formed using field encoding, field hash values, vector representation, or a combination of multiple representation methods.

[0029] In one specific embodiment, the platform concatenates the tenant identifier, target platform, business stage, knowledge version, and risk rule identifier according to a preset field order, and encodes the concatenation result to obtain a task constraint fingerprint. In another embodiment, the platform vectorizes the tenant identifier, business stage, task type, product category, target user group, and risk level respectively, and then combines the vectors according to preset weights to obtain a task constraint vector as the task constraint fingerprint.

[0030] Different constraint fields can be assigned different comparison priorities. Tenant ID, risk rules, knowledge version, and business stage can be set as high-priority fields, while target user group, resource budget, and task type can be set as medium-priority fields. When filtering historical tasks later, high-priority fields can be compared first, and then the similarity between medium-priority fields can be calculated.

[0031] Knowledge retrieval records are used to document the process by which an agent invokes knowledge objects when handling tasks. These records may include information such as retrieval request, retrieval scope, retrieval path, candidate knowledge objects, candidate knowledge object ranking, actual knowledge object invoked, knowledge object version, knowledge object metadata, context assembly order, context length, and resource consumption.

[0032] The search path can be a knowledge base directory path, a business category path, a permission path, a knowledge tag path, or other data paths used to limit the scope of knowledge. For example, a tenant's marketing content task can be searched under the tenant-specific directory, product information directory, platform rules directory, and review rules directory.

[0033] A knowledge retrieval context snapshot is used to record the actual knowledge environment used by the agent when generating the raw output. A knowledge retrieval context snapshot can include the retrieval path, the set of candidate knowledge objects, the ranking of candidate knowledge objects, the actual knowledge object invoked, the version status of each knowledge object, summary content, overview content, detailed content, and the context assembly result.

[0034] By generating knowledge retrieval context snapshots, the knowledge retrieval state corresponding to the original output can be preserved. When the knowledge base content is updated, the retrieval scope changes, or the knowledge object version is switched, historical tasks can still be analyzed based on the corresponding snapshots, thus enabling subsequent processing to obtain knowledge conditions that match the original output.

[0035] In step S1, the original output and the manually revised result are aligned to identify the corresponding difference positions and semantic change types. The knowledge objects associated with each difference position are determined according to the knowledge call records. The revision action is determined based on the difference position, semantic change type and associated knowledge object. The attribution relationship between the revision action and the output segment, associated knowledge object and associated risk rule is established.

[0036] Specifically, the platform can first process the original output and the manually revised results by sentence segmentation, paragraphing, or semantic unit segmentation to form multiple output segments. Subsequently, the platform can use edit distance calculation, semantic similarity calculation, text alignment algorithm, or pre-trained language model encoding method to align the original output segments and the manually revised result segments, and identify deletion positions, insertion positions, replacement positions, and rearrangement positions.

[0037] For the identified discrepancies, the platform further determines the type of semantic change. For example, changing "suitable for all people" to "recommended for use based on skin type" can be identified as an adjustment to the scope of application. Changing "the product has therapeutic effects" to "the product has an auxiliary improvement effect" can be identified as an adjustment to risk expression. Adding information such as applicable time periods, service procedures, applicable restrictions, or precautions can be identified as supplementing business rules.

[0038] The platform determines the knowledge objects associated with the differences based on knowledge call records. These associations can be determined based on text citation location, semantic similarity, the order in which the knowledge objects appear in the context, the attention weights corresponding to the knowledge objects during generation, or the number of times the knowledge objects are called.

[0039] For example, if the product efficacy description in the output fragment has a high semantic similarity to the efficacy statement in the product information, and the product information is recorded as an actual knowledge object being called, the platform can identify the product information as an associated knowledge object. For differences involving risk expression, the platform can also identify associated risk rules, such as advertising compliance rules, brand expression rules, privacy protection rules, industry regulatory rules, or platform review rules.

[0040] Revision actions can be stored in a structured record format. Each revision action can include an action identifier, action type, original fragment, revised fragment, difference location, semantic change type, associated knowledge object, associated risk rule, task identifier, and reviser identifier.

[0041] Attribution relationships can be stored in the form of relationship tables, graph structures, or event logs. When stored in a graph structure, revision actions, output fragments, knowledge objects, and risk rules can be used as different types of nodes, and connections can be established between nodes through modification relationships, reference relationships, trigger relationships, or application relationships.

[0042] Step S2: Construct an attribution control constraint set based on the task constraint fingerprint and knowledge call context snapshot. Under the condition of satisfying the attribution control constraint set, generate the original policy output based on the basic policy path and the revised policy output based on the revised policy path containing the revision action, thus forming a counterfactual control sample.

[0043] The attribution control constraint set is used to limit the consistent task conditions and knowledge retrieval conditions when constructing counterfactual control samples. The attribution control constraint set can be jointly determined by task constraint fingerprints and knowledge retrieval context snapshots, so that the difference between the basic policy path and the revised policy path focuses on the revision actions and their corresponding policy adjustment content.

[0044] In step S2, the attribution control constraint set includes the tenant identifier, target platform, business stage, knowledge version, and risk rule in the task constraint fingerprint, as well as the retrieval path and candidate knowledge object set in the knowledge invocation context snapshot; the original policy output and the revised policy output are generated based on the same task input, knowledge invocation context snapshot, and candidate knowledge object set.

[0045] Tenant identifier is used to identify the tenant to which the task belongs. Target platform is used to identify the platform where content is published, business interactions occur, or the task is executed. Business stage is used to identify whether the task is in a new product launch, daily operations, after-sales service, risk review, or other business stage. Knowledge version is used to identify the versions of business data, rule data, and product data that can be accessed during task processing. Risk rules are used to identify the applicable review requirements and risk handling requirements for the current task.

[0046] The retrieval path and candidate knowledge object set are used to limit the scope of knowledge sources when generating output. The candidate knowledge object set can consist of knowledge objects that meet the task constraints under the retrieval path, where each knowledge object can have a different sorting position and invocation priority.

[0047] When constructing counterfactual comparison samples, the platform uses the basic policy path as the comparison path. The basic policy path can adopt policy parameters, prompt word parameters, knowledge object ranking weights, context reading levels, or candidate output selection parameters used when generating the original output in historical tasks.

[0048] The revision strategy path includes the revision actions corresponding to the manual revision results. Revision actions can be translated into adjustments to strategy parameters, cue word constraints, output sorting, knowledge object references, or risk thresholds.

[0049] For example, when a manual revision involves replacing a product term with one that conforms to brand specifications, the revision strategy path can add expression constraints related to that term. When a manual revision involves deleting risk expressions, the revision strategy path can increase the constraint weight of associated risk rules. When a manual revision involves supplementing business process descriptions, the revision strategy path can increase the priority of calling the corresponding process knowledge objects.

[0050] The basic strategy path and the revised strategy path generate corresponding outputs based on the same task input, the same knowledge retrieval context snapshot, and the same set of candidate knowledge objects. The platform can generate the outputs of both strategy paths using the same model version, the same input template, the same context assembly order, and the same generation budget.

[0051] When using random sampling to generate output, the platform can fix the random seed or generate multiple sets of original policy outputs and revised policy outputs, and construct counterfactual control samples based on the statistical results of each set of outputs, thereby reducing the fluctuations caused by random sampling.

[0052] Counterfactual comparison samples may include task identifiers, task constraint fingerprints, knowledge retrieval context snapshots, basic policy paths, revised policy paths, original policy outputs, revised policy outputs, revision actions, and associated knowledge objects.

[0053] Step S3: Determine the business feedback baseline value based on the historical task execution records matched with the counterfactual control sample, and perform attribution separation processing on the business feedback results based on the business feedback baseline value, external environment characteristics and task constraint fingerprints to obtain the net feedback gain and tenant isolation reward value corresponding to the revision action.

[0054] The business feedback baseline is used to represent the level of feedback that historical tasks typically achieve under task conditions similar to the current counterfactual control sample. The business feedback baseline can be a statistical result of click-through rate, conversion rate, adoption rate, task completion rate, risk pass rate, or other business feedback metrics.

[0055] In different business scenarios, the platform can choose a single feedback metric as the baseline value for business feedback, or it can construct a comprehensive feedback baseline value based on multiple feedback metrics. For example, content generation tasks can use a combination of content adoption rate, conversion rate, and risk pass rate as the comprehensive feedback metric. Enterprise Q&A tasks can use a combination of problem resolution rate, the proportion of cases referred to human agents, and user satisfaction as the comprehensive feedback metric.

[0056] In step S3, historical task execution records are filtered according to the field correspondence between the task constraint fingerprint and the knowledge call context snapshot. Historical task execution records that match the constraint fields in the task constraint fingerprint with the attribution control constraint set and match the retrieval path in the knowledge call context snapshot with the candidate knowledge object set are identified as historical task samples.

[0057] Specifically, the platform can first perform field matching based on tenant identifier, target platform, business stage, knowledge version, and risk rules, and then compare the search path and candidate knowledge object set.

[0058] For cases where field values ​​are completely identical, the platform can directly determine a field match. For cases where field values ​​have hierarchical or semantically similar relationships, the platform can calculate the field match degree. For example, if two tasks are both in the content operation phase, one in the new product pre-launch phase and the other in the new product launch phase, the platform can determine that the two have a corresponding phase match degree based on the preset business phase relationship.

[0059] Candidate knowledge object sets can be compared using set intersection and union ratio, knowledge object version consistency, knowledge object category matching degree, or knowledge object vector similarity. Search paths can be compared using path prefix matching, directory level matching, or tag set matching. When the task constraint fields, search path, and candidate knowledge object sets of a historical task meet preset conditions, the historical task execution record is determined as a historical task sample.

[0060] In step S3, based on the task constraint fingerprint of the counterfactual comparison sample, historical task execution records that meet the preset similarity threshold are selected from the historical task samples. The business feedback results of each historical task execution record are weighted according to the task constraint similarity, knowledge version matching degree, time decay coefficient and feedback credibility to obtain the business feedback baseline value.

[0061] Task constraint similarity describes the degree of similarity between the counterfactual control sample and the historical task sample in terms of task conditions. The platform can calculate the field similarity between tenant, platform, business stage, task type, product category, target user group, and risk level, and obtain the task constraint similarity according to preset weights.

[0062] Knowledge version matching describes the degree of similarity between the versions of knowledge objects used in the current task and those used in historical tasks. The time decay coefficient reduces the impact of historical feedback results with large time intervals on the current baseline value. Feedback credibility describes the reliability of historical feedback results and can be determined based on feedback data completeness, feedback collection source, feedback sample size, task execution duration, and anomaly labeling.

[0063] In one specific embodiment, for the first Based on historical task execution records, the platform calculates the business feedback weight using the following formula. :

[0064] in, Indicates the first The similarity of task constraints between historical task execution records and counterfactual control samples. Indicates the first The degree of knowledge version matching between historical task execution records and counterfactual comparison samples. Indicates the first The time interval between the feedback time corresponding to each historical task execution record and the current task time. Indicates the first The reliability of feedback from historical task execution records. , , , This represents the preset weighting coefficients, and satisfies: .

[0065] The platform uses business feedback results from historical task execution records. and corresponding business feedback weights Perform a weighted average to obtain the business feedback baseline value. :

[0066] in, This indicates the number of historical task execution records that participated in the calculation of the business feedback baseline value. Using the above calculation method, historical tasks that are closer to the current task conditions, have a more similar knowledge version, a shorter time interval, and higher feedback reliability can occupy a higher weight in the calculation of the business feedback baseline value.

[0067] External environment characteristics are used to describe the external conditions that affect business feedback results. External environment characteristics may include task execution time periods, platform traffic status, marketing campaign status, user activity, product inventory status, price adjustment status, holiday information, business system load, and abnormal event markers, etc.

[0068] The platform can collect external environmental characteristics from business systems, content delivery systems, operations systems, inventory systems, or log systems. For example, external environmental characteristics corresponding to content delivery tasks may include the delivery time period, platform exposure, number of similar activities, and product promotion status. External environmental characteristics corresponding to enterprise Q&A tasks may include peak user access times, system load, and question type distribution.

[0069] The platform performs attribution separation processing on business feedback results based on business feedback baseline values, external environment characteristics, and task constraint fingerprints. Attribution separation processing can be implemented using rule correction, statistical regression, weighted residual calculation, causal effect estimation, or trained feedback prediction models.

[0070] In one implementation, the platform trains a feedback prediction model based on historical task samples. The feedback prediction model takes task constraint fingerprints, external environment features, and knowledge retrieval features as input and outputs predicted business feedback values. The platform compares the actual business feedback results with the predicted business feedback values ​​to obtain a feedback deviation value, and then combines this with the business feedback baseline value to determine the net feedback gain corresponding to the revision action.

[0071] For example, the business feedback result is the content conversion rate, the business feedback baseline is the average conversion rate of similar tasks, and the feedback prediction model predicts the expected conversion rate based on external environmental characteristics such as platform traffic, marketing activities, and product prices on that day. The improvement in the actual conversion rate relative to the expected conversion rate can be considered as a component of the net feedback gain.

[0072] Net feedback gain can be the gain value of a single metric or a weighted average of multiple business metrics. For content generation tasks, the platform can construct a comprehensive net feedback gain based on conversion rate gain, content adoption rate gain, and risk pass rate gain. Different metric weights can be set for different business metrics to reflect the current tenant's business objectives and risk requirements.

[0073] The initial reward value is determined based on the gain of the corresponding business feedback result output by the revision strategy relative to the business feedback baseline value. The initial reward value is then adjusted for confidence based on the feedback delay duration, the number of historical task samples, and the risk interception results to obtain the tenant isolation reward value. Revision actions with confidence values ​​less than a preset threshold are stored in the experience set to be verified, and isolation verification processing is performed based on the experience set to be verified.

[0074] Specifically, the platform can map the net feedback gain to the initial reward value. The higher the net feedback gain, the higher the initial reward value. The higher the value, the better. Feedback latency describes the time interval between task completion and obtaining the corresponding business feedback result. The number of historical task samples describes the number of available historical task samples when determining the business feedback baseline value. Risk interception results describe whether the revised policy output triggers risk review or business interception processing.

[0075] In one specific embodiment, the platform adjusts the initial reward value based on the feedback delay duration and the number of historical task samples to obtain the confidence level. :

[0076] in, This represents the delay correction factor. The value range is greater than zero and less than or equal to one. The longer the feedback delay, the better. The smaller. Represents the sample correction coefficient. The value of is greater than zero and less than or equal to one. The larger the number of historical task samples, the better. The larger.

[0077] The platform determines risk penalties based on the risk interception results. Risk penalty items The number of triggered risk rules, the risk level, and the processing result can be determined. The higher the risk level, the greater the risk penalty. The larger.

[0078] Tenant Isolation Bonus Value It can be determined using the following formula:

[0079] in, Indicates the first The initial reward value corresponding to each revision action. Indicates the first The confidence level corresponding to each revision action Indicates the first Each revision action corresponds to a risk penalty item. This calculation method allows revision actions with long feedback delays or few historical samples to participate in the reward calculation with lower confidence, while ensuring that risk interception results are directly reflected in the tenant isolation reward value through the risk penalty item.

[0080] The set of experiences to be verified is used to store revision actions with low confidence but with further verification value. Each experience to be verified may include a revision action, task constraint fingerprint, knowledge retrieval context snapshot, initial reward value, confidence level, associated risk rules, and associated business feedback records.

[0081] The platform can deploy the set of experiences to be verified to an isolated verification environment for verification processing. The isolated verification environment can be a sandbox environment, a shadow namespace, a test tenant environment, or a policy evaluation environment isolated from the production environment.

[0082] In an isolated verification environment, the platform can apply the revision strategy path corresponding to the experience to be verified to simulation tasks, historical replay tasks, or real-time shadow tasks, and record the revision strategy output, risk review results, and business feedback prediction results. After a preset number of verification tasks, the platform can update the confidence level and tenant isolation reward value corresponding to the experience to be verified based on the verification results.

[0083] Step S4: Based on the consistency of the reward direction of the tenant isolation reward value among multiple tenants and the matching relationship between the task constraint fingerprints, determine the migration eligibility corresponding to the revision action, and based on the migration eligibility, use the revision action with shared update eligibility to update the shared policy model, and use the revision action with tenant private update eligibility to update the private policy adapter of the corresponding tenant.

[0084] Shared policy models are used to learn knowledge retrieval and output generation strategies that are universally applicable across multiple tenants. A shared policy model can be a neural network-based policy model, a policy layer in a language model, a candidate output ranking model, a knowledge retrieval ranking model, or a policy model composed of multiple models.

[0085] The shared strategy model can output knowledge call weights, context reading levels, generation parameters, prompt word constraint weights, or candidate output ranking results based on task constraint fingerprints, knowledge call context snapshots, and task inputs.

[0086] Tenant-specific policy adapters are used to express local policy information related to a specific tenant. Different tenants can save their own private policy adapters, allowing rule preferences, expression habits, risk requirements, and business requirements accumulated within the same tenant to participate in policy determination within that tenant's tasks.

[0087] In step S4, the migration eligibility corresponding to the revision action is determined, including obtaining the predefined risk rule set corresponding to each tenant; determining the number and risk level of risk rules triggered in each predefined risk rule set based on the knowledge object and output fragment associated with the revision action, and calculating the degree of conflict between the revision action and the risk rules of each tenant; and determining the migration eligibility corresponding to the revision action based on the consistency of reward direction, matching relationship and degree of conflict.

[0088] Each tenant's predefined risk rule set can be pre-configured by business managers, auditors, or system administrators. Risk rules can include content compliance rules, brand expression rules, privacy protection rules, sensitive word rules, business process rules, product claim rules, and platform publishing rules, among others.

[0089] Each risk rule can have a rule identifier, rule conditions, risk level, applicable tenants, applicable platforms, and processing requirements. The platform can match the entities, attributes, semantic relationships, keywords, and related knowledge objects in the output fragment with the rule conditions in the risk rule to determine whether the risk rule has been triggered.

[0090] For example, when a revision action changes a product description to emphasize effects, the platform can match the revised output snippet with each tenant's product claim rules to determine if the expression triggers the corresponding rule. When a revision action adds specific brand marketing language, the platform can determine whether the language conflicts with other tenants' brand guidelines based on related knowledge objects and output snippets.

[0091] The conflict level can be determined based on the number of triggered risk rules and the risk level. In one implementation, the platform sets different risk weights for different risk levels and sums the risk weights corresponding to the triggered risk rules to obtain the conflict level. In another implementation, the platform inputs the number of risk rules, the highest risk level, and the scope of application of the rules into the risk assessment model, and the risk assessment model outputs the conflict level.

[0092] Reward direction consistency describes whether the tenant isolation reward values ​​obtained by the same revision action in different tenants have similar directions of change. For example, if a revision action increases the tenant isolation reward value in multiple tenants, the revision action has high reward direction consistency. If a revision action increases the reward value in some tenants and decreases the reward value in others, the revision action has low reward direction consistency.

[0093] The matching relationship between task constraint fingerprints is used to describe whether tasks corresponding to different tenants have similar task conditions, knowledge access conditions, and business requirements. The platform can calculate the matching degree based on factors other than the tenant, such as the target platform, business stage, task type, product category, knowledge version, risk level, and candidate knowledge object set.

[0094] When the tenant isolation reward value of the revision action is greater than the preset shared reward threshold in multiple tenants, the matching degree between the corresponding task constraint fingerprints is greater than the preset matching threshold, and the conflict degree between the revision action and the risk rules of each tenant is less than the preset conflict threshold, the revision action is determined to be eligible for shared update.

[0095] Revision actions eligible for shared updates can be converted into training samples for the shared policy model. Shared training samples may include revision actions, task constraint fingerprints, knowledge invocation context snapshots, revision policy paths, tenant isolation reward values, and risk rule information.

[0096] The shared policy model can be updated using supervised learning, preference learning, reinforcement learning, policy gradient optimization, or other policy learning methods. The updated shared policy model can learn knowledge retrieval and output generation methods that are highly adaptable across multiple tenants.

[0097] When the tenant isolation reward value of the revision action in the current tenant is greater than the preset private reward threshold, and the matching degree between the revision action and the task constraint fingerprint corresponding to the current tenant is greater than the preset private matching threshold, and the conflict degree between the revision action and other tenant risk rules is greater than the preset private conflict threshold, the revision action is determined to have tenant private update qualification.

[0098] Revision actions that qualify for tenant-private updates can be converted into private training samples for the current tenant. The platform adjusts the current tenant's private policy adapter based on the private training samples, enabling the business rules, expression preferences, or knowledge retrieval preferences corresponding to the revision action to participate in policy determination within the current tenant's tasks.

[0099] Step S5: Obtain the task to be processed, generate the current task constraint fingerprint and the current knowledge call context snapshot; determine the knowledge call strategy and output generation strategy based on the shared policy model and the corresponding private policy adapter, and generate the agent output results.

[0100] Tasks to be processed can be generated by user input, business system triggers, task queue scheduling, or preset workflows. The platform parses the tasks to be processed, extracts data such as tenant identifier, task type, target platform, business stage, product information, target user group, risk level, and resource budget, and generates the current task constraint fingerprint.

[0101] The platform performs a constrained search in the knowledge base based on the current task constraint fingerprint to obtain a snapshot of the current knowledge call context. The constrained search can first determine the allowed search scope based on tenant identifier, permission scope, business stage, knowledge version, and risk rules, and then select candidate knowledge objects from the search scope.

[0102] The platform can first read the summary content of candidate knowledge objects, and then determine whether to read the overview content or detailed content based on task relevance and resource budget. The current knowledge retrieval context snapshot records the retrieval path, candidate knowledge object set, knowledge object version, actual retrieval content, and context assembly result for this task.

[0103] The shared strategy model determines a general knowledge retrieval strategy and a general output generation strategy based on the current task constraint fingerprint, the current knowledge retrieval context snapshot, and the task input. The knowledge retrieval strategy may include candidate knowledge object ranking weights, knowledge object retrieval levels, context length allocation, and risk rule priorities. The output generation strategy may include generation template selection, expression style weights, generation parameters, the number of candidate outputs, and candidate output ranking rules.

[0104] In step S5, the private policy adapter outputs a residual policy vector based on the current task constraint fingerprint and the intermediate features of the shared policy model, and then fuses the residual policy vector with the basic policy vector output by the shared policy model to obtain the output generation policy.

[0105] Specifically, when the shared policy model processes the current task, it can output a basic policy vector. The basic policy vector is used to represent the general policy tendency under the current task, including the priority of knowledge object invocation, the length of generated content, the formality of expression, the weight of risk rules, and the preference for ranking candidate outputs.

[0106] The private policy adapter receives intermediate features from the current task constraint fingerprint and the shared policy model, and outputs the residual policy vector. The residual policy vector is used to represent the adjustments made by the current tenant relative to the general policy.

[0107] The platform can determine the fusion weights based on the matching degree between the current task constraint fingerprint and the private training samples. fusion weight The value ranges from zero to one. The higher the matching degree between the current task constraint fingerprint and the private training samples, the higher the fusion weight. The larger.

[0108] Output generated policy vector It can be determined using the following formula:

[0109] in, This represents the base policy vector output by the shared policy model. This represents the residual policy vector output by the private policy adapter. This indicates the fusion weight.

[0110] When the fusion weight is zero, the output generation strategy is determined by the basic strategy vector. When the fusion weight is greater than zero, the residual strategy vector participates in adjusting the basic strategy vector so that the output generation strategy can reflect the local business requirements of the current tenant.

[0111] The output generation strategy controls the agent's generation of the final output. The platform generates one or more candidate outputs based on the output generation strategy, the current knowledge retrieval context snapshot, and the task to be processed, and selects the agent's output based on risk rules, task matching degree, knowledge reference consistency, and strategy score.

[0112] In one specific embodiment, a tenant needs to generate marketing content for a specific product on a target platform. After acquiring the task to be processed, the platform generates a current task constraint fingerprint and retrieves the corresponding knowledge object based on the tenant identifier, product information, target platform, business stage, and risk rules. The shared strategy model determines the general content structure, knowledge call order, and expression strength based on the historical training results of multiple tenants. The current tenant's private strategy adapter adjusts the brand terminology, expression methods, and risk expressions based on the tenant's historical revision experience. The platform merges the basic strategy vector and the residual strategy vector to generate an output generation strategy, and generates an intelligent agent output result that meets the current tenant's business requirements accordingly.

[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A multi-tenant agent policy learning method, characterized in that, Includes the following steps: S1: Obtain historical task execution records including task constraint data, knowledge call records, original output, manual revision results, and business feedback results; based on the task constraint data and knowledge call records, generate task constraint fingerprints and knowledge call context snapshots respectively, and determine the revision actions corresponding to the manual revision results and their attribution relationships; S2: Construct an attribution control constraint set based on the task constraint fingerprint and knowledge call context snapshot. Under the condition of satisfying the attribution control constraint set, generate the original policy output based on the basic policy path and the revised policy output based on the revised policy path containing the revised action, thus forming a counterfactual comparison sample. S3: Determine the business feedback baseline value based on the historical task execution records matched with the counterfactual control sample, and perform attribution separation processing on the business feedback results based on the business feedback baseline value, external environment characteristics and task constraint fingerprints to obtain the net feedback gain and tenant isolation reward value corresponding to the revision action; S4: Based on the consistency of the reward direction of the tenant isolation reward value among multiple tenants and the matching relationship between the task constraint fingerprints, determine the migration eligibility corresponding to the revision action, and based on the migration eligibility, use the revision action with shared update eligibility to update the shared policy model, and use the revision action with tenant private update eligibility to update the private policy adapter of the corresponding tenant. S5: Obtain the tasks to be processed, and generate the current task constraint fingerprint and the current knowledge call context snapshot; Based on the shared policy model and the corresponding private policy adapter, the knowledge retrieval policy and output generation policy are determined, and the intelligent agent output results are generated. 2.The multi-tenant agent policy learning method of claim 1, wherein, In step S2, the attribution control constraint set includes the tenant identifier, target platform, business stage, knowledge version, and risk rule in the task constraint fingerprint, as well as the retrieval path and candidate knowledge object set in the knowledge invocation context snapshot; The original policy output and the revised policy output are generated based on the same task input, knowledge invocation context snapshot, and candidate knowledge object set.

3. The multi-tenant agent policy learning method of claim 2, wherein, In step S3, historical task execution records are filtered according to the field correspondence between the task constraint fingerprint and the knowledge call context snapshot. Historical task execution records that match the constraint fields in the task constraint fingerprint with the attribution control constraint set and match the retrieval path in the knowledge call context snapshot with the candidate knowledge object set are identified as historical task samples.

4. The multi-tenant agent policy learning method of claim 1, wherein, In step S4, determining the migration eligibility corresponding to the revision action includes: Obtain the predefined risk rule set corresponding to each tenant; Based on the knowledge objects and output fragments associated with the revision action, determine the number and risk level of risk rules triggered in each predefined risk rule set, and calculate the degree of conflict between the revision action and each tenant's risk rule accordingly. Based on the consistency of the reward direction, the matching relationship, and the degree of conflict, the migration eligibility corresponding to the revision action is determined.

5. The multi-tenant agent policy learning method of claim 4, wherein, When the tenant isolation reward value of the revision action in multiple tenants is greater than the preset shared reward threshold, the matching degree between the corresponding task constraint fingerprints is greater than the preset matching threshold, and the conflict degree between the revision action and the risk rules of each tenant is less than the preset conflict threshold, the revision action is determined to be eligible for shared update. When the tenant isolation reward value of the revision action in the current tenant is greater than the preset private reward threshold, and the matching degree between the revision action and the task constraint fingerprint corresponding to the current tenant is greater than the preset private matching threshold, and the conflict degree between the revision action and other tenant risk rules is greater than the preset private conflict threshold, the revision action is determined to have tenant private update qualification.

6. The multi-tenant agent policy learning method of claim 3, wherein, In step S3, based on the task constraint fingerprint of the counterfactual comparison sample, historical task execution records that meet the preset similarity threshold are selected from the historical task samples. The business feedback results of each historical task execution record are weighted according to the task constraint similarity, knowledge version matching degree, time decay coefficient and feedback credibility to obtain the business feedback baseline value.

7. The multi-tenant agent policy learning method of claim 6, wherein, The initial reward value is determined based on the gain of the corresponding business feedback result output by the revised strategy relative to the business feedback baseline value. The confidence of the initial reward value is adjusted based on the feedback delay duration and the number of historical task samples. The risk penalty item is determined based on the risk interception result to obtain the tenant isolation reward value. Revision actions with a confidence level less than a preset threshold are stored in a set of experiences to be verified, and isolation verification is performed based on the set of experiences to be verified.

8. The multi-tenant agent policy learning method of claim 1, wherein, In step S1, the original output and the manually revised result are aligned to identify the corresponding difference positions and semantic change types. The knowledge objects associated with each difference position are determined according to the knowledge call records. Based on the difference position, semantic change type and associated knowledge object, the revision action is determined, and the attribution relationship between the revision action and the output segment, associated knowledge object and associated risk rule is established.

9. The multi-tenant agent policy learning method according to claim 1, characterized in that, In step S5, the private policy adapter outputs a residual policy vector based on the current task constraint fingerprint and the intermediate features of the shared policy model, and fuses the residual policy vector with the basic policy vector output by the shared policy model to obtain the output generation policy.

Images