Online advertisement putting optimization method based on knowledge graph
By constructing a delivery subgraph on the knowledge graph and introducing feasible domain reconstruction and EXP3-IX decision-making, the problem of insufficient stability and exploration capability of online advertising delivery in existing technologies is solved, and compliant continuous optimization and robust delivery strategy updates are achieved under complex constraints.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU ZHANGNUO NETWORK TECHNOLOGY CO LTD
- Filing Date
- 2026-02-06
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies have shortcomings in terms of complex deployment constraints and online learning stability. Rule engines or graph-based recall and filtering often use the method of recall first and then filtering to deal with hard constraints, which weakens the exploration ability when the action space changes suddenly. The strategy is prone to getting stuck in local optima or slow recovery in new scenarios. In addition, traditional filtering is difficult to restructure the feasible domain and lacks alternative mapping mechanisms under the premise of compliance.
The deployment graph structure is constructed based on knowledge graph, generating candidate arms and rule constraint information. Feasible candidate arms are obtained through feasible domain reconstruction. The selection probability is generated using EXP3-IX and deployment is collected and feedback is performed. The cumulative loss value is updated based on the interpretation path gating, achieving compliant and robust online continuous optimization.
Maintaining decision space stability in scenarios with frequent rule changes or complex constraints helps avoid exploration breakdowns and policy degradation, improves exploration efficiency and learning stability under conditions of sparse data, cold start, or distribution drift, and enhances the robustness of online optimization.
Smart Images
Figure CN122048446A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of knowledge graph advertising optimization technology, and in particular to a knowledge graph-based method for optimizing online advertising delivery. Background Technology
[0002] As internet advertising transactions evolve from manual placement to automation and real-time processing, advertisers and media outlets generally adopt request-based online placement methods. When each placement request arrives, candidate configuration recall, bidding, and creative selection are completed in real time, and placement strategies are continuously adjusted based on feedback such as impressions, clicks, and conversions.
[0003] In existing technologies, one type of solution typically relies on rule engines and empirical thresholds to filter and rank advertising targets. Another type uses machine learning models or reinforcement learning methods to train static or quasi-static strategies on historical data, and then executes them with fixed parameters in an online environment. Some solutions also use knowledge graphs to organize advertising entities and relationships to improve recall coverage, enhance feature representation, or provide some interpretability. Meanwhile, to address the distribution drift caused by traffic non-stationarity and bidding competition, the industry often introduces online learning frameworks such as multi-armed gambling machines to explore and utilize different advertising actions in order to achieve better advertising returns or lower advertising losses in a constantly changing environment.
[0004] However, the aforementioned existing technologies still have shortcomings in terms of complex deployment constraints and online learning stability. Rule engines or graph-based recall and filtering often use a recall-then-filter approach to handle hard constraints. Candidates that trigger hard constraints are usually directly eliminated, which weakens the exploration ability when the action space changes abruptly. This can easily lead to the strategy getting stuck in local optima or slow recovery in new scenarios. Furthermore, when there are multi-level deployment objects and multiple constraint relationships, traditional filtering is difficult to restructure the "feasible domain" and lacks a mechanism to replace the eliminated actions while maintaining compliance.
[0005] Therefore, how to provide a knowledge graph-based online advertising optimization method is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] One objective of this invention is to propose an online advertising delivery optimization method based on knowledge graphs. This invention constructs a delivery graph structure based on knowledge graphs and retrieves delivery subgraphs, generates candidate arms and rule constraint information, obtains feasible candidate arms through feasible domain reconstruction, determines arm-level implicit exploration parameters according to structural uncertainty, uses EXP3-IX to generate selection probabilities and performs delivery collection feedback, updates cumulative loss and recalculates probabilities based on interpretation path gating, and achieves compliant and robust online continuous optimization.
[0007] An online advertising optimization method based on knowledge graphs according to an embodiment of the present invention includes the following steps:
[0008] Acquire online advertising data, construct an online advertising knowledge graph and generate a corresponding graph structure, and establish a set of advertising optimization parameters;
[0009] Receive an ad delivery request, retrieve candidate delivery configurations in the graph structure, and obtain the delivery subgraph;
[0010] Generate a candidate arm set based on the deployment subgraph and determine the rule constraint information;
[0011] Based on the rule constraint information, the feasible region reconstruction process is performed on the candidate arm set to obtain the feasible candidate arm set;
[0012] For each candidate arm in the feasible candidate arm set, calculate the structural uncertainty index and determine the arm-level implicit exploration parameters of the candidate arm, and write them into the deployment optimization parameter set;
[0013] The selection probability distribution of the feasible candidate arm set is generated according to the EXP3-IX decision rule. The target candidate arm is sampled according to the selection probability distribution and the corresponding deployment action is executed. The deployment feedback data of the current deployment action is collected.
[0014] Retrieve the set of explanatory paths corresponding to the delivery feedback data, calculate the gating coefficient, and gating the delivery feedback data to obtain the gating loss value. According to the EXP3-IX implicit exploration update rule, update the cumulative loss value of the target candidate arm using the gating loss value, regenerate the selection probability distribution, and write back the delivery optimization parameter set.
[0015] Optionally, obtaining the graph structure and the deployment optimization parameter set specifically includes:
[0016] Acquire online advertising data and perform structured processing, uniformly encode the target object identifier, relationship type and relationship attribute, map the target object to the node of the graph, and map the association relationship between the target objects and the constraint relationship of the advertising rule to the edge of the graph;
[0017] Perform relation labeling and weight configuration operations on the graph, set relation category identifiers for different types of edges to distinguish between association and constraint relationships, configure weight information for each edge, store the graph with completed relation labeling and weight configuration as an online advertising knowledge graph, and obtain the corresponding graph structure;
[0018] Establish and initialize a set of deployment optimization parameters, and create corresponding parameter records. The parameter records include cumulative loss values, arm-level implicit exploration parameters, and corresponding selection probability values.
[0019] Optionally, obtaining the delivery sub-graph specifically includes:
[0020] Receive and parse advertising delivery requests to obtain delivery request context information, and convert the delivery request context information into an internal code consistent with the node identifier in the graph structure to generate a request feature set for graph structure retrieval.
[0021] Based on the request feature set, a candidate retrieval operation is performed in the graph structure. By locating the starting node corresponding to the request feature set and traversing along the association and constraint edges, a candidate deployment configuration set is obtained.
[0022] Using the node corresponding to the candidate delivery configuration set as the central node, a subgraph extraction operation is performed in the graph structure according to the preset neighborhood range to collect the central node, associated nodes and corresponding edges, forming a delivery subgraph containing delivery object information and relationship information.
[0023] Optionally, the generation of the candidate arm set and the determination of the rule constraint information specifically include:
[0024] The nodes in the delivery subgraph are classified, and the edges in the delivery subgraph are identified by type. Edges that represent the relationship between delivery objects are marked as relationship edges, and edges that represent the constraint relationship between delivery rules and delivery objects are marked as constraint edges, thus obtaining a delivery subgraph with complete node and edge type markings.
[0025] In the delivery subgraph after completing the node and edge type labeling, select the material nodes that are in an available state from the delivery object nodes to form a material candidate set, select the targeting nodes that match the delivery request context from the delivery object nodes to form a targeting candidate set, and select the bidding nodes that meet the platform bidding strategy constraints from the delivery object nodes to form a discretized bidding candidate set.
[0026] Based on the candidate set of materials, the candidate set of targeting, and the discretized bid candidate set, different delivery action configurations are generated according to preset combination rules. Each delivery action configuration is defined as a candidate arm, a unique candidate arm identifier is assigned to each candidate arm, and a one-to-one correspondence between the candidate arm identifier and the delivery action configuration is established.
[0027] For each candidate arm, locate the delivery object node associated with the corresponding delivery action configuration, retrieve the rule node connected to the delivery object node along the constraint relationship edge in the delivery subgraph, read the constraint conditions in the rule node and perform the judgment to obtain the corresponding constraint judgment result, associate the constraint judgment result with the candidate arm identifier to form rule constraint information, and output the candidate arm set and rule constraint information.
[0028] Optionally, obtaining the set of feasible candidate arms specifically includes:
[0029] Read the candidate arm set and rule constraint information, extract the associated rule entries and constraint judgment results for each candidate arm, form a correspondence table between candidate arms and rule entries, and mark the constraint judgment results as two states: pass and trigger hard constraint.
[0030] Perform a removal process on candidate arms marked as triggering hard constraint rules, remove the candidate arms from the candidate arm set and generate a removed candidate arm set, and record the corresponding trigger rule identifier and trigger type in the rule constraint information;
[0031] For each candidate arm in the candidate arm elimination set, perform an exploration retention judgment. Based on the preset exploration retention conditions, mark the candidate arms that meet the exploration retention conditions as candidate arms to be replaced, and extract the entry of the deployment action configuration and trigger hard constraint rule for each candidate arm to be replaced as the replacement retrieval condition.
[0032] For each candidate arm to be replaced, a replacement candidate arm is retrieved in the delivery subgraph. During the retrieval, candidate arms whose delivery action configuration satisfies the preset similarity relationship with the candidate arm to be replaced and whose hard constraint rules are met are selected as replacement candidate arms. A mapping relationship between the candidate arm to be replaced and the replacement candidate arm is established. The replacement candidate arm is added to the feasible candidate arm set and the corresponding candidate arm to be replaced is removed. Candidate arms that have not triggered hard constraints are merged to obtain the feasible candidate arm set.
[0033] Optionally, the determination of the structural uncertainty index and the arm-level implicit exploration parameters specifically includes:
[0034] Based on the set of feasible candidate arms, locate the delivery object node and relationship edge associated with the delivery action configuration for each feasible candidate arm, forming a candidate arm associated substructure;
[0035] Based on the candidate arm associated substructure, the connection status of the delivery object nodes associated with the candidate arm in the delivery subgraph is statistically analyzed. The number of node connections is normalized with the preset maximum connection scale to obtain a node sparsity index that reflects the sparsity of the candidate arm structure.
[0036] Based on the candidate arm association substructure, the weight information of the relationship edge is used as the relationship edge weight. The relationship edge weight is aggregated and normalized with the preset maximum weight to obtain the relationship confidence index that reflects the stability of the candidate arm association relationship.
[0037] Based on the rule constraint information corresponding to the candidate arm, the number of rules that are determined to be mutually exclusive or conflicting in the rule constraints associated with the candidate arm is counted, and normalized with a preset conflict upper limit to obtain a rule conflict index that reflects the degree of inconsistency of the candidate arm rules.
[0038] Based on node sparsity index, relationship confidence index, and rule conflict index, the three types of indexes are processed to unify direction, and uncertainty contribution is set. The three types of uncertainty contributions are weighted and merged according to the preset weight ratio rule to obtain structural uncertainty index. The corresponding arm-level implicit exploration parameters are determined based on the structural uncertainty index and written into the deployment optimization parameter set.
[0039] Optionally, the collection of delivery feedback data for the current delivery action specifically includes:
[0040] Read the cumulative loss value and arm-level implicit exploration parameters that correspond one-to-one with each feasible candidate arm identifier from the deployment optimization parameter set, and at the same time read the set of alternative candidate arm identifiers and mapping relationship;
[0041] Based on the exponential weight decision in the EXP3-IX decision rule, the cumulative loss value of the current feasible candidate arm is subjected to exponential weight transformation, and the basic weight of all feasible candidate arms is subjected to summation and normalization to obtain the basic selection probability distribution based on the cumulative loss value.
[0042] Simultaneously, an exploration weight is generated for each feasible candidate arm. The exploration weight is directly determined by the arm-level implicit exploration parameters of the current feasible candidate arm. The exploration weights of all feasible candidate arms are summed and normalized to obtain the structure-aware implicit exploration probability distribution.
[0043] The basic selection probability distribution and the implicit exploration probability distribution are weighted and fused according to a preset mixing coefficient to obtain the final selection probability distribution. Two types of improvement operations are performed to enhance synergy. The first type of improvement operation is to set a minimum selection probability lower limit for the alternative candidate arms obtained by mapping the alternative arms, and at the same time normalize and backfill the probabilities of the remaining candidate arms. The second type of improvement operation is to perform exploration suppression processing on feasible candidate arms in the boundary state in the rule constraint information. The exploration suppression processing is to reduce the final selection probability while keeping the probability sum to one, and to redistribute the difference to other feasible candidate arms. Then, the final selection probability distribution is written into the deployment optimization parameter set.
[0044] Sampling is performed on the set of feasible candidate arms based on the final selection probability distribution. During sampling, random numbers are generated, and the final selection probability is accumulated in the order of candidate arm identifiers until a random number is reached. The target candidate arm identifier is determined, the deployment action configuration corresponding to the target candidate arm identifier is read and the deployment action is executed, and the deployment feedback data corresponding to the deployment action is collected.
[0045] Optionally, the updating of the cumulative loss value of the target candidate arm and the regeneration of the selection probability distribution specifically include:
[0046] In the delivery subgraph, locate the delivery object node corresponding to the target candidate arm identifier and the feedback event node corresponding to the delivery feedback data. Determine the delivery object node and the feedback event node as the starting node and ending node of the path search. Perform path search along the association relationship edge and constraint relationship edge in the delivery subgraph, filter the node sequence that meets the preset path length range and contains a rule node, and obtain the set of explanatory paths corresponding to the delivery feedback data.
[0047] For each interpretation path in the interpretation path set, read the relation edge weights of each relation edge in the path, perform aggregation on the edge weights in the same path to obtain the path association strength, and generate path usability tags based on rule entries and constraint judgment results;
[0048] The gating coefficients are determined based on path association strength and path availability marking. The gating coefficients of the explanatory paths marked as disallowed are set to zero, while the gating coefficients of the explanatory paths marked as allowed are set to the coefficient values obtained by normalizing the path association strength. The upper and lower bounds of the gating coefficients are truncated, and the gating coefficients of each path in the explanatory path set are aggregated to obtain the final gating coefficients.
[0049] Read the original loss value corresponding to the target candidate arm, generate the gated loss value based on the final gate coefficient, read the selection probability and arm-level implicit exploration parameters corresponding to the target candidate arm in the deployment optimization parameter set, update the cumulative loss value of the target candidate arm according to the EXP3-IX implicit exploration update rule, use the gated loss value as the update increment, and use the sum of the selection probability of the target candidate arm and the arm-level implicit exploration parameters as the normalization denominator. After normalizing the update increment, add it to the cumulative loss value to obtain the updated cumulative loss value.
[0050] Based on the updated cumulative loss value of the target candidate arm, the selection probability distribution of the feasible candidate arm set is regenerated according to the EXP3-IX decision rule, and the updated cumulative loss value and the regenerated selection probability distribution are written back to the deployment optimization parameter set.
[0051] The beneficial effects of this invention are:
[0052] This invention forms a delivery subgraph on the knowledge graph and explicitly incorporates rule constraint information into the candidate arm generation and feasible domain reconstruction process. This enables the delivery action space to no longer rely solely on simple elimination when hard constraints are triggered. Instead, it can complete the structured reconstruction of the feasible candidate arm set within the compliance boundary. This maintains the stability of the decision space in scenarios with frequent rule changes or complex constraints, avoiding the exploration break and strategy degradation problems of the traditional "filtering equals loss".
[0053] This invention addresses the differences in candidate arms within a graph structure by introducing a structural uncertainty index and determining implicit exploration parameters at the arm level. This allows the exploration intensity to adaptively change with the sparsity of the graph structure, relation confidence, and rule conflict, thereby improving exploration efficiency and learning stability under conditions of data sparsity, cold start, or distribution drift, and reducing blind trial-and-error and slow convergence caused by a uniform exploration strategy.
[0054] This invention introduces the calculation of gating coefficients from the interpretation path set in the EXP3-IX online update. It then performs gating on the delivery feedback to generate a gating loss value, followed by cumulative loss updates and probability recalculation. This allows the update process to utilize the interpretable relationships and rule states of the knowledge graph to suppress the interference of noise feedback and abnormal attribution on the model, thereby enhancing the robustness of online optimization in non-stationary traffic and adversarial competitive environments, and achieving continuous iterative delivery optimization under the premise of compliance. Attached Figure Description
[0055] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0056] Figure 1 This is a flowchart of an online advertising delivery optimization method based on knowledge graphs proposed in this invention;
[0057] Figure 2 This is a schematic diagram of EXP3-IX decision-making and gating update for an online advertising delivery optimization method based on knowledge graphs proposed in this invention. Detailed Implementation
[0058] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0059] refer to Figure 1-2 A knowledge graph-based method for optimizing online advertising delivery includes the following steps:
[0060] Acquire online advertising data, construct an online advertising knowledge graph and generate a corresponding graph structure, and establish a set of advertising optimization parameters. The set of advertising optimization parameters is used to store the cumulative loss value, arm-level implicit exploration parameters and selection probability distribution corresponding to the candidate arms.
[0061] Receive an ad delivery request, retrieve candidate delivery configurations in the graph structure, and obtain the delivery subgraph;
[0062] A candidate arm set is generated based on the deployment subgraph and the rule constraint information is determined. Each candidate arm corresponds to a deployment action.
[0063] Based on the rule constraint information, the feasible region reconstruction process is performed on the candidate arm set. Candidate arms that trigger hard constraints are eliminated, and candidate arms that trigger hard constraints and need to retain exploration capabilities are mapped as alternative candidate arms to obtain the feasible candidate arm set.
[0064] For each candidate arm in the feasible candidate arm set, calculate the structural uncertainty index and determine the arm-level implicit exploration parameters of the candidate arm, and write them into the deployment optimization parameter set;
[0065] The selection probability distribution of the feasible candidate arm set is generated according to the EXP3-IX decision rule. The selection probability distribution is obtained by transforming and normalizing the cumulative loss value of each candidate arm through exponential weight. The target candidate arm is sampled according to the selection probability distribution and the corresponding deployment action is executed. The deployment feedback data of the current deployment action is collected.
[0066] Retrieve the set of explanatory paths corresponding to the delivery feedback data, calculate the gating coefficient, and gating the delivery feedback data to obtain the gating loss value. According to the EXP3-IX implicit exploration update rule, update the cumulative loss value of the target candidate arm using the gating loss value, regenerate the selection probability distribution, and write back the delivery optimization parameter set.
[0067] In this embodiment, obtaining the graph structure and the deployment optimization parameter set specifically includes:
[0068] Acquire online advertising data and perform structured processing, uniformly encode the target object identifier, relationship type and relationship attribute, map the target object to the node of the graph, and map the association relationship between the target objects and the constraint relationship of the advertising rule to the edge of the graph;
[0069] Perform relationship labeling and weight configuration operations on the graph, set relationship category identifiers for different types of edges to distinguish between association and constraint relationships, configure weight information for each edge to represent relationship strength or confidence, and configure node attributes to reflect the delivery status and availability. Store the graph with completed relationship labeling and weight configuration as an online advertising delivery knowledge graph and obtain the corresponding graph structure.
[0070] Establish and initialize a set of deployment optimization parameters, and create corresponding parameter records. The parameter records include cumulative loss values, arm-level implicit exploration parameters, and selection probability values. Initialize the cumulative loss value to zero and the arm-level implicit exploration parameters to preset non-zero values.
[0071] In this embodiment, obtaining the deployment sub-map specifically includes:
[0072] The system receives and parses advertising delivery requests to obtain delivery request context information, which includes user identifier, media identifier, ad slot identifier, delivery scenario identifier, and request time information. The delivery request context information is then converted into an internal code that is consistent with the node identifier in the graph structure to generate a request feature set for graph structure retrieval.
[0073] Based on the request feature set, a candidate retrieval operation is performed in the graph structure. By locating the starting node corresponding to the request feature set and traversing along the association and constraint edges, the candidate delivery configuration set is obtained by filtering delivery objects that meet the requirements of delivery status availability, scenario matching and not being prohibited by hard constraint rules.
[0074] Using the node corresponding to the candidate placement configuration set as the central node, a subgraph extraction operation is performed in the graph structure according to the preset neighborhood range to collect the central node, associated nodes and corresponding edges, forming a placement subgraph containing placement object information and relationship information. The hard constraint rule refers to the rule constraint that, once triggered during the advertising placement process, prohibits the execution of the corresponding placement action and does not allow relaxation or replacement by adjusting parameters or weights.
[0075] In this embodiment, the generation of the candidate arm set and the determination of the rule constraint information specifically include:
[0076] The nodes in the delivery subgraph are classified and processed. Nodes carrying delivery object information are marked as delivery object nodes, and nodes carrying delivery rule information are marked as rule nodes. At the same time, the edges in the delivery subgraph are identified by type. Edges that represent the relationship between delivery objects are marked as relationship edges, and edges that represent the constraint relationship between delivery rules and delivery objects are marked as constraint edges. The delivery subgraph with complete node and edge type marking is obtained.
[0077] In the delivery subgraph after completing the node and edge type labeling, material nodes in an available state are selected from the delivery object nodes to form a material candidate set, targeted nodes that match the delivery request context are selected from the delivery object nodes to form a targeted candidate set, and bidding nodes that meet the platform bidding strategy constraints are selected from the delivery object nodes to form a discretized bidding candidate set. Together, they serve as the basic elements for constructing the delivery action.
[0078] Based on the candidate set of materials, the candidate set of targeting, and the candidate set of discretized bidding, different delivery action configurations are generated according to preset combination rules. Each delivery action configuration is defined as a candidate arm, a unique candidate arm identifier is assigned to each candidate arm, and a one-to-one correspondence between the candidate arm identifier and the delivery action configuration is established, so that the candidate arm identifier serves as the index identifier of the delivery optimization parameter set.
[0079] For each candidate arm, locate the delivery object node associated with the corresponding delivery action configuration, retrieve the rule node connected to the delivery object node along the constraint relationship edge in the delivery subgraph, read the constraint conditions in the rule node and perform the judgment to obtain the corresponding constraint judgment result, associate the constraint judgment result with the candidate arm identifier to form rule constraint information, and output the candidate arm set and rule constraint information.
[0080] In this embodiment, obtaining the set of feasible candidate arms specifically includes:
[0081] Read the candidate arm set and rule constraint information, extract the associated rule entries and constraint judgment results for each candidate arm, form a correspondence table between candidate arms and rule entries, and mark the constraint judgment results as two states: pass and trigger hard constraint.
[0082] Candidate arms marked as triggering hard constraint rules are removed by removing them from the candidate arm set and generating a set of removed candidate arms. At the same time, the corresponding triggering rule identifier and triggering type are recorded in the rule constraint information to maintain a traceable association between the set of removed candidate arms and the entries that trigger hard constraint rules.
[0083] For each candidate arm in the candidate arm elimination set, perform an exploration retention judgment. Based on the preset exploration retention conditions, mark the candidate arms that meet the exploration retention conditions as candidate arms to be replaced, and extract the entry of the deployment action configuration and trigger hard constraint rule for each candidate arm to be replaced as the replacement retrieval condition.
[0084] For each candidate arm to be replaced, a replacement candidate arm is retrieved in the delivery subgraph. During the retrieval, candidate arms whose delivery action configuration satisfies the preset similarity relationship with the candidate arm to be replaced and whose hard constraint rules are met are selected as replacement candidate arms. A mapping relationship between the candidate arm to be replaced and the replacement candidate arm is established. The replacement candidate arm is added to the feasible candidate arm set and the corresponding candidate arm to be replaced is removed. Candidate arms that have not triggered hard constraints are merged to obtain the feasible candidate arm set.
[0085] This invention determines the correspondence between candidate arms and rule entries, eliminates candidate arms that trigger hard constraints, and performs alternative arm retrieval and mapping for non-compliant candidate arms that need to retain exploration capabilities, replacing them with alternative candidate arms that meet similar deployment action configurations and pass status. This allows for the structured reconstruction of the feasible domain under compliance constraints, maintaining the continuous explorability of the action space, reducing the risk of decision space collapse and strategy degradation caused by hard constraints, and improving the stability and recovery speed of online optimization.
[0086] In this embodiment, the determination of the structural uncertainty index and the arm-level implicit exploration parameters specifically includes:
[0087] Based on the set of feasible candidate arms, locate the delivery object node and relationship edge associated with the delivery action configuration for each feasible candidate arm, forming a candidate arm associated substructure;
[0088] Based on the candidate arm associated substructure, the connection status of the delivery object nodes associated with the candidate arm in the delivery subgraph is statistically analyzed. The number of node connections is normalized with the preset maximum connection scale to obtain a node sparsity index that reflects the sparsity of the candidate arm structure.
[0089] Based on the candidate arm association substructure, the weight information of the relationship edge is used as the relationship edge weight. The relationship edge weight is aggregated and normalized with the preset maximum weight to obtain the relationship confidence index that reflects the stability of the candidate arm association relationship.
[0090] Based on the rule constraint information corresponding to the candidate arm, the number of rules that are determined to be mutually exclusive or conflicting in the rule constraints associated with the candidate arm is counted, and normalized with a preset conflict upper limit to obtain a rule conflict index that reflects the degree of inconsistency of the candidate arm rules.
[0091] Based on node sparsity, relationship confidence, and rule conflict indicators, the three types of indicators are processed to unify their directions. An uncertainty contribution is set, and the three types of uncertainty contributions are weighted and merged according to a preset weight ratio rule to obtain a structural uncertainty indicator. The preset weight ratio rule is that the higher the node sparsity, the lower the relationship confidence, and the higher the degree of rule conflict, the greater the uncertainty contribution. The corresponding arm-level implicit exploration parameters are determined based on the structural uncertainty indicator and written into the deployment optimization parameter set.
[0092] This invention utilizes the sparsity of node connections in the candidate arm's delivery subgraph, the stability of relationships represented by the edge weights, and the degree of rule conflict to construct a structural uncertainty index. Based on this index, arm-level implicit exploration parameters corresponding one-to-one with the candidate arms are generated and written into the delivery optimization parameter set. This allows the exploration intensity to be adaptively adjusted according to the consistency of graph structure information and rules, improving exploration efficiency and learning stability under conditions of data sparsity and scene changes, and reducing the risks of blind trial and slow convergence brought about by a unified exploration strategy.
[0093] In this embodiment, the collection of delivery feedback data for the current delivery action specifically includes:
[0094] Read the cumulative loss value and arm-level implicit exploration parameters corresponding one-to-one with each feasible candidate arm identifier from the deployment optimization parameter set, and at the same time read the set of alternative candidate arm identifiers and mapping relationship, so that each feasible candidate arm has the cumulative loss value input and arm-level implicit exploration parameter input for generating selection probability;
[0095] Based on the exponential weight decision in the EXP3-IX decision rule, an exponential weight transformation is performed on the cumulative loss value of the current feasible candidate arm, so that the basic weight of the feasible candidate arm with the smaller cumulative loss value is larger. Then, the basic weight of all feasible candidate arms is summed and normalized to obtain the basic selection probability distribution based on the cumulative loss value.
[0096] Simultaneously, an exploration weight is generated for each feasible candidate arm. The exploration weight is directly determined by the arm-level implicit exploration parameter of the current feasible candidate arm, so that the exploration weight of the feasible candidate arm with the larger arm-level implicit exploration parameter is larger. The exploration weights of all feasible candidate arms are summed and normalized to obtain the structure-aware implicit exploration probability distribution.
[0097] The basic selection probability distribution and the implicit exploration probability distribution are weighted and fused according to a preset mixing coefficient to obtain the final selection probability distribution. Two types of improvement operations are performed to enhance synergy. The first type of improvement operation is to set a minimum selection probability lower limit for the alternative candidate arms obtained by mapping the alternative arms, and at the same time normalize and backfill the probabilities of the remaining candidate arms so that the alternative candidate arms remain explorable after the feasible domain is reconstructed. The second type of improvement operation is to perform exploration suppression processing on the feasible candidate arms in the boundary state in the rule constraint information. The exploration suppression processing is to reduce the final selection probability while keeping the probability sum to one, and to redistribute the difference to other feasible candidate arms. Then, the final selection probability distribution is written into the deployment optimization parameter set.
[0098] Sampling is performed on the set of feasible candidate arms based on the final selection probability distribution. During sampling, random numbers are generated, and the final selection probability is accumulated in the order of candidate arm identifiers until a random number is reached. The target candidate arm identifier is determined, the deployment action configuration corresponding to the target candidate arm identifier is read and the deployment action is executed, and the deployment feedback data corresponding to the deployment action is collected.
[0099] This invention performs exponential weighted decision-making based on cumulative loss values and generates exploration weights based on arm-level implicit exploration parameters, thereby jointly forming a selection probability distribution. It sets a minimum selection probability on alternative candidate arms after the feasible region is reconstructed to retain exploration capabilities. At the same time, it implements exploration suppression on feasible candidate arms in the rule boundary state to reduce violations and unstable attempts. This enables the sampled deployment actions to achieve a more robust balance of exploration and utilization under compliance constraints, and improves the effectiveness of online deployment feedback collection and the efficiency of strategy iteration.
[0100] In this embodiment, the updating of the cumulative loss value of the target candidate arm and the regeneration of the selection probability distribution specifically include:
[0101] In the delivery subgraph, locate the delivery object node corresponding to the target candidate arm identifier and the feedback event node corresponding to the delivery feedback data. Determine the delivery object node and the feedback event node as the starting node and ending node of the path search. Perform path search along the association relationship edge and constraint relationship edge in the delivery subgraph, filter the node sequence that meets the preset path length range and contains a rule node, and obtain the set of explanatory paths corresponding to the delivery feedback data.
[0102] For each interpretation path in the interpretation path set, the relation edge weights of each relation edge in the path are read, the edge weights in the same path are aggregated to obtain the path association strength, and a path availability flag is generated based on the rule entries and constraint judgment results. The path availability flag is used to characterize whether the current interpretation path is allowed to be used for gating computation.
[0103] The gating coefficient is determined based on the path association strength and the path availability flag. The gating coefficient of the explanatory path whose path availability flag is not allowed is set to zero, while the gating coefficient of the explanatory path whose path availability flag is allowed is set to the coefficient value obtained by normalizing the path association strength, so that the higher the path association strength, the larger the corresponding gating coefficient. The upper and lower bounds of the gating coefficient are truncated so that it falls within the range of zero to one. The gating coefficients of each path in the explanatory path set are aggregated to obtain the final gating coefficient.
[0104] Read the original loss value corresponding to the target candidate arm, generate the gated loss value based on the final gate coefficient, read the selection probability and arm-level implicit exploration parameters corresponding to the target candidate arm in the deployment optimization parameter set, update the cumulative loss value of the target candidate arm according to the EXP3-IX implicit exploration update rule, use the gated loss value as the update increment, and use the sum of the selection probability of the target candidate arm and the arm-level implicit exploration parameters as the normalization denominator. After normalizing the update increment, add it to the cumulative loss value to obtain the updated cumulative loss value.
[0105] Based on the updated cumulative loss value of the target candidate arms, the selection probability distribution of the feasible candidate arm set is regenerated according to the EXP3-IX decision rule. The updated cumulative loss value and the regenerated selection probability distribution are written back to the delivery optimization parameter set, so that subsequent delivery requests can directly read and inherit the learned decision state when generating the selection probability distribution and sampling target candidate arms. This enables continuous optimization of the online delivery strategy and avoids recalculating from the initial state for each decision.
[0106] This invention constructs an explanation path oriented towards feedback events in the delivery subgraph and generates gating coefficients based on path association strength and rule availability to achieve effective gating of delivery feedback, suppressing the interference of noise attribution and violation risks on learning updates. At the same time, in the EXP3-IX implicit exploration update, selection probability and arm-level implicit exploration parameters are used to jointly participate in the normalized loss update mechanism, taking into account both stability and continuous exploration capability. The updated cumulative loss value and the regenerated selection probability distribution are written back to the delivery optimization parameter set to realize the inheritance and reuse of decision states, thereby improving the robustness, convergence efficiency and long-term optimization effect of the delivery strategy in non-stationary traffic environments.
[0107] Example 1:
[0108] To verify the feasibility of this invention in practice, it was applied to a programmatic advertising scenario on a large-scale integrated e-commerce platform. This platform has multiple media entry points, including search, news feed, and recommendation slots. The target audience includes multi-level advertising configurations and frequently changing rules such as review, inventory, budget, frequency control, and targeting compliance. Moreover, the traffic exhibits significant non-stationarity, such as audience migration caused by hot products, competitor bidding disturbances, and changes in exposure structure due to media strategy adjustments. In this scenario, common problems of existing technologies are that candidate configurations are directly eliminated after triggering hard constraints, resulting in a sharp reduction in the space of optional actions. Online learning and exploration are forced to stop, leading to strategy collapse. Using a uniform exploration intensity for different candidate configurations results in slow convergence and high trial costs in data sparsity and new scenarios. When feedback data contains attribution noise and abnormal fluctuations, directly using it for updates amplifies instability and causes violent fluctuations in probability distribution and increased costs. The above problems are more prominent when rules are frequently adjusted and traffic fluctuations occur in a concentrated manner, directly affecting the stability of the campaign and the optimization benefits.
[0109] When applying this invention, the platform structures and encodes online advertising data such as exposure, clicks, conversions, consumption, bids, material attributes, targeting attributes, rule constraints, and review status to construct an online advertising knowledge graph and generate a graph structure. Simultaneously, it establishes a set of advertising optimization parameters to persistently record the cumulative loss value of candidate arms, arm-level implicit exploration parameters, and selection probabilities. Whenever a new advertising request arrives, the system maps the request context to internal encoding within the graph structure, retrieves candidate advertising configurations in the graph structure, extracts an advertising subgraph, and then generates a set of candidate arms on the advertising subgraph. Rule nodes are retrieved along the constraint relationship edges to form rule constraint information. For candidate arms that trigger hard constraints, the system maps candidate arms that need to retain exploration capabilities to alternative candidate arms that satisfy similar advertising action configurations and pass hard constraint judgment during the feasible domain reconstruction process. This results in a set of feasible candidate arms and ensures that the action space is continuously explorable within the compliance boundary.
[0110] Subsequently, the system calculates a structural uncertainty index based on the sparsity of connections in the deployment subgraph of feasible candidate arms, the stability of relationships reflected by the aggregation of relation edge weights, and the degree of rule conflict. Based on this, it determines the arm-level implicit exploration parameters, giving stronger exploration weights to candidate arms with sparser structures, more unstable relationships, or higher rule conflicts. In the decision-making phase, according to the exponential weighting mechanism of EXP3-IX, the system generates a basic probability from the cumulative loss value and combines it with the arm-level implicit exploration parameters to form a structure-aware exploration allocation to obtain the final selection probability distribution. At the same time, it samples and executes deployment actions and collects feedback data. In the update phase, it locates the deployment object node and feedback event node in the deployment subgraph and retrieves the set of explanatory paths. Based on the path association strength and rule availability, it calculates the gating coefficient, gating the original loss to obtain the gating loss value. Then, according to the EXP3-IX implicit exploration update rules, it normalizes and updates the cumulative loss value by the sum of the selection probability and the arm-level implicit exploration parameters, and recalculates the selection probability and writes it back to the deployment optimization parameter set, so that subsequent deployment requests can inherit the learned state and form an online closed-loop optimization.
[0111] To verify the beneficial effects, the same group of advertisers, the same budget cap, and the same category constraints were used. Comparative verification was conducted in a low-traffic gray-scale environment and a historical replay environment on the platform. The statistical window covered multiple natural periods to ensure that typical non-stationary factors such as rule updates, hotspot fluctuations, and bidding disturbances were included. Comparison methods included a baseline scheme with only rule filtering and static ranking, a scheme using knowledge graph retrieval but without feasible domain reconstruction and gating updates, a scheme using EXP3 but without implicit exploration normalization and arm-level exploration parameters, and the scheme of this invention. Specific comparative data are shown in Table 1.
[0112] Table 1. Comparison of the effects of different campaign optimization schemes
[0113] index Rule filtering and static sorting Knowledge Graph Recall EXP3 This invention Click-through rate (%) 1.62 1.74 1.80 1.93 Conversion rate (%) 2.15 2.22 2.28 2.41 Cost per conversion (RMB) 86.4 83.7 81.9 75.6 Input-output ratio 3.18 3.26 3.34 3.62 Hard constraint trigger rate (one in a thousand) 7.9 7.4 7.1 2.3 Number of feasible candidate arms (mean) 42 55 54 63 Replacement arm acceptance rate (%) 0.0 0.0 0.0 68.5 Fluctuations in probability distribution (mean L1) 0.31 0.29 0.44 0.22
[0114] As shown in Table 1, this invention achieves improvements in both campaign performance and online stability. The click-through rate reaches 1.93%, and the conversion rate reaches 2.41%, both higher than the three comparative schemes. Simultaneously, the cost per conversion is reduced to 75.6 yuan, and the return on investment (ROI) increases to 3.62, demonstrating superior campaign returns. Regarding compliance and usability, the hard constraint trigger rate is reduced to 2.3‰, and the average number of feasible candidate arms is increased to 63 through 68.5% replacement arm acceptance, indicating that feasible domain reconstruction can avoid candidate space collapse and maintain exploration continuity under compliance conditions. In terms of online learning stability, the average probability distribution fluctuation L1 is 0.22, significantly lower than the EXP3 scheme's 0.44, and also lower than the static scheme's 0.31 and 0.29, indicating that gating updates and arm-level implicit exploration parameters synergistically suppress strategy oscillations caused by noise feedback, thereby improving the robustness and optimization efficiency of online iteration.
[0115] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A knowledge graph-based method for optimizing online advertising delivery, characterized in that, Includes the following steps: Acquire online advertising data, construct an online advertising knowledge graph and generate a corresponding graph structure, and establish a set of advertising optimization parameters; Receive an ad delivery request, retrieve candidate delivery configurations in the graph structure, and obtain the delivery subgraph; Generate a candidate arm set based on the deployment subgraph and determine the rule constraint information; Based on the rule constraint information, the feasible region reconstruction process is performed on the candidate arm set to obtain the feasible candidate arm set; For each candidate arm in the feasible candidate arm set, calculate the structural uncertainty index and determine the arm-level implicit exploration parameters of the candidate arm, and write them into the deployment optimization parameter set; The selection probability distribution of the feasible candidate arm set is generated according to the EXP3-IX decision rule. The target candidate arm is sampled according to the selection probability distribution and the corresponding deployment action is executed. The deployment feedback data of the current deployment action is collected. Retrieve the set of explanatory paths corresponding to the delivery feedback data, calculate the gating coefficient, and gating the delivery feedback data to obtain the gating loss value. According to the EXP3-IX implicit exploration update rule, update the cumulative loss value of the target candidate arm using the gating loss value, regenerate the selection probability distribution, and write back the delivery optimization parameter set.
2. The method for optimizing online advertising based on knowledge graphs according to claim 1, characterized in that, The specific steps involved in obtaining the graph structure and the deployment optimization parameter set are as follows: Acquire online advertising data and perform structured processing, uniformly encode the target object identifier, relationship type and relationship attribute, map the target object to the node of the graph, and map the association relationship between the target objects and the constraint relationship of the advertising rule to the edge of the graph; Perform relation labeling and weight configuration operations on the graph, set relation category identifiers for different types of edges to distinguish between association and constraint relationships, configure weight information for each edge, store the graph with completed relation labeling and weight configuration as an online advertising knowledge graph, and obtain the corresponding graph structure; Establish and initialize a set of deployment optimization parameters, and create corresponding parameter records. The parameter records include cumulative loss values, arm-level implicit exploration parameters, and corresponding selection probability values.
3. The method for optimizing online advertising based on knowledge graphs according to claim 1, characterized in that, The specific steps for obtaining the deployment sub-graph include: Receive and parse advertising delivery requests to obtain delivery request context information, and convert the delivery request context information into an internal code consistent with the node identifier in the graph structure to generate a request feature set for graph structure retrieval. Based on the request feature set, a candidate retrieval operation is performed in the graph structure. By locating the starting node corresponding to the request feature set and traversing along the association and constraint edges, a candidate deployment configuration set is obtained. Using the node corresponding to the candidate delivery configuration set as the central node, a subgraph extraction operation is performed in the graph structure according to the preset neighborhood range to collect the central node, associated nodes and corresponding edges, forming a delivery subgraph containing delivery object information and relationship information.
4. The method for optimizing online advertising based on knowledge graphs according to claim 1, characterized in that, The generation of the candidate arm set and the determination of the rule constraint information specifically include: The nodes in the delivery subgraph are classified, and the edges in the delivery subgraph are identified by type. Edges that represent the relationship between delivery objects are marked as relationship edges, and edges that represent the constraint relationship between delivery rules and delivery objects are marked as constraint edges, thus obtaining a delivery subgraph with complete node and edge type markings. In the delivery subgraph after completing the node and edge type labeling, select the material nodes that are in an available state from the delivery object nodes to form a material candidate set, select the targeting nodes that match the delivery request context from the delivery object nodes to form a targeting candidate set, and select the bidding nodes that meet the platform bidding strategy constraints from the delivery object nodes to form a discretized bidding candidate set. Based on the candidate set of materials, the candidate set of targeting, and the discretized bid candidate set, different delivery action configurations are generated according to preset combination rules. Each delivery action configuration is defined as a candidate arm, a unique candidate arm identifier is assigned to each candidate arm, and a one-to-one correspondence between the candidate arm identifier and the delivery action configuration is established. For each candidate arm, locate the delivery object node associated with the corresponding delivery action configuration, retrieve the rule node connected to the delivery object node along the constraint relationship edge in the delivery subgraph, read the constraint conditions in the rule node and perform the judgment to obtain the corresponding constraint judgment result, associate the constraint judgment result with the candidate arm identifier to form rule constraint information, and output the candidate arm set and rule constraint information.
5. The method for optimizing online advertising based on knowledge graphs according to claim 1, characterized in that, The specific steps to obtain the set of feasible candidate arms include: Read the candidate arm set and rule constraint information, extract the associated rule entries and constraint judgment results for each candidate arm, form a correspondence table between candidate arms and rule entries, and mark the constraint judgment results as two states: pass and trigger hard constraint. Perform a removal process on candidate arms marked as triggering hard constraint rules, remove the candidate arms from the candidate arm set and generate a removed candidate arm set, and record the corresponding trigger rule identifier and trigger type in the rule constraint information; For each candidate arm in the candidate arm elimination set, perform an exploration retention judgment. Based on the preset exploration retention conditions, mark the candidate arms that meet the exploration retention conditions as candidate arms to be replaced, and extract the entry of the deployment action configuration and trigger hard constraint rule for each candidate arm to be replaced as the replacement retrieval condition. For each candidate arm to be replaced, a replacement candidate arm is retrieved in the delivery subgraph. During the retrieval, candidate arms whose delivery action configuration satisfies the preset similarity relationship with the candidate arm to be replaced and whose hard constraint rules are met are selected as replacement candidate arms. A mapping relationship between the candidate arm to be replaced and the replacement candidate arm is established. The replacement candidate arm is added to the feasible candidate arm set and the corresponding candidate arm to be replaced is removed. Candidate arms that have not triggered hard constraints are merged to obtain the feasible candidate arm set.
6. The method for optimizing online advertising based on knowledge graphs according to claim 1, characterized in that, The specific methods for obtaining the structural uncertainty index and the arm-level implicit exploration parameters include: Based on the set of feasible candidate arms, locate the delivery object node and relationship edge associated with the delivery action configuration for each feasible candidate arm, forming a candidate arm associated substructure; Based on the candidate arm associated substructure, the connection status of the delivery object nodes associated with the candidate arm in the delivery subgraph is statistically analyzed. The number of node connections is normalized with the preset maximum connection scale to obtain a node sparsity index that reflects the sparsity of the candidate arm structure. Based on the candidate arm association substructure, the weight information of the relationship edge is used as the relationship edge weight. The relationship edge weight is aggregated and normalized with the preset maximum weight to obtain the relationship confidence index that reflects the stability of the candidate arm association relationship. Based on the rule constraint information corresponding to the candidate arm, the number of rules that are determined to be mutually exclusive or conflicting in the rule constraints associated with the candidate arm is counted, and normalized with a preset conflict upper limit to obtain a rule conflict index that reflects the degree of inconsistency of the candidate arm rules. Based on node sparsity index, relationship confidence index, and rule conflict index, the three types of indexes are processed to unify direction, and uncertainty contribution is set. The three types of uncertainty contributions are weighted and merged according to the preset weight ratio rule to obtain structural uncertainty index. The corresponding arm-level implicit exploration parameters are determined based on the structural uncertainty index and written into the deployment optimization parameter set.
7. The method for optimizing online advertising based on knowledge graphs according to claim 1, characterized in that, The collection of delivery feedback data for the current delivery action specifically includes: Read the cumulative loss value and arm-level implicit exploration parameters that correspond one-to-one with each feasible candidate arm identifier from the deployment optimization parameter set, and at the same time read the set of alternative candidate arm identifiers and mapping relationship; Based on the exponential weight decision in the EXP3-IX decision rule, the cumulative loss value of the current feasible candidate arm is subjected to exponential weight transformation, and the basic weight of all feasible candidate arms is subjected to summation and normalization to obtain the basic selection probability distribution based on the cumulative loss value. Simultaneously, an exploration weight is generated for each feasible candidate arm. The exploration weight is directly determined by the arm-level implicit exploration parameters of the current feasible candidate arm. The exploration weights of all feasible candidate arms are summed and normalized to obtain the structure-aware implicit exploration probability distribution. The basic selection probability distribution and the implicit exploration probability distribution are weighted and fused according to a preset mixing coefficient to obtain the final selection probability distribution. Two types of improvement operations are performed to enhance synergy. The first type of improvement operation is to set a minimum selection probability lower limit for the alternative candidate arms obtained by mapping the alternative arms, and at the same time normalize and backfill the probabilities of the remaining candidate arms. The second type of improvement operation is to perform exploration suppression processing on feasible candidate arms in the boundary state in the rule constraint information. The exploration suppression processing is to reduce the final selection probability while keeping the probability sum to one, and to redistribute the difference to other feasible candidate arms. Then, the final selection probability distribution is written into the deployment optimization parameter set. Sampling is performed on the set of feasible candidate arms based on the final selection probability distribution. During sampling, random numbers are generated, and the final selection probability is accumulated in the order of candidate arm identifiers until a random number is reached. The target candidate arm identifier is determined, the deployment action configuration corresponding to the target candidate arm identifier is read and the deployment action is executed, and the deployment feedback data corresponding to the deployment action is collected.
8. The method for optimizing online advertising based on knowledge graphs according to claim 1, characterized in that, The updating of the cumulative loss value of the target candidate arm and the regeneration of the selection probability distribution specifically include: In the delivery subgraph, locate the delivery object node corresponding to the target candidate arm identifier and the feedback event node corresponding to the delivery feedback data. Determine the delivery object node and the feedback event node as the starting node and ending node of the path search. Perform path search along the association relationship edge and constraint relationship edge in the delivery subgraph, filter the node sequence that meets the preset path length range and contains a rule node, and obtain the set of explanatory paths corresponding to the delivery feedback data. For each interpretation path in the interpretation path set, read the relation edge weights of each relation edge in the path, perform aggregation on the edge weights in the same path to obtain the path association strength, and generate path usability tags based on rule entries and constraint judgment results; The gating coefficients are determined based on path association strength and path availability marking. The gating coefficients of the explanatory paths marked as disallowed are set to zero, while the gating coefficients of the explanatory paths marked as allowed are set to the coefficient values obtained by normalizing the path association strength. The upper and lower bounds of the gating coefficients are truncated, and the gating coefficients of each path in the explanatory path set are aggregated to obtain the final gating coefficients. Read the original loss value corresponding to the target candidate arm, generate the gated loss value based on the final gate coefficient, read the selection probability and arm-level implicit exploration parameters corresponding to the target candidate arm in the deployment optimization parameter set, update the cumulative loss value of the target candidate arm according to the EXP3-IX implicit exploration update rule, use the gated loss value as the update increment, and use the sum of the selection probability of the target candidate arm and the arm-level implicit exploration parameters as the normalization denominator. After normalizing the update increment, add it to the cumulative loss value to obtain the updated cumulative loss value. Based on the updated cumulative loss value of the target candidate arm, the selection probability distribution of the feasible candidate arm set is regenerated according to the EXP3-IX decision rule, and the updated cumulative loss value and the regenerated selection probability distribution are written back to the deployment optimization parameter set.