A real-time advertising evaluation and decision-making system based on large language model agent

Abstract

The present invention discloses a real-time advertising evaluation decision system based on a large language model intelligent agent, the system includes: a problem modeling module, a decision agent construction module, and an advertising evaluation result output module; the problem modeling module models the original advertising evaluation decision problem as a Markov process; the decision agent construction module constructs a decision agent based on a large language model, including: a configuration file guiding unit, a click-through rate prediction model training unit, a basic decision factor generating unit, a memory module construction unit, a reflection module construction unit, an insight reasoning unit, and an action formulation unit; the advertising evaluation result output module obtains the final advertising evaluation result of each advertisement. The present invention constructs a decision agent that integrates a click-through rate prediction model, an expert rule strategy, a multi-memory retrieval mechanism, and a reflection mechanism based on the decision agent construction module to simulate the real-world advertising environment, reduce the learning difficulty and prediction difficulty of the model, and improve the efficiency and effect of advertising evaluation.

Description

Technical Field

[0001] The present invention relates to the technical field of large language model evaluation and decision-making, and in particular to a real-time advertising evaluation and decision-making system based on a large language model intelligent agent. Background Art

[0002] Although real-time advertising evaluation and decision-making technology has benefited widely from technologies such as deep learning and reinforcement learning, the reliability of related technologies often faces challenges from differences between online and offline environments and rapid fluctuations in the online environment. Existing rule-based strategies are too rigid and cannot adapt to dynamic changes in the market; while reinforcement learning methods have better adaptability, they face problems such as large training data requirements, difficulty in training convergence, and lack of explainability of decisions, which affect their applicability, credibility, and stability. Therefore, it is necessary to apply more advanced machine learning models in real-time advertising evaluation decisions to adapt to market changes in real time, so as to make more adaptive decisions in real time.

[0003] Large language models have made significant progress in achieving general artificial intelligence by enhancing the capabilities of intelligent agents, improving autonomy, responsiveness, and interactive skills, and enabling intelligent agents to handle complex problems. For example, in the field of education, cognitive science principles are integrated to guide large language models, enhancing their ability to model and understand different learning behaviors and outcomes; in the medical field, large language models are used to simulate hospital environments, allowing medical institutions to adapt and improve their treatment strategies through interactive learning. However, due to the need for targeted problem modeling and the specificity of the evaluation decision process in the field of advertising evaluation and decision-making, there is no existing solution for advertising evaluation and decision-making based on large model agents. Summary of the invention

[0004] In order to overcome the defects and shortcomings of the prior art, the present invention provides a real-time advertising evaluation and decision-making system based on a large language model intelligent agent. The present invention models the original real-time advertising evaluation and decision-making problem as a Markov process based on a problem modeling module. Through corpus organization and LoRA supervised fine-tuning, the large language model is equipped with knowledge of the advertising evaluation and decision-making field. At the same time, a decision-making intelligent agent is constructed based on a decision-making intelligent agent construction module for the advertising evaluation and decision-making field, which integrates a click-through rate prediction model, an expert rule strategy, a multi-memory retrieval mechanism and a reflection mechanism, simulates the real-world advertising environment, and reduces the difficulty of adapting the model under dynamic market conditions. Based on the memory module, historical decision data is stored and analyzed to help the model learn from historical experience and improve the accuracy of decision-making. Based on the daily reflection mechanism of the reflection module, the effect of each decision is evaluated to ensure the long-term performance of the model, thereby significantly improving the efficiency and effect of advertising evaluation. At the same time, a transparent decision-making process is provided. Based on the insight reasoning unit and the action formulation unit, a two-step decision-making process is executed in cooperation to dynamically adjust the advertising decision strategy to ensure that the system's actions are strategically consistent with the goal of achieving the highest decision return in the decision-making process, fully improve the evaluation and decision-making ability of the intelligent agent, and improve the effect of advertising evaluation.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides a real-time advertising evaluation and decision-making system based on a large language model intelligent agent, comprising: a problem modeling module, a decision intelligent agent construction module, and an advertising evaluation result output module;

[0007] The problem modeling module is used to model the original advertisement evaluation decision problem as a Markov process;

[0008] The decision agent building module is used to build a decision agent based on a large language model, and specifically includes: a configuration file guiding unit, a click rate prediction model training unit, a basic decision factor generating unit, a memory module building unit, a reflection module building unit, an insight reasoning unit and an action formulation unit;

[0009] The configuration file guiding unit is used to generate a configuration file, the click rate prediction model training unit is used to train a click rate prediction model, and output the advertising click rate prediction result of historical data, the basic decision factor generating unit is used to calculate the basic decision factor according to the advertising click rate prediction result, the memory module building unit is used to build a memory module, the memory module calls the operation of the large language model to obtain the memory summary result, the reflection module building unit is used to build a reflection module, the reflection module inputs the memory of the current time step into the large language model to obtain the daily reflection result, the insight reasoning unit is used to input the advertising activity status of the current time step, the memory summary result and the basic decision factor into the large language model to obtain the insight reasoning result, and the action formulation unit is used to input the advertising activity status of the current time step, the memory summary result, the insight reasoning result and the basic decision factor into the large language model to obtain the action formulation result;

[0010] The advertisement evaluation result output module is used to obtain the final advertisement evaluation result of each advertisement based on the advertisement click rate estimation result, the action formulation result and the basic decision factor.

[0011] As a preferred technical solution, the problem modeling module is used to model the original advertisement evaluation decision problem as a Markov process, specifically including:

[0012] The original advertising evaluation decision problem is formally modeled as a problem of maximizing the value of a given budget, expressed as:

[0013] ;

[0014] ;

[0015] in, , Respectively represent advertising display i value and cost, Indicates winning or losing a display ad impression, B represents the budget of the advertising campaign;

[0016] In order to solve the problem of maximizing the value of a given budget, advertising display is introduced i Advertising decision results It is expressed as:

[0017] ;

[0018] in, is a feasible decision factor;

[0019] The formal problem is modeled as a Markov model, where at each time step , the agent’s advertising activity status based on the current time step , according to the strategy Execution tuning parameters To update the feasible decision factors , transition to a new state according to the transition mapping;

[0020] in, represents the set of ad campaign states at the current time step, Represents the action space, transfer mapping , represents the Cartesian product, yes The set of probabilities and distributions on , Represents a mapping from elements in one set to elements in another set;

[0021] The environment provides the agent with immediate rewards based on the current state of the advertising campaign and the agent's behavior, and obtains a strategy that maximizes the expected cumulative reward, expressed as:

[0022] ;

[0023] ;

[0024] ;

[0025] in, Indicates that at time step t Status of the campaign and internal reasoning information Under the condition of , the strategy of the large language model agent to output actions, G(·) represents the operation parsing function of the large language model, is the strategy parameter, Indicated in strategy The expected cumulative reward under Indicates that at time step t Status of the campaign Assign the value to the corresponding campaign status value s , Indicates that at time step t Internal reasoning information Assign the corresponding internal reasoning information value , It is i The reward discount factor for time steps, Indicates reward, Indicates the memory summary result, Indicates the tool call results, including the ad click-through rate estimation results and basic decision factors. Indicates insight into reasoning results, Indicates the result of action formulation. Indicates the results of daily reflection.

[0026] As a preferred technical solution, the click rate prediction model includes an input layer, an embedding layer for feature mapping, an interaction layer for capturing feature interactions, and an output layer. The input layer uses the feature vector of the advertisement display as input, and the output layer is based on sigmoid The function converts the output into probability form;

[0027] The click-through rate prediction model uses binary cross entropy loss to measure the difference between the probability distribution predicted by the model and the actual probability distribution.

[0028] As a preferred technical solution, the basic decision factor generation unit is used to calculate the basic decision factor according to the advertisement click rate estimation result, specifically including:

[0029] Calculate the ratio of the estimated click-through rate of each historical traffic data to the ad display cost to get the cost-effectiveness, sort them from large to small, and calculate the maximum top that can be obtained under the set budget. flow, use The inverse ratio of the traffic cost performance is used as the basic decision factor.

[0030] As a preferred technical solution, the memory module calls the operation of the large language model to obtain the memory summary result, which is specifically expressed as:

[0031] ;

[0032] in, Indicates the memory summary result, Is the memory type i At time step t Memory, It is a template for the reminder words of memory summary. Refers to the operation of calling a large language model. bid represents the decision memory type, env Indicates the environment memory type, ref Indicates reflective memory type.

[0033] As a preferred technical solution, the reflection module inputs the memory of the current time step into the large language model to obtain the daily reflection result, which is specifically expressed as:

[0034] ;

[0035] in, Indicates the results of daily reflection, It is a template for daily reflection prompts. Refers to the operation of calling a large language model. Is the memory type i At time stept Memory, bid represents the decision memory type, env Indicates the environment memory type, ref Indicates reflective memory type.

[0036] As a preferred technical solution, the insight reasoning unit is used to input the advertising activity status, memory summary results and basic decision factors of the current time step into the large language model to obtain the insight reasoning result, which is specifically expressed as:

[0037] ;

[0038] in, Indicates insight into reasoning results, Refers to the operation of calling a large language model. It is a prompt word template for insight reasoning. is the time step t The campaign status, Indicates the memory summary result, Represents the basic decision factor.

[0039] As a preferred technical solution, the action formulation unit is used to input the advertising activity status, memory summary results, insight reasoning results and basic decision factors of the current time step into the large language model to obtain the action formulation result, which is specifically expressed as:

[0040] ;

[0041] in, It is a prompt word template for action formulation. Refers to the operation of calling a large language model. is the time step t The campaign status, Indicates the memory summary result, represents the basic decision factor, Represents insight reasoning results.

[0042] As a preferred technical solution, the advertisement evaluation result output module is used to obtain the final advertisement evaluation result of each advertisement based on the advertisement click rate estimation result, the action formulation result and the basic decision factor, which is specifically expressed as:

[0043] ;

[0044] in, represents the basic decision factor, is the time step t Middle i Click-through rate estimates for ad impressions, Represents the time stept No. i The evaluation result of the ad display, the action formulation result is composed of a binary Composition, among which, is the action adjustment parameter of the decision, is the reason for the decision;

[0045] The final advertising evaluation result of each advertisement is interacted with the market environment to make a decision at the current time step, and the advertising activity status at the next time step is obtained based on the decision feedback from the market environment.

[0046] As a preferred technical solution, the large language model is fine-tuned by corpus organization and LoRA supervision to obtain a fine-tuned large language model, and the corpus is constructed in the form of question-answer pairs.

[0047] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0048] (1) The present invention constructs a decision-making agent for the field of advertising evaluation decision-making based on the decision-making agent building module, which integrates a click-through rate prediction model, an expert rule strategy, a multi-memory retrieval mechanism, and a reflection mechanism. It simulates the real-world advertising environment, reduces the difficulty of the model to adapt under dynamic market conditions, and significantly reduces the learning difficulty and prediction difficulty of the model, which is conducive to improving the efficiency and effectiveness of advertising evaluation.

[0049] (2) The present invention stores and analyzes historical decision data based on a memory module, helping the model learn from historical experience and improving the accuracy of decision-making, so that the system can better integrate historical data and current market conditions and improve the accuracy and adaptability of real-time advertising evaluation.

[0050] (3) The present invention evaluates the effect of each decision based on the daily reflection mechanism of the reflection module, solves the adaptation problem of the model under dynamic market conditions, improves the decision-making ability of the model under rapid market changes, improves the long-term performance and stability of the model, and thus significantly improves the efficiency and effectiveness of advertising evaluation, while providing a transparent decision-making process.

[0051] (4) The present invention is based on the two-step decision-making process executed by the insight reasoning unit and the action formulation unit, automatically creating multiple evaluation paths for the input data, so that the model can predict the advertising effect from multiple angles, improve the robustness of the evaluation results, and ensure that the system's actions are strategically consistent with the goal of achieving the highest decision return in the decision-making process. By dynamically adjusting the advertising decision-making strategy, the evaluation and decision-making ability of the intelligent body is fully improved, thereby improving the effect of advertising evaluation.

[0052] (5) The present invention obtains a fine-tuned large language model through corpus organization and LoRA supervised fine-tuning, so that the large language model has domain knowledge of advertising evaluation decision-making, can better process and understand complex advertising environments, and improve the depth and breadth of advertising evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of the overall architecture of the real-time advertising evaluation and decision-making system based on a large language model intelligent agent of the present invention;

[0054] Figure 2 Schematic diagram of the network architecture of the click rate prediction model of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0056] like Figure 1 As shown, this embodiment provides a real-time advertising evaluation decision system based on a large language model agent, including: a problem modeling module, a decision agent building module, and an advertising evaluation result output module;

[0057] The problem modeling module is used to model the original advertising evaluation decision problem as a Markov process, capturing the scenario modeling involving the state transition of advertising activities in the decision-making process, including:

[0058] The original advertising evaluation decision problem is formally modeled as a problem of maximizing the value of a given budget, which is specifically expressed as:

[0059] ;

[0060] ;

[0061] in, , Respectively represent advertising display i value and cost, Indicates whether the display ad impression was won or lost, and is a binary value. B represents the budget of the advertising campaign;

[0062] In order to solve the problem of maximizing the value of a given budget, advertising display is introduced i Advertising decision results It is expressed as:

[0063] ;

[0064] in, is a feasible decision factor. Since the market environment is dynamic and highly non-stationary, the optimal It is difficult to determine simply and directly, so the key to this embodiment is to dynamically adjust To adapt to the ever-changing market environment;

[0065] The formal problem is modeled as a Markov process, where the key feature is a set representing the state of the advertising campaign at the current time step , a feasible decision factor Adjustment parameters Action Space ;

[0066] The state of the campaign at the current time step Contains a variety of metrics, including historical total ad impressions, historical average estimated CTR for ad impressions, historical average actual CTR for ad impressions, historical average actual win rate for campaign impressions, historical average cost of ad impressions, total budget allocated to the campaign, and remaining budget. These elements are critical because they provide a comprehensive snapshot of the campaign situation at a specific time.

[0067] At each time step , the agent’s advertising activity status based on the current time step , according to the strategy Execution tuning parameters To update the feasible decision factors , and then transition to a new state according to the transition map. , represents the Cartesian product, yes The set of probabilities and distributions on , Represents a mapping from elements in one set to elements in another set;

[0068] The environment provides the agent with an immediate reward based on the current ad campaign status and agent behavior, expressed as ,in, is a reward space. To this end, considering the budget constraint, the total number of time steps is set T To maximize the total discounted reward, the strategy sought is the one that maximizes the expected cumulative reward, as follows:

[0069] ;

[0070] in, is the strategy parameter, Representation parameters The strategy under this is to change the state Mapping to Actions , Indicated in strategy The expected cumulative reward under It is i The reward discount factor for each time step;

[0071] In order to utilize external reasoning information to optimize the strategy, the strategy is first formalized and expanded as follows:

[0072] ;

[0073] in, Indicates the conditional state. Indicates that at time step t Status of the campaign Assign the value to the corresponding campaign status value s ; Indicates that at time step t Internal reasoning information Assign the corresponding internal reasoning information value ; is a specialized module that encapsulates the useful internal reasoning process. The strategy is further expressed as:

[0074] ;

[0075] ;

[0076] in, Indicates that at time step t Status of the campaign and internal reasoning information Under the condition of , the strategy of the large language model agent to output actions; G(·) is an operation parsing function used to perform compatible formal operations in the environment. The operation parsing function is driven by the large language model and its reasoning information Aggregated by different information, including campaign status , memorize the summary results , Tool call results (including ad click-through rate estimates , basic decision factors ), insight into the inference results , Action Development Results , Daily reflection results ;

[0077] Through the analysis of the large language model, the large language model agent can continuously interact with the market environment to drive it to achieve the optimization goal, thereby ensuring that the agent's actions are closely integrated with the strategy that provides high expected decision returns, while taking into account the state of the advertising campaign and internal reasoning information at each time step, as shown in the following formula:

[0078] ;

[0079] in, refers to Based on this, the large language model agent can continuously interact with the market environment to maximize the expected cumulative reward. strategy;

[0080] In this embodiment, the decision agent building module is used to build a decision agent based on a large language model, including a profile guiding unit, a click rate prediction model training unit, a basic decision factor generating unit, a memory module building unit, a reflection module building unit, an insight reasoning unit and an action formulation unit;

[0081] In this embodiment, 20,000 corpora in the field of advertising evaluation and decision-making are constructed in the form of question-answer pairs by manually sorting and summarizing by experts in the field of advertising evaluation and decision-making, and the large language model is fine-tuned by LoRA supervision using the LLaMA-Factory framework, so that the large language model has knowledge in the field of advertising evaluation and decision-making, thereby constructing a fine-tuned large language model in the field of advertising evaluation and decision-making;

[0082] In this embodiment, the profile guiding unit is used to generate a profile carefully designed for the advertising expert, endow the advertising expert with acumen, and based on the context carefully designed for his role, serve as an input prompt word for the large language model;

[0083] Specifically, a carefully designed profile for advertising experts is defined in text form as follows: "You are a senior data analyst specializing in in-depth research and strategy development in the field of real-time advertising delivery. You use advanced data analysis tools and algorithms to guide advertisers to gain an advantage in the fierce market competition. Your goal is to maximize clicks through data analysis and algorithm adjustments given the budget for the day." This enables the agent to understand complex advertising processes and make more professional and accurate evaluations and decisions in real-time advertising campaigns.

[0084] In this embodiment, the click rate prediction model training unit is used to learn the statistical association between the features in the historical data and the click behavior, and obtain the advertising click rate prediction result of the historical data. , predict future click behavior, specifically expressed as:

[0085] ;

[0086] in, is the time step t Middle i Click-through rate estimates for ad impressions, is the time step t Total number of ad impressions;

[0087] In this embodiment, the click rate prediction model uses the feature vector of the advertisement display as input, including factors that affect the user's click behavior, such as user information, advertisement size, advertisement category, and advertisement placement location. The model is trained based on the historical placement data of each advertiser, which contains the user's click feedback on the advertisement, so that the model can learn and predict future click behavior;

[0088] like Figure 2 As shown in Figure 1, the click rate prediction model includes an input layer, an embedding layer for feature mapping, an interaction layer for capturing feature interactions, and an output layer. Finally, sigmoid The function converts the output into a probability form to predict the possibility of user clicks. The click rate prediction model uses a binary cross entropy loss to measure the difference between the probability distribution predicted by the model and the actual probability distribution, which is specifically expressed as:

[0089] ;

[0090] in, y is the actual label, with a value of 0 or 1. is the probability of the model predicting a click;

[0091] In this embodiment, the basic decision factor generation unit is used to obtain the basic decision factor according to the historical data. To maximize returns, specifically, calculate the ratio of the estimated click-through rate of each historical traffic data to the ad display cost to get the cost-effectiveness, sort them from large to small, and then calculate the maximum number of top ads that can be obtained under the specified budget. flow, use The inverse ratio of traffic cost performance is used as the basic decision factor , where the click-through rate estimate is predicted by the offline trained click-through rate prediction model of the corresponding advertiser.

[0092] In this embodiment, the memory module construction unit is used to construct a memory module. t Get the memory summary result , specifically expressed as:

[0093] ;

[0094] in, Is the memory type i At time step t Memory, It is a template for the reminder words of memory summary. Refers to the operation of calling fine-tuning a large language model;

[0095] The memory module is designed to provide a powerful multi-dimensional retrieval system that progressively segments and updates information, ensuring that the system has access to accurate and relevant data, including environmental memory , Decision Memory and reflective memory , respectively storing the market environment, decision context, and daily reflection results after decision making;

[0096] in, , The update comes from the current time step of the advertising activity state ; The update comes from the memory summary result of the current time step , Tool call results (including ad click-through rate estimates , basic decision factors ), insight into the inference results , Action Development Results ; Reflection on memory The update comes from the daily reflection result of the current time step .

[0097] In this embodiment, the information of the current time step (including the advertising activity status) needs to be , memorize the summary results , Insight into Reasoning Results , Tool call results , Action Development Results , Daily reflection results ) are stored in real time in the memory space to facilitate the agent’s continuous learning and improvement in the ever-changing market environment, ensuring that it can remain competitive, including environmental memory, decision memory, and reflective memory until the budget is consumed, the complete advertising campaign, and the agent’s entire advertising evaluation decision process.

[0098] In this embodiment, the reflection module construction unit is used to construct the reflection module. The reflection module embeds the memory of the current time step into the prompt word template of daily reflection. , input into the fine-tuned large language model to get the daily reflection results , used for regular decision review to cultivate rewarding insights to enhance subsequent actions, helping the agent learn from daily decisions and results to continuously improve its strategic decision-making, ensuring that the agent can remain competitive in an ever-changing market environment.

[0099] In this embodiment, the reflection module performs daily reflection after the advertising decision at each time step and obtains the daily reflection results. , specifically expressed as:

[0100] ;

[0101] in, It is a template for daily reflection prompts. Refers to the operation of calling fine-tuning a large language model. Reflective memory provides an evaluation of past decisions, helping to identify which strategies are effective and which need improvement. This information can be used to adjust and optimize future delivery strategies, thereby improving the accuracy and effectiveness of advertising evaluation. Reflective memory enables intelligence to continuously learn and adapt to market changes. By reflecting on past decisions, the agent can adjust its evaluation decision-making strategy to adapt to these changes.

[0102] In this embodiment, the insight reasoning unit converts the advertising activity status of the current time step into , memorize the summary results And the basic decision factors Cue word template embedded in insight reasoning , input into the fine-tuned large language model to obtain insight into the inference results , get analyzed benefits, drawbacks, possible outcomes and associated risks;

[0103] Specifically, the agent analyzes the discrete set of potential decision ranges {[-0.5,-0.4), [-0.4,-0.3), [-0.3,-0.2), [-0.2,-0.1), [-0.1,0.0), [0.0,+0.1), [+0.1,+0.2), [+0.2,+0.3), [+0.3,+0.4), [+0.4,+0.5]}, analyzing their benefits, drawbacks, possible outcomes, and associated risks.

[0104] In this embodiment, insight into the inference results To analyze the advantages, disadvantages, benefits and risks of different potential decision ranges, specifically expressed as:

[0105] ;

[0106] in, It is a prompt word template for insight reasoning. is the time step tThe campaign status, Refers to the operation of calling fine-tuning a large language model;

[0107] In this embodiment, the action formulation unit calculates the advertising activity status of the current time step. , memorize the summary results , Insight into Reasoning Results And the basic decision factors Template for embedding prompts into action planning , input into the fine-tuned large language model, and obtain the action formulation result , the result is a two-tuple Composition, among which, is the action adjustment parameter of the decision, The action making unit converts the results of insight reasoning into specific decision actions and determines the actual evaluation value and its reasons.

[0108] Action Development Results It is expressed as:

[0109] ;

[0110] in, It is a prompt word template for action formulation. Refers to the operation of calling fine-tuning a large language model;

[0111] In this embodiment, the insight reasoning unit and the action formulation unit cooperate to perform a two-step decision process to formulate and execute well-thought-out operations, ensuring that the system's actions are strategically aligned with the goal of achieving the highest decision return in the decision process.

[0112] In this embodiment, the advertisement evaluation result output module is used to obtain the final advertisement evaluation result of each advertisement, which is specifically expressed as follows:

[0113] ;

[0114] in, is the time step t Middle i Click-through rate estimates for ad impressions, Represents the time step t No. iThe evaluation results of each advertisement are collected, and the final advertisement evaluation results of each advertisement are interacted with the market environment to make a decision at the current time step. The advertising activity status at the next time step is obtained based on the decision feedback from the market environment, including the total number of historical advertisement displays, the average estimated click-through rate of historical advertisement displays, the average actual click-through rate of historical advertisement displays, the average actual win rate of historical advertisement activity displays, the average cost of historical advertisement displays, the total budget allocated to the advertising activity, and the remaining budget, thereby realizing the conversion of the advertising activity status.

[0115] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A real-time advertising evaluation and decision-making system based on a large language model agent, characterized in that: include: Problem modeling module, decision-making agent building module, advertising evaluation result output module; The problem modeling module is used to model the original advertisement evaluation decision problem as a Markov process; The decision agent building module is used to build a decision agent based on a large language model, and specifically includes: a configuration file guiding unit, a click rate prediction model training unit, a basic decision factor generating unit, a memory module building unit, a reflection module building unit, an insight reasoning unit and an action formulation unit; The configuration file guiding unit is used to generate a configuration file, the click rate prediction model training unit is used to train a click rate prediction model, and output the advertising click rate prediction result of historical data, the basic decision factor generating unit is used to calculate the basic decision factor according to the advertising click rate prediction result, the memory module building unit is used to build a memory module, the memory module calls the operation of the large language model to obtain the memory summary result, the reflection module building unit is used to build a reflection module, the reflection module inputs the memory of the current time step into the large language model to obtain the daily reflection result, the insight reasoning unit is used to input the advertising activity status of the current time step, the memory summary result and the basic decision factor into the large language model to obtain the insight reasoning result, and the action formulation unit is used to input the advertising activity status of the current time step, the memory summary result, the insight reasoning result and the basic decision factor into the large language model to obtain the action formulation result; The memory module calls the operation of the large language model to obtain the memory summary result, which is specifically expressed as: ； in, Indicates the memory summary result, Is the memory type i At time step t Memory, It is a template for the reminder words of memory summary. Refers to the operation of calling a large language model. bid represents the decision memory type, env Indicates the environment memory type, ref Indicates reflective memory type; The reflection module inputs the memory of the current time step into the large language model to obtain the daily reflection result, which is specifically expressed as: ； in, Indicates the results of daily reflection, It is a template for prompt words for daily reflection; The insight reasoning unit is used to input the advertising activity status, memory summary results and basic decision factors of the current time step into the large language model to obtain the insight reasoning result, which is specifically expressed as: ； in, Indicates insight into reasoning results, It is a prompt word template for insight reasoning. is the time step t The campaign status, represents the basic decision factor; The action formulation unit is used to input the advertising activity status, memory summary results, insight reasoning results and basic decision factors of the current time step into the large language model to obtain the action formulation result, which is specifically expressed as: ； in, It is a prompt word template for action formulation; The advertisement evaluation result output module is used to obtain the final advertisement evaluation result of each advertisement based on the advertisement click rate estimation result, the action formulation result and the basic decision factor, which is specifically expressed as: ； in, is the time step t Middle i Click-through rate estimates for ad impressions, Represents the time step t No. i The evaluation result of the ad display, the action formulation result is composed of a binary composition, is the action adjustment parameter of the decision, is the reason for the decision; The final advertising evaluation results of each advertisement are interacted with the market environment to make decisions at the current time step, and the advertising activity status at the next time step is obtained based on the decision feedback from the market environment; The advertisement evaluation result output module is used to obtain the final advertisement evaluation result of each advertisement based on the advertisement click rate estimation result, the action formulation result and the basic decision factor.

2. The real-time advertising evaluation and decision-making system based on a large language model agent according to claim 1 is characterized in that: The CTR prediction model includes an input layer, an embedding layer for feature mapping, an interaction layer for capturing feature interactions, and an output layer. The input layer uses the feature vector of the advertisement display as input, and the output layer is based on sigmoid The function converts the output into probability form; The click-through rate prediction model uses binary cross entropy loss to measure the difference between the probability distribution predicted by the model and the actual probability distribution.

3. The real-time advertising evaluation and decision-making system based on a large language model agent according to claim 1 is characterized in that: The basic decision factor generation unit is used to calculate the basic decision factor according to the advertisement click rate estimation result, specifically including: Calculate the ratio of the estimated click-through rate of each historical traffic data to the ad display cost to get the cost-effectiveness, sort them from large to small, and calculate the maximum top that can be obtained under the set budget. flow, use The inverse ratio of the traffic cost performance is used as the basic decision factor.

4. The real-time advertising evaluation and decision-making system based on a large language model agent according to claim 1 is characterized in that: The large language model is fine-tuned by corpus organization and LoRA supervision to obtain a fine-tuned large language model, and the corpus is constructed in the form of question-answer pairs.

Images