A Method and System for Generating Advertising Decisions Based on Neural Temporal Processes

Abstract

This application relates to the field of advertising delivery technology, and more particularly to an advertising delivery decision generation method and system based on neural time-series point processes. The method includes acquiring historical user interaction data and converting it into a continuous event sequence, while simultaneously calculating the time interval between adjacent events; calculating the user's continuous time state using a state encoder based on the continuous event sequence and time intervals; constructing a quantile regression value evaluation network to perform quantile evaluation on the long-term cumulative reward distribution of candidate actions in the continuous time state to obtain a quantile representation of the value distribution; calculating cognitive uncertainty based on the quantile representation and constructing an enhanced reward to adjust the reward items in the evaluation network training; calculating the risk-adjusted value of candidate actions using the quantile representation and combined with preset risk preference parameters; determining the target candidate action according to the principle of maximizing the risk-adjusted value and generating the corresponding advertising delivery decision. This application enables the accurate determination of advertising delivery strategies.

Description

Technical Field

[0001] This application relates to the field of advertising delivery technology, and in particular to an advertising delivery decision generation method and system based on neural time-series point processes. Background Technology

[0002] With the development of advertising on internet platforms, mobile applications, and e-commerce systems, ad placement is no longer simply about displaying ad content to users. It now requires a comprehensive approach, combining user profiles, historical interaction behavior, and the current context to determine the target audience, content, and timing of ad display. A common approach in existing technologies is ad ranking and placement based on click-through rate (CTR) prediction models. This involves using supervised learning to jointly model user characteristics, ad characteristics, and contextual features, outputting the probability of a user clicking on the current ad, and then selecting ads with higher CTR probabilities for display. While this method can improve the click-through rate in a single exposure scenario, its focus is primarily on the immediate response result. Essentially, it judges based on the revenue from a single exposure and fails to reflect the sustained impact of ad placement on subsequent user interactions and long-term revenue.

[0003] Building upon this foundation, some existing technologies introduce reinforcement learning methods into advertising scenarios, modeling the interaction between users and the platform as a sequential decision-making process. These methods typically first encode the user's historical browsing, clicks, conversions, and other behavioral sequences to obtain a feature vector representing the user's current state. Then, a reinforcement learning model selects the current action from multiple candidate advertising strategies based on this state, and updates the model parameters based on the reward signal generated by user feedback. This allows the system to continuously adjust its advertising strategy across multiple rounds of interaction. For ease of modeling and training, these methods typically divide the user interaction process into fixed discrete time steps, performing state updates, action selection, and value evaluation at each discrete time point. Simultaneously, during the strategy evaluation phase, the expected return of the state-action value function is usually used as a criterion for evaluating the merits of different advertising strategies, and the final advertising strategy is determined accordingly.

[0004] However, the aforementioned existing technologies still have significant limitations in practical applications. On the one hand, user behavior in real-world scenarios occurs within a continuous time domain. Different users exhibit significant differences in their active moments, behavioral intervals, and the rhythm of interest changes. Existing technologies, by modeling the interaction process using fixed discrete time steps, essentially compress the continuously changing user behavior process into a few preset time nodes. Therefore, they struggle to accurately reflect the dynamic changes in user states over real time and to accurately determine the timing of ad placement. On the other hand, the returns from ad placement are inherently uncertain. Even if different ad strategies have similar expected returns, their return volatility can vary significantly. Existing technologies rely solely on expected returns for value assessment, making it difficult to further differentiate the differences in return stability and risk levels among different strategies. Thus, existing technologies cannot simultaneously ensure both the accuracy of ad placement timing and the stability of ad strategy selection. Therefore, how to characterize the dynamic changes of user behavior over continuous time during ad placement and, based on this, comprehensively evaluate the long-term returns and volatility characteristics of ad strategies to achieve accurate determination of ad placement strategies has become a technical problem that needs to be solved in this field. Summary of the Invention

[0005] This application provides a method and system for generating advertising delivery decisions based on neural time-series point processes, which can accurately determine advertising delivery strategies. This application provides the following technical solutions:

[0006] In a first aspect, this application provides a method for generating advertising delivery decisions based on neural time-series point processes, the method comprising:

[0007] Acquire user historical interaction data and transform it into a continuous event sequence with absolute timestamps, event types, and event-related features, while calculating the time interval between adjacent events;

[0008] Based on the continuous event sequence and the time interval, a pre-built state encoder is used to calculate the user's continuous time state through event-triggered state updates and state evolution between events;

[0009] A quantile regression value assessment network is constructed to perform quantile assessment on the long-term cumulative return distribution of candidate actions in the continuous time state, thereby obtaining the quantile representation of the value distribution.

[0010] The cognitive uncertainty corresponding to the continuous-time state and the candidate action is calculated based on the quantile representation of the value distribution, and an enhanced reward is constructed based on the cognitive uncertainty to adjust the reward item in the training of the quantile regression value evaluation network.

[0011] The risk-adjusted value of candidate actions is calculated using the quantile representation of the value distribution and in combination with preset risk preference parameters. Target candidate actions are determined according to the principle of maximizing risk-adjusted value, and corresponding advertising placement decisions are generated.

[0012] In one specific implementation, the step of acquiring user historical interaction data and converting it into a continuous event sequence with absolute timestamps, event types, and event-related characteristics, while calculating the time interval between adjacent events, includes:

[0013] Extract the target user's original historical interaction data from the log database, sort each interaction record according to the time of the event, and represent the sorted historical interaction sequence as the user's historical trajectory.

[0014] For any user Its historical trajectory is recorded as ,in, This indicates the total number of events contained in the user's historical trajectory. Indicates the first The consecutive absolute time of the occurrence of this event Indicates the first The event tag type corresponding to this event. Indicates the first The event-related feature vector corresponding to each event; the user's historical trajectory is a continuous sequence of events arranged in chronological order of their occurrence.

[0015] For the This event, and the corresponding time interval, is denoted as . ,in, Indicates the first The second event and the first The time interval between events Indicates the first The consecutive absolute time of the occurrence of this event Indicates the first The continuous absolute time of the occurrence of the event; when season .

[0016] In one specific implementation, the step of calculating the user's continuous-time state using a pre-built state encoder based on the continuous event sequence and the time interval, through event-triggered state updates and inter-event state evolution, includes:

[0017] The state encoder is a state calculation structure composed of a long short-term memory network and a continuous-time decay mechanism. It is used to receive a continuous sequence of events and the time interval between adjacent events, and output the user's state result in continuous time.

[0018] When the This event occurred at the time When this occurs, the user's hidden state is updated using a Long Short-Term Memory (LSTM) network. The update formula is as follows:

[0019] ;

[0020] in, Indicates the first The hidden state after the event occurs. Indicates the first The hidden state corresponding to this event. Indicates event type The corresponding embedding vector, This represents a vector concatenation operation;

[0021] Obtain the hidden state Then, the state changes between adjacent events are continuously calculated; under any condition that... The moment Continuous time state Calculate according to the following formula:

[0022] ;

[0023] in, Indicates time The corresponding continuous time state, Represents the gradually decaying target vector; This represents the learnable decay rate parameter. Indicates satisfaction At any time, This indicates exponentiation.

[0024] In a specific implementation scheme, the construction of the quantile regression value assessment network to perform quantile assessment on the long-term cumulative return distribution of candidate actions in the continuous-time state, and to obtain the quantile representation of the value distribution, includes:

[0025] Constructing a quantile regression value assessment network , Representing network parameters in continuous-time states and candidate actions As input, the number of quantiles is set to... and set the first The quantile probability corresponding to each quantile is:

[0026] ;

[0027] in, Indicates the number of quantiles. Indicates the first The quantile probability corresponding to each quantile. Represents the quantile index; the quantile regression value assessment network Output One percentile value: ;in, Indicates candidate actions In continuous time state The corresponding long-term cumulative return distribution is at the quantile probability. The value at that location.

[0028] In one specific implementation, the step of constructing a quantile regression value assessment network to perform quantile assessment on the long-term cumulative return distribution of candidate actions in the continuous-time state, and obtaining a quantile representation of the value distribution, further includes:

[0029] Based on the Bellman update relation in reinforcement learning, a target value distribution is constructed to provide a reference direction for the return distribution of the value evaluation network, so that the network output gradually approaches the true long-term cumulative return distribution.

[0030] In a continuous time frame, a continuous time discount factor is introduced. The target value distribution is represented as:

[0031] ;

[0032] in, Represents continuous time state Next, execute candidate actions The corresponding target value distribution Indicates an immediate reward. Indicates the discount factor. This represents the time interval between state transitions. Indicates the next continuous time state Next action The corresponding value distribution Indicates the next consecutive time state. Indicates the action in the next state;

[0033] The network parameters are updated using a loss function. Let the error between the predicted quantile and the target quantile be . Predicted quantiles refer to the quantile values ​​output. The target quantile is derived from the target value distribution. The corresponding quantile is denoted as The Huber loss function is used to smooth the error; its expression is as follows:

[0034] ;

[0035] in, Indicates the first The target quantile and the first The error between each predicted quantile This represents the smoothing threshold parameter in the Huber loss function;

[0036] The total loss function is obtained by weighting and summing all quantile errors using the quantile Huber loss function:

[0037] ;

[0038] in, Represents the total loss function. Indicates the first The quantile probability corresponding to each target quantile. Represents the characteristic function, when The value is 1 if the condition is met, and 0 otherwise. Indicates error The corresponding Huber loss value;

[0039] Using loss function to evaluate network parameters Iterative updates will be performed.

[0040] In one specific implementation, the step of calculating the cognitive uncertainty corresponding to continuous-time states and candidate actions based on the quantile representation of the value distribution, and constructing enhanced rewards based on the cognitive uncertainty to adjust the reward items in the training of the quantile regression value evaluation network, includes:

[0041] Calculating cognitive uncertainty based on quantile representation The calculation formula is:

[0042] ;

[0043] in, Represents continuous time state With candidate actions The corresponding cognitive uncertainty measure, Indicates the number of quantiles. Indicates the output of the first quantile value. This represents the average of the quantile values;

[0044] Constructing Enhanced Rewards Based on Cognitive Uncertainty The calculation formula is as follows:

[0045] ;

[0046] in, This represents the enhancement reward used for training. This represents a real reward for environmental feedback. Indicates the exploration of hyperparameters;

[0047] Instant rewards will be given during the training phase. Replace with enhanced rewards for building .

[0048] In a specific implementation scheme, the step of using the quantile representation of the value distribution and combining it with preset risk preference parameters to calculate the risk-adjusted value of candidate actions, determining target candidate actions according to the principle of maximizing risk-adjusted value, and generating corresponding advertising placement decisions includes:

[0049] Introducing risk preference parameters Its value range is This is used to characterize advertisers' preference between stable returns and high return potential; based on risk preference parameters... Constructing quantile weight vectors Each weight The first in the corresponding value distribution The quantiles are calculated as follows: ,in, Indicates the first The quantile probability corresponding to each quantile;

[0050] The quantile representations are weighted to obtain the continuous-time state. Next candidate action The risk-adjusted value of is calculated using the following formula:

[0051] ;

[0052] in, Indicates the risk preference parameter Under continuous time state With candidate actions The corresponding risk-adjusted value, Indicates the first Each value distribution quantile;

[0053] After obtaining the risk-adjusted value for each candidate action, the candidate action set is selected according to the principle of maximizing risk-adjusted value. The formula for selecting the target candidate action is as follows:

[0054] ;

[0055] in, Indicates the target candidate action, This represents the set of candidate actions that can be selected at the current moment. Indicates the current decision-making moment;

[0056] Each candidate action in the candidate action set corresponds to a pre-defined set of ad delivery parameters. This is done when the target candidate action is determined. Next, read the target candidate action. The system pre-associates ad delivery parameters and combines these parameters to output the ad delivery decision for the current user at the current decision-making moment.

[0057] Secondly, this application provides an advertising delivery decision generation system based on neural time-series point processes, employing the following technical solution:

[0058] An advertising delivery decision generation system based on neural time-series point processes includes:

[0059] The data acquisition module is used to acquire users' historical interaction data and transform it into a continuous event sequence with absolute timestamps, event types, and event-related features, while calculating the time interval between adjacent events.

[0060] The state calculation module is used to calculate the user's continuous time state based on the continuous event sequence and the time interval, using a pre-built state encoder through event-triggered state updates and state evolution between events;

[0061] The value assessment module is used to construct a quantile regression value assessment network to perform quantile assessment on the long-term cumulative return distribution of candidate actions in the continuous time state, and obtain the quantile representation of the value distribution.

[0062] The reward construction module is used to calculate the cognitive uncertainty corresponding to the continuous time state and the candidate action based on the quantile representation of the value distribution, and to construct an enhanced reward based on the cognitive uncertainty to adjust the reward items in the training of the quantile regression value evaluation network.

[0063] The decision generation module is used to calculate the risk-adjusted value of candidate actions by using the quantile representation of the value distribution and combining it with preset risk preference parameters, determine the target candidate actions according to the principle of maximizing risk-adjusted value, and generate corresponding advertising placement decisions.

[0064] Thirdly, this application provides an electronic device, the device including a processor and a memory; the memory stores a program, the program being loaded and executed by the processor to implement an advertising delivery decision generation method based on neural time-series processes as described in the first aspect.

[0065] Fourthly, this application provides a computer-readable storage medium storing a program that, when executed by a processor, is used to implement an advertising delivery decision generation method based on neural time-series processes as described in the first aspect.

[0066] First, user historical interaction data is acquired and transformed into a continuous event sequence with absolute timestamps, event types, and event-related features. Simultaneously, the time interval between adjacent events is calculated. Based on this, a pre-constructed state encoder jointly calculates the state updates triggered by events and the state evolution between events over time, thus obtaining a continuous time state that reflects the changing patterns of user behavior over time. Next, a quantile regression value evaluation network is constructed to evaluate the long-term cumulative reward distribution of each candidate action under the continuous time state using quantiles. The obtained quantile representation of the value distribution is used to calculate the cognitive uncertainty corresponding to the continuous time state and the candidate action. Enhanced rewards are constructed based on this cognitive uncertainty to adjust the reward items in the value evaluation network training. Finally, the risk-adjusted value of the candidate action is calculated using the quantile representation of the value distribution and a preset risk preference parameter. Target candidate actions are determined according to the principle of maximizing risk-adjusted value, and advertising placement decisions are generated. The above technical solution can depict the dynamic changes of user behavior over continuous time during the advertising process, enabling the timing of advertising to be determined based on changes in user status over real time. At the same time, by evaluating the quantile representation of the long-term return distribution of candidate actions and combining it with cognitive uncertainty and risk preference for comprehensive calculation, the differences in the degree of return volatility of different advertising strategies can be reflected. Thus, a balance can be struck between the accuracy of determining the timing of advertising and the stability of the choice of advertising strategy, thereby achieving accurate determination of advertising strategy.

[0067] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0068] Figure 1 This is a flowchart illustrating the advertising decision generation method based on neural time-series point processes in this application embodiment.

[0069] Figure 2 This is a schematic diagram of the overall process of the advertising placement decision generation method based on neural time-series point processes in the embodiments of this application.

[0070] Figure 3 This is a structural block diagram of the advertising delivery decision generation system based on neural time-series point processes in the embodiments of this application.

[0071] Figure 4This is a block diagram of an electronic device that generates advertising delivery decisions based on neural time-series point processes in an embodiment of this application. Detailed Implementation

[0072] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0073] Optionally, this application uses the advertising delivery decision generation method based on neural time-series process provided in various embodiments as an example for application in an electronic device. The electronic device is a terminal or a server. The terminal can be a computer, tablet computer, etc. This embodiment does not limit the type of electronic device.

[0074] Reference Figure 1 This is a flowchart illustrating an embodiment of an advertising delivery decision generation method based on neural time-series processes provided in this application. The method includes at least the following steps:

[0075] Step S101: Obtain user historical interaction data and convert it into a continuous event sequence with absolute timestamps, event types and event-related features, while calculating the time interval between adjacent events.

[0076] In step S101, the aim is to organize the raw interaction records in the log database into a continuous event sequence that reflects the order, timing, and attributes of user behavior. Raw log data is typically stored as discrete records. Although it contains interaction information such as browsing, ad exposure, clicks, conversions, and application closing, each record only represents scattered behavioral entries, making it difficult to directly describe the continuous evolution of user behavior over time. Therefore, it is necessary to uniformly extract and serialize the user's historical interaction data, enabling various user behaviors occurring at different times to be organized according to their actual occurrence order. Furthermore, the time intervals between adjacent events are calculated, allowing the category information, feature information, and temporal differences between events to be expressed within the same data structure.

[0077] Specifically, the original historical interaction data of the target user is first extracted from the log database, and each interaction record is sorted according to the time of event occurrence. The sorted historical interaction sequence is represented as the user's historical trajectory, which is a continuous sequence of events arranged in chronological order. For any user Its historical trajectory is recorded as ,in, This indicates the total number of events contained in the user's historical trajectory. Indicates the first The consecutive absolute time of the occurrence of this event Indicates the first The event tag type corresponding to this event. Indicates the first The event-related feature vector corresponds to each event. The event tag type distinguishes different interaction categories, including organic browsing, ad impressions, clicks, conversions, and app closures. The event-related feature vector characterizes the contextual information and business attributes of the event, including contextual features, ad creative identifiers, and the price of the interaction. Through these definitions, the original historical interaction data is transformed into a continuous event sequence with absolute timestamps, event types, and event-related features, ensuring that each user interaction is clearly expressed within a unified data structure.

[0078] After obtaining the above continuous event sequence, the time interval between adjacent events is further calculated to enhance the sequence's ability to characterize the patterns of time variation. Specifically, for the first event... This event, and the corresponding time interval, is denoted as . ,in, Indicates the first The second event and the first The time interval between events Indicates the first The consecutive absolute time of the occurrence of this event Indicates the first The continuous absolute time of the occurrence of the event; when season Based on this time interval definition, the original continuous event sequence is further reconstructed into an input form containing information about the time of event occurrence and the event interval. This allows each interactive event to not only indicate what action the user performed, but also the specific time interval between that action and the previous event. Through this step, a continuous event sequence with absolute timestamps, event types, event-related features, and time intervals between adjacent events is obtained.

[0079] Step S102: Based on the continuous event sequence and time interval, use a pre-built state encoder to calculate the user's continuous time state through event-triggered state updates and state evolution between events.

[0080] In step S102, this step aims to calculate the user's state change results in the continuous time domain based on the continuous event sequence obtained in step S101 and the time interval between adjacent events. This characterizes that the user's interest state not only changes instantly when an event occurs, but also continues to evolve over time between adjacent events. Updating the state only when discrete events occur typically only yields discrete states at a few event nodes, making it difficult to reflect the natural decay, recovery, or change process of user interest between two events. Therefore, a pre-built state encoder is used to calculate the continuous event sequence. The pre-built state encoder is a state calculation structure composed of a Long Short-Term Memory (LSTM) network and a continuous-time decay mechanism. It receives the continuous event sequence and the time interval between adjacent events and outputs the user's state results in continuous time. The LSTM network updates the previous state based on the event type, event-related features, and time interval information when an event occurs, obtaining the state result corresponding to that event. The continuous-time decay mechanism calculates the continuous change process of the state over time between adjacent events based on the state result corresponding to that event, obtaining the continuous-time state at any given moment. Based on this processing method, the user state no longer corresponds to only a few discrete event nodes, but can be represented as a state function in continuous time.

[0081] Furthermore, after obtaining the continuous-time state, the conditional intensity of specific types of events over continuous time is calculated based on the continuous-time state to establish a correspondence between user state and event occurrence intensity. Since this conditional intensity is used to characterize the occurrence pattern of events over continuous time, the state encoding process forms a user behavior modeling method under the neural temporal point process through the joint calculation of continuous-time state and conditional intensity.

[0082] Specifically, the continuous event sequence and time interval obtained in step S101 are first input into the state encoder. When the... This event occurred at the time When this occurs, the user's hidden state is updated using a Long Short-Term Memory (LSTM) network. The update formula is as follows:

[0083] ;

[0084] in, Indicates the first The hidden state after the event occurs. Indicates the first The hidden state corresponding to this event. Indicates event type The corresponding embedding vector, specifically the embedding vector, is a feature vector obtained by vectorizing discrete event types. It is used to map different event types into numerical representations that can participate in neural network calculations. Indicates the first The event tag type corresponding to this event. Indicates the first The event-related feature vector corresponding to this event. Indicates the first The second event and the first The time interval between events This represents a vector concatenation operation. Indicates the time interval The result after logarithmic transformation. Through the above calculations, event type information, event-related feature information, and time interval information are all incorporated into the state update process to obtain the hidden state after the event is triggered. Upon obtaining the hidden state Then, the state changes between adjacent events are continuously calculated. Under any condition that... The moment Continuous time state Calculate according to the following formula:

[0085] ;

[0086] in, Indicates time The corresponding continuous time state, This represents the progressively decaying target vector, used to characterize the baseline activity under conditions of no external stimulus. No external stimulus means that no new event input is received after the current event occurs. This represents a learnable decay rate parameter used to control the continuous-time state. To the state decay target vector The speed of approach, and These are all model parameters in the state encoder, which are determined by iteratively optimizing the model parameters when training the state encoder using historical user interaction data. Indicates the first At the time of this incident Indicates satisfaction At any time, This represents exponential operations. Through the state updates triggered by the aforementioned events and the state evolution between events, the user's state representation over continuous time can be obtained. .

[0087] Obtaining the continuous-time state Subsequently, based on this continuous time state, the specific type of event at time [time] is further calculated. The conditional intensity function is calculated using the following formula:

[0088] ;

[0089] in, Indicates time Next event type The conditional strength function value, Indicates the event type flag, Indicates event type The corresponding weight vector, Represents the weight vector transpose, Indicates time The corresponding continuous time state, Indicates event type The corresponding bias term, The Softplus activation function is represented by the following expression:

[0090] ;

[0091] in, This represents the input variables of the activation function. It is used to obtain the continuous-time states. Subsequently, a conditional strength function for a specific type of event is calculated based on the continuous-time state, which is used to characterize different types of events at a given user state at time intervals. The instantaneous intensity of the event. This conditional intensity function represents the continuous-time state. By establishing a correspondence between user states and the probability of events occurring over continuous time, user states not only represent changes in user interests over continuous time but can also be used to describe the patterns of different behavioral events occurring over continuous time. By introducing conditional strength functions, the state encoding process no longer merely provides state representations but further establishes the relationship between states and event occurrence mechanisms, thus forming a user behavior modeling structure that conforms to the neural time-series process modeling approach.

[0092] Step S103: Construct a quantile regression value assessment network to perform quantile assessment on the long-term cumulative return distribution of candidate actions in continuous time states, and obtain the quantile representation of the value distribution.

[0093] In step S103, this step aims to evaluate the long-term cumulative return distribution that a candidate action may generate under the continuous-time state obtained in step S102. Traditional value assessment methods typically only output a single expected return value, while long-term cumulative returns often have significant uncertainty in actual advertising campaigns. Using only a single value is insufficient to reflect the differences in return distributions corresponding to different candidate actions. Therefore, this step constructs a quantile regression value assessment network to perform quantile assessment of the long-term cumulative return distribution, enabling the value assessment result to represent the return distribution in the form of multiple quantile values, thus obtaining a more complete assessment result than a single value.

[0094] Specifically, the continuous time state obtained in step S102 and candidate actions As input, a quantile regression value assessment network is constructed. ,in, This represents the network parameters. The quantile regression value assessment network adopts a feedforward neural network structure consisting of an input layer, hidden layers, and an output layer, where the input layer is used to receive continuous-time states. With candidate actions The network combines features, with hidden layers performing a non-linear mapping between state and action features, and an output layer outputting multiple quantile values ​​corresponding to the long-term cumulative reward distribution. This network structure establishes a mapping relationship between continuous-time states and candidate actions, thereby obtaining the corresponding long-term cumulative reward distribution representation.

[0095] To provide a discrete representation of the long-term cumulative return distribution, the number of quantiles is set to be... and set the first The quantile probability corresponding to each quantile is:

[0096] ;

[0097] in, Indicates the number of quantiles. Indicates the first The quantile probability corresponding to each quantile. This represents the quantile index. Based on the above quantile settings, the quantile regression value assessment network... Output One percentile value: ;in, Indicates candidate actions In continuous time state The corresponding long-term cumulative return distribution is at the quantile probability. The value at that position can also be written as... In the form of. Through The quantile values ​​can be used to represent the long-term cumulative return distribution corresponding to candidate actions. It is represented as a discrete set of quantiles, thereby enabling quantile assessment of the value distribution.

[0098] It should be noted that the quantile values ​​output by the quantile regression value assessment network in the initial state are... Predictions based solely on randomly initialized network parameters may not accurately reflect the true long-term cumulative reward distribution. Therefore, it is necessary to construct a reference distribution that reflects the changing trend of the true reward distribution, allowing the quantile values ​​of the network output to gradually approximate the true reward distribution during training. To this end, a target value distribution is constructed based on the Bellman update relation in reinforcement learning, ensuring that the value distribution estimation satisfies the recursive relation of long-term rewards. In a continuous-time framework, a continuous-time discount factor is introduced. The target value distribution is represented as:

[0099] ;

[0100] in, Represents continuous time state Next, execute candidate actions The corresponding target value distribution Indicates an immediate reward. Indicates the discount factor. This represents the time interval between state transitions. Indicates the next continuous time state Next action The corresponding value distribution Indicates the next consecutive time state. This represents the action in the next state. By constructing a target value distribution, a reference direction for the return distribution can be provided for the value assessment network, allowing the network output to gradually approximate the true long-term cumulative return distribution.

[0101] To ensure that the quantile results output by the network gradually approximate the target value distribution, it is necessary to measure the error between the network output and the target value distribution, i.e., the difference between the predicted quantile and the target quantile. Here, the predicted quantile refers to the quantile regression value assessment network... Input continuous time state and candidate actions The quantile values ​​output later The quantile is used to represent the value distribution predicted by the current network; the target quantile is derived from the target value distribution constructed based on the Bellman update relation. The corresponding quantile is denoted as By measuring the error between the predicted quantiles and the target quantiles, we can characterize the deviation between the current network-predicted value distribution and the target value distribution. This provides a basis for updating network parameters using the loss function, allowing the network's output quantile results to gradually approach the true long-term cumulative return distribution.

[0102] The network parameters are then updated using a loss function. Let the error between the predicted quantile and the target quantile be... The Huber loss function is used to smooth the error; its expression is as follows:

[0103] ;

[0104] in, Indicates the first The target quantile and the first The error between each predicted quantile This represents the smoothing threshold parameter in the Huber loss function. Further, the total loss function is obtained by weighted summation of all quantile errors using the quantile Huber loss function:

[0105] ;

[0106] in, Represents the total loss function. Indicates the first The quantile probability corresponding to each target quantile. Represents the characteristic function, when The value is 1 if the condition is met, and 0 otherwise. Indicates error The corresponding Huber loss value. This is achieved by applying the aforementioned loss function to the network parameters. Iterative updates are performed to gradually bring the predicted quantiles closer to the target value distribution, thereby achieving an accurate estimate of the long-term cumulative return distribution.

[0107] Through the above processing, the continuous-time state can be obtained. Distribution of long-term cumulative returns for each candidate action And using quantile sets Represent the value distribution.

[0108] Step S104: Calculate the cognitive uncertainty corresponding to continuous time states and candidate actions based on the quantile representation of value distribution, and construct an enhanced reward based on the cognitive uncertainty to adjust the reward items in the training of the quantile regression value evaluation network.

[0109] In step S104, the aim is to utilize the quantile representation of the value distribution obtained in step S103 to further measure the cognitive uncertainty of the quantile regression value evaluation network regarding the combination of continuous-time states and candidate actions, and to transform this cognitive uncertainty into an enhanced reward used during training, thereby adjusting the reward terms used during the training of the value evaluation network. Step S103 has already output the continuous-time states through the quantile regression value evaluation network. With candidate actions The corresponding value distribution quantiles represent the results. These quantile values ​​represent the values ​​of the long-term cumulative return distribution at different quantile probabilities. Since this set of quantiles reflects the dispersion of the value distribution, its statistical properties can be directly used to characterize the degree of cognitive sufficiency of the value assessment network regarding the combination of state and action. Therefore, based on the output of step S103, this step calculates the dispersion of the quantile set to obtain a measure of cognitive uncertainty, and incorporates this measure into the reward term used during the training of the value assessment network.

[0110] Specifically, for continuous time states With candidate actions Step S103 has yielded the corresponding value distribution quantile representation. Cognitive uncertainty measure is calculated based on this quantile set. The calculation formula is:

[0111] ;

[0112] in, Represents continuous time state With candidate actions The corresponding cognitive uncertainty measure, Indicates the number of quantiles. This indicates the output of step S103. quantile value. The average of the quantile values ​​is expressed by the following formula:

[0113] ;

[0114] in, The quantiles represent the mean of the results. Through the above calculations, the variance of the quantile set obtained in step S103 can be used to characterize the network's cognitive uncertainty regarding the combination of states and actions. When the differences between quantile values ​​are large, it indicates that the value distribution is relatively dispersed, and the network's estimation of the long-term cumulative return of this state-action combination is not yet stable; when the differences between quantile values ​​are small, it indicates that the value distribution is relatively concentrated, and the network's estimation of this state-action combination is relatively stable.

[0115] In obtaining a measure of cognitive uncertainty Subsequently, an enhanced reward system was constructed based on this cognitive uncertainty. The calculation formula is as follows:

[0116] ;

[0117] in, This represents the enhancement reward used for training. This represents the actual reward for environmental feedback, specifically click-through rate or conversion rate. This indicates the exploration of hyperparameters used to control the strength of the impact of cognitive uncertainty on training rewards. This represents the square root form of the cognitive uncertainty measure. Step S103 uses immediate rewards when constructing the target value distribution. Instant rewards will be given during the training phase. Replace with the enhanced rewards constructed in this step. This process ensures that the reward term in the target value distribution simultaneously incorporates the real benefits from environmental feedback and the adjustment introduced by cognitive uncertainty. After this treatment, the target value distribution in step S103 is no longer solely determined by environmental rewards, but is constructed with the participation of enhanced rewards, thus affecting the loss calculation between the predicted quantile and the target quantile. Therefore, the target value distribution upon which the quantile regression value assessment network relies during training will be simultaneously influenced by both real benefits and cognitive uncertainty.

[0118] Step S105: Calculate the risk-adjusted value of candidate actions using the quantile representation of the value distribution and in combination with preset risk preference parameters. Determine the target candidate actions according to the principle of maximizing risk-adjusted value and generate the corresponding advertising placement decision.

[0119] In step S105, this step aims to utilize the value distribution quantile representation obtained in step S103 to perform risk-adjusted calculations on the long-term returns of candidate actions, taking into account the advertiser's risk preferences. The step with the optimal risk-adjusted value is then selected from the candidate action set as the final action to be executed, thereby generating the corresponding advertising placement decision. Step S103 has already addressed the continuous-time state. With candidate actions Output the quantile representation of the value distribution. This quantile set characterizes the probability distribution structure of long-term cumulative returns when performing different candidate actions in the current user state. However, different advertisers often have different risk preferences during the campaign process; for example, some advertisers focus more on revenue stability, while others focus more on potentially high-return outcomes. Therefore, when generating actual ad placement decisions, it is necessary to adjust and calculate the value distribution based on preset risk preferences to obtain action value evaluation results that align with the advertiser's placement strategy.

[0120] Specifically, the risk preference parameter is first introduced. Its value range is This is used to characterize the degree of an advertiser's preference between revenue stability and high revenue potential. When this indicates a risk-averse preference, more attention is paid to the results corresponding to lower quantiles in the value distribution; when... This indicates a risk-seeking preference, where the focus in the value distribution is on outcomes corresponding to higher quantiles. This is based on risk preference parameters. Constructing quantile weight vectors Each weight The first in the corresponding value distribution The quantile is calculated as follows: ,in, Indicates the first The quantile probabilities corresponding to each quantile are then normalized for all weights, so that... This weighting method allows different levels of importance to be assigned to different quantiles in the value distribution based on risk preference parameters.

[0121] After obtaining the quantile weights, the quantile representation of the value distribution obtained in step S103 is weighted and calculated to obtain the continuous-time state. Next candidate action The risk-adjusted value of is calculated using the following formula:

[0122] ;

[0123] in, Indicates the risk preference parameter Under continuous time state With candidate actions The corresponding risk-adjusted value, Indicates the first Risk weights corresponding to each quantile This indicates that the first step obtained in step S103 is... Each value distribution quantile. Through the above calculation, the long-term cumulative return, originally expressed as a probability distribution, can be transformed into an action value evaluation result that takes into account risk preference.

[0124] After obtaining the risk-adjusted value for each candidate action, the candidate action set is selected according to the principle of maximizing risk-adjusted value. The formula for selecting the target candidate action is as follows:

[0125] ;

[0126] in, Indicates the target candidate action, This represents the set of candidate actions that can be selected at the current moment. This indicates the current decision-making moment. It's important to note that each candidate action in the candidate action set corresponds to a pre-defined set of advertising delivery parameters, which include at least one or more of the following: target audience, delivery timing, ad creative, and delivery intensity. Therefore, when determining the target candidate action... Next, read the target candidate action. The system pre-associates ad delivery parameters and combines these parameters to output the ad delivery decision for the current user at the current decision-making moment. In other words, the ad delivery decision generation process involves: first, determining target candidate actions based on the principle of maximizing risk-adjusted value; then, extracting ad delivery parameters from the pre-defined parameter mapping relationship corresponding to the target candidate actions, thereby forming a directly executable ad delivery decision.

[0127] Furthermore, preferably, by adjusting the risk preference parameter This can change the weight of different quantiles in the value distribution in the calculation of risk-adjusted value, thereby adjusting the focus of the return distribution. When When the value is small, the risk weight is more concentrated in the lower quantiles, making the decision focus more on the stability of returns; when When the risk preference parameter is large, the risk weights are concentrated in the high quantiles, making the decision focus more on potentially high-return outcomes. Adjustments can be made to balance between revenue stability and high revenue potential, thereby mathematically satisfying advertisers' different preferences for ROI targets under budget constraints.

[0128] In summary, combining Figure 2First, historical user interaction data is acquired, and discrete log records are organized into continuous event sequences containing absolute time information, event types, and event-related features. Simultaneously, the time intervals between adjacent events are calculated, forming a user behavior trajectory that reflects the sequence and temporal relationships of user actions. Based on this, a state encoding structure composed of a Long Short-Term Memory network and a continuous-time state evolution mechanism is used to jointly calculate state changes at the time of an event and state changes between adjacent events over time, thus obtaining a representation of the user's state in the continuous-time domain. Subsequently, the long-term returns of different candidate advertising actions are valued based on this continuous-time state, and the long-term cumulative return distribution is represented using quantiles, allowing the distribution characteristics of the advertising strategy's returns to be characterized. Simultaneously, during the training phase, the statistical properties of the value distribution are used to calculate cognitive uncertainty, and the training rewards are adjusted accordingly to improve the reliability of the value assessment results. Finally, in the actual delivery decision-making phase, the quantile representation of the value distribution is used, combined with risk preference, to perform risk-adjusted calculations on the values ​​of different candidate actions. The target action with the optimal risk-adjusted value is determined from the candidate action set, and an advertising delivery decision is generated based on the delivery parameters corresponding to this target action.

[0129] Through the aforementioned technical solution, this application first models user behavior using continuous event sequences and time interval information, enabling user states to dynamically evolve over time in a continuous time domain. This allows for a more accurate depiction of changes in user interest states, and on this basis, determines more reasonable advertising timing. Simultaneously, by modeling the probability distribution of long-term returns of advertising strategies, rather than relying solely on a single expected return for evaluation, the differences in return volatility among different advertising strategies can be identified and utilized. Furthermore, the value distribution is adjusted in conjunction with risk preference, thereby achieving a balance between return stability and potential returns. Therefore, this application can comprehensively evaluate the long-term returns and volatility characteristics of advertising strategies based on depicting continuous changes in user behavior, thus achieving a more accurate and stable determination of advertising placement strategies.

[0130] Figure 3 This is a structural block diagram of an advertising delivery decision generation system based on neural time-series point processes provided in one embodiment of this application. The system includes at least the following modules:

[0131] The data acquisition module is used to acquire users' historical interaction data and transform it into a continuous event sequence with absolute timestamps, event types, and event-related features, while calculating the time interval between adjacent events.

[0132] The state calculation module is used to calculate the user's continuous time state based on a continuous event sequence and time interval, using a pre-built state encoder through event-triggered state updates and state evolution between events.

[0133] The value assessment module is used to construct a quantile regression value assessment network to perform quantile assessment on the long-term cumulative return distribution of candidate actions in a continuous time state, and obtain the quantile representation of the value distribution.

[0134] The reward construction module is used to calculate the cognitive uncertainty corresponding to continuous time states and candidate actions based on the quantile representation of the value distribution, and to construct enhanced rewards based on the cognitive uncertainty to adjust the reward items in the training of the quantile regression value evaluation network.

[0135] The decision generation module is used to calculate the risk-adjusted value of candidate actions by using the quantile representation of the value distribution and combining it with preset risk preference parameters. It then determines the target candidate actions according to the principle of maximizing risk-adjusted value and generates the corresponding advertising placement decision.

[0136] For relevant details, please refer to the above method implementation examples.

[0137] Figure 4 This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 401 and a memory 402.

[0138] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0139] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 402 are used to store at least one instruction, which is executed by the processor 401 to implement the advertising delivery decision generation method based on neural time-series processes provided in the method embodiments of this application.

[0140] In some embodiments, the electronic device may also optionally include: a peripheral device interface and at least one peripheral device. The processor 401, memory 402, and peripheral device interface can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface via a bus, signal line, or circuit board. Indicatively, peripheral devices include, but are not limited to: radio frequency circuits, touch displays, audio circuits, and power supplies.

[0141] Of course, electronic devices may also include fewer or more components, and this embodiment does not limit this.

[0142] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the advertising delivery decision generation method based on neural time-series process described in the above method embodiments.

[0143] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program, which is loaded and executed by a processor to implement the advertising delivery decision generation method based on neural time-series process described in the above method embodiments.

[0144] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for generating advertising placement decisions based on neural time-series point processes, characterized in that, The method includes: Acquire user historical interaction data and transform it into a continuous event sequence with absolute timestamps, event types, and event-related features, while calculating the time interval between adjacent events; Based on the continuous event sequence and the time interval, a pre-built state encoder is used to calculate the user's continuous time state through event-triggered state updates and state evolution between events, including: The state encoder is a state calculation structure composed of a long short-term memory network and a continuous time decay mechanism, used to receive the continuous event sequence and the time interval between adjacent events, and output the user's state result in continuous time. When the This event occurred at the time When this occurs, the user's hidden state is updated using a Long Short-Term Memory (LSTM) network. The update formula is as follows: ； in, Indicates the first The hidden state after the event occurs. Indicates the first The hidden state corresponding to this event. Indicates event type The corresponding embedding vector, This represents a vector concatenation operation. Indicates the first The second event and the first The time interval between events Indicates the first The consecutive absolute time of the occurrence of this event; Obtain the hidden state Then, the state changes between adjacent events are continuously calculated; under any condition that... The moment Continuous time state Calculate according to the following formula: ； in, Indicates time The corresponding continuous time state, Represents the gradually decaying target vector; This represents the learnable decay rate parameter. Indicates satisfaction At any time, Indicates exponentiation; Obtaining the continuous-time state Subsequently, based on this continuous time state, the specific type of event at time [time] is further calculated. The conditional intensity function is calculated using the following formula: ； in, Indicates time Next event type The conditional strength function value, Indicates the event type flag, Indicates event type The corresponding weight vector, Represents the weight vector transpose, Indicates time The corresponding continuous time state, Indicates event type The corresponding bias term, This represents the Softplus activation function; the conditional strength function is used to characterize the event type in a given user state. At any moment The instantaneous intensity of the event, and the establishment of a continuous time state. The correspondence between the probability of an event occurring over a continuous period of time and the probability of its occurrence. A quantile regression value assessment network is constructed to perform quantile assessment on the long-term cumulative return distribution of candidate actions in the continuous time state, thereby obtaining the quantile representation of the value distribution. The cognitive uncertainty corresponding to the continuous-time state and the candidate action is calculated based on the quantile representation of the value distribution, and an enhanced reward is constructed based on the cognitive uncertainty to adjust the reward item in the training of the quantile regression value evaluation network. The risk-adjusted value of candidate actions is calculated using the quantile representation of the value distribution and in combination with preset risk preference parameters. Target candidate actions are determined according to the principle of maximizing risk-adjusted value, and corresponding advertising placement decisions are generated.

2. The advertising placement decision generation method based on neural time-series point processes according to claim 1, characterized in that, The process of acquiring user historical interaction data and converting it into a continuous event sequence with absolute timestamps, event types, and event-related features, while simultaneously calculating the time interval between adjacent events, includes: Extract the target user's original historical interaction data from the log database, sort each interaction record according to the time of the event, and represent the sorted historical interaction sequence as the user's historical trajectory. For any user Its historical trajectory is recorded as ,in, This indicates the total number of events contained in the user's historical trajectory. Indicates the first The consecutive absolute time of the occurrence of this event Indicates the first The event tag type corresponding to this event. Indicates the first The event-related feature vector corresponding to each event; the user's historical trajectory is a continuous sequence of events arranged in chronological order of their occurrence. For the This event, and the corresponding time interval, is denoted as . ,in, Indicates the first The second event and the first The time interval between events Indicates the first The consecutive absolute time of the occurrence of this event Indicates the first The continuous absolute time of the occurrence of the event; when season .

3. The advertising placement decision generation method based on neural time-series point processes according to claim 1, characterized in that, The construction of the quantile regression value assessment network, which performs quantile assessment on the long-term cumulative return distribution of candidate actions in the continuous-time state, yields a quantile representation of the value distribution, including: Constructing a quantile regression value assessment network , Representing network parameters in continuous-time states and candidate actions As input, the number of quantiles is set to... and set the first The quantile probability corresponding to each quantile is: ； in, Indicates the number of quantiles. Indicates the first The quantile probability corresponding to each quantile. This represents the quantile index; the quantile regression value assessment network Output One percentile value: ;in, Indicates candidate actions In continuous time state The corresponding long-term cumulative return distribution is at the quantile probability. The value at that location.

4. The advertising placement decision generation method based on neural time-series point processes according to claim 3, characterized in that, The construction of the quantile regression value assessment network, which performs quantile assessment on the long-term cumulative return distribution of candidate actions in the continuous-time state to obtain the quantile representation of the value distribution, further includes: Based on the Bellman update relation in reinforcement learning, a target value distribution is constructed to provide a reference direction for the return distribution of the value evaluation network, so that the network output gradually approaches the true long-term cumulative return distribution. In a continuous time frame, a continuous time discount factor is introduced. The target value distribution is represented as: ； in, Represents continuous time state Next, execute candidate actions The corresponding target value distribution Indicates an immediate reward. Indicates the discount factor. This represents the time interval between state transitions. Indicates the next continuous time state Next action The corresponding value distribution Indicates the next consecutive time state. Indicates the action in the next state; The network parameters are updated using a loss function. Let the error between the predicted quantile and the target quantile be . Predicted quantiles refer to the quantile values ​​output. The target quantile is derived from the target value distribution. The corresponding quantile is denoted as The Huber loss function is used to smooth the error; its expression is as follows: ； in, Indicates the first The target quantile and the first The error between each predicted quantile This represents the smoothing threshold parameter in the Huber loss function; The total loss function is obtained by weighting and summing all quantile errors using the quantile Huber loss function: ； in, Represents the total loss function. Indicates the first The quantile probability corresponding to each target quantile. Represents the characteristic function, when The value is 1 if the condition is met, and 0 otherwise. Indicates error The corresponding Huber loss value; Using loss function to evaluate network parameters Iterative updates will be performed.

5. The advertising placement decision generation method based on neural time-series point processes according to claim 4, characterized in that, The quantile representation based on the value distribution is used to calculate the cognitive uncertainty corresponding to continuous-time states and candidate actions, and to construct enhanced rewards based on the cognitive uncertainty to adjust the reward items in the training of the quantile regression value evaluation network, including: Calculating cognitive uncertainty based on quantile representation The calculation formula is: ； in, Represents continuous time state With candidate actions The corresponding cognitive uncertainty measure, Indicates the number of quantiles. Indicates the output of the first quantile value. This represents the average of the quantile values; Constructing Enhanced Rewards Based on Cognitive Uncertainty The calculation formula is as follows: ； in, This represents the enhancement reward used for training. This represents a real reward for environmental feedback. Indicates the exploration of hyperparameters; Instant rewards will be given during the training phase. Replace with enhanced rewards for building .

6. The advertising placement decision generation method based on neural time-series point processes according to claim 3, characterized in that, The process of using the quantile representation of the value distribution and combining it with preset risk preference parameters to calculate the risk-adjusted value of candidate actions, determining target candidate actions according to the principle of maximizing risk-adjusted value, and generating corresponding advertising placement decisions includes: Introducing risk preference parameters Its value range is This is used to characterize advertisers' preference between stable returns and high return potential; based on risk preference parameters... Constructing quantile weight vectors Each weight The first in the corresponding value distribution The quantiles are calculated as follows: ,in, Indicates the first The quantile probability corresponding to each quantile; The quantile representations are weighted to obtain the continuous-time state. Next candidate action The risk-adjusted value of is calculated using the following formula: ； in, Indicates the risk preference parameter Below, continuous time state With candidate actions The corresponding risk-adjusted value, Indicates the first Each value distribution quantile; After obtaining the risk-adjusted value for each candidate action, the candidate action set is selected according to the principle of maximizing risk-adjusted value. The target candidate action is selected from the following, and its calculation formula is as follows: ； in, Indicates the target candidate action, This represents the set of candidate actions that can be selected at the current moment. Indicates the current decision-making moment; Each candidate action in the candidate action set corresponds to a pre-defined set of ad delivery parameters. This is done when the target candidate action is determined. Next, read the target candidate action. The system pre-associates ad delivery parameters and combines these parameters to output the ad delivery decision for the current user at the current decision-making moment.

7. An advertising placement decision generation system based on neural time-series point processes, characterized in that, include: The data acquisition module is used to acquire users' historical interaction data and transform it into a continuous event sequence with absolute timestamps, event types, and event-related features, while calculating the time interval between adjacent events; The state calculation module is used to calculate the user's continuous time state based on the continuous event sequence and the time interval using a pre-built state encoder through event-triggered state updates and state evolution between events. The state encoder is a state calculation structure composed of a long short-term memory network and a continuous time decay mechanism, used to receive the continuous event sequence and the time interval between adjacent events, and output the user's state result in continuous time. When the This event occurred at the time When this occurs, the user's hidden state is updated using a Long Short-Term Memory (LSTM) network. The update formula is as follows: ； in, Indicates the first The hidden state after the event occurs. Indicates the first The hidden state corresponding to this event. Indicates event type The corresponding embedding vector, This represents a vector concatenation operation. Indicates the first The second event and the first The time interval between events Indicates the first The consecutive absolute time of the occurrence of this event; Obtain the hidden state Then, the state changes between adjacent events are continuously calculated; under any condition that... The moment Continuous time state Calculate according to the following formula: ； in, Indicates time The corresponding continuous time state, Represents the gradually decaying target vector; This represents the learnable decay rate parameter. Indicates satisfaction At any time, Indicates exponentiation; Obtaining the continuous-time state Subsequently, based on this continuous time state, the specific type of event at time [time] is further calculated. The conditional intensity function is calculated using the following formula: ； in, Indicates time Next event type The conditional strength function value, Indicates the event type flag, Indicates event type The corresponding weight vector, Represents the weight vector transpose, Indicates time The corresponding continuous time state, Indicates event type The corresponding bias term, This represents the Softplus activation function; the conditional strength function is used to characterize the event type in a given user state. At any moment The instantaneous intensity of the event, and the establishment of a continuous time state. The correspondence between the probability of an event occurring over a continuous period of time and the probability of its occurrence. The value assessment module is used to construct a quantile regression value assessment network to perform quantile assessment on the long-term cumulative return distribution of candidate actions in the continuous time state, and obtain the quantile representation of the value distribution. The reward construction module is used to calculate the cognitive uncertainty corresponding to the continuous time state and the candidate action based on the quantile representation of the value distribution, and to construct an enhanced reward based on the cognitive uncertainty to adjust the reward items in the training of the quantile regression value evaluation network. The decision generation module is used to calculate the risk-adjusted value of candidate actions by using the quantile representation of the value distribution and combining it with preset risk preference parameters, determine the target candidate actions according to the principle of maximizing risk-adjusted value, and generate corresponding advertising placement decisions.

8. An electronic device, characterized in that, The device includes a processor and a memory; the memory stores a program, which is loaded and executed by the processor to implement an advertising delivery decision generation method based on neural time-series processes as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when executed by a processor, is used to implement an advertising delivery decision generation method based on neural time-series processes as described in any one of claims 1 to 6.

Images