A method for constructing a behavior feature mapping simulation agent

Abstract

The application relates to the field of artificial intelligence and simulation testing technology, and discloses a behavior feature mapping simulation agent construction method, which cuts real entity behavior logs into paired behavior sequences and context sequences according to conversations, encodes the context conditionally, applies context invariance confrontation constraints and same body consistency constraints, extracts internal behavior style representations decoupled from the context, models the distribution density of the internal behavior style representations by using reversible normalization flow, rejects sampling in a low-density support domain according to a density band to obtain controllable style encoding, jointly trains a style conditional strategy by using a time sequence strategy network with multilayer feature linear modulation and a double-layer hidden variable and an encoder, rolls in a closed-loop environment in which a context transfer model acts as a loop and applies style cycle consistency and dynamic support determination, fine-tunes and configures according to a target style encoding, and finally outputs a simulation agent. The application can construct a simulation agent with self-consistent style, stable reaction and controllable generation of rare and reasonable behaviors.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and simulation testing technology, and more specifically, to a method for constructing a simulated intelligent agent based on behavioral feature mapping. Background Technology

[0002] Simulated agents are used to reproduce the behavioral patterns of real-world entities in controlled simulation testing environments, serving system testing, synthetic behavioral data generation, and policy pre-playing. They are also widely used in the verification of recommendation, interaction, and process-oriented systems. The construction methods in related technologies mainly include rule-based and parameterized model-based approaches, behavior playback-based approaches, and data-driven approaches based on imitation learning and generative models. Their common goal is to map the behavioral characteristics of real-world entities into the decision-making mechanisms of intelligent agents.

[0003] However, the above methods have technical problems in practical applications. Parameterization and rule-based methods tend to homogeneous group behavior due to fixed parameters, making it difficult to reflect individual differences; behavior replay methods are not reactive in interactive environments, and become disconnected from the environment once the environmental state deviates from the collection conditions; imitation learning and generative model methods face the problems of group behavior pattern collapse, impure style representation due to the coupling of behavioral style and context, and style drift caused by the accumulation of errors in closed-loop rolling. These technical problems make it difficult to balance individual realism and group diversity in the constructed simulated intelligent agent.

[0004] Furthermore, simulation testing requires rare but reasonable long-tail behaviors, but related technologies can either only reproduce common behaviors or create distorted behaviors that deviate from the real feasible domain through perturbation. It is difficult to controllably generate rare behaviors without distortion. The above technical problems limit the performance of simulated intelligent agents in terms of realism, behavioral diversity and controllability. Summary of the Invention

[0005] This invention provides a method for constructing a simulated intelligent agent by mapping behavioral features, which solves the technical problems in related technologies, such as impure style representation caused by the coupling of behavioral style with context, homogenization of group behavior of simulated intelligent agents, difficulty in controlling the generation of rare but reasonable behaviors, and style drift in closed-loop interaction.

[0006] This invention provides a method for constructing a simulated intelligent agent based on behavioral feature mapping, comprising the following steps: S1, obtain entity behavior logs, segment them by session, and split and align each time step into behavior side and context side to obtain paired individual behavior sequence and context sequence samples; S2, pairs of individual behavior sequences and situational sequence samples are fed into a situational conditional encoder. Situational invariance adversarial constraints and subject consistency constraints are applied during the encoding process to extract intrinsic behavioral style representations that are decoupled from the situation. S3 models the distribution density of intrinsic behavioral style representations decoupled from context using reversible normalized flow, and rejects sampling in low-density regions that are still within the support domain according to density bands, thus obtaining controllable style codes that cover common and rare behaviors. S4, with controllable style coding as condition and context sequence as input, uses a temporal policy network with multi-layer feature linear modulation and persistent style and instantaneous intention dual-layer latent variables, and is jointly trained with context-conditional encoder to obtain style-conditional policy; S5 allows the style-conditioning strategy to roll in the closed-loop environment served by the context-transfer model, applies style cycle consistency constraints, and applies dynamic support judgment to the re-encoded style representation obtained by the context-conditioning encoder of the closed-loop behavior, and fine-tunes to obtain the simulated agent. S6. Obtain the target behavior profile, map it to the target style code along the style space behavior tendency direction, configure the simulated agent accordingly, and output the simulated agent that can be configured as needed and passes the verification after multi-index consistency verification.

[0007] Preferably, the interval threshold used for session segmentation in S1 is obtained by statistically analyzing the distribution of time differences between adjacent events in the log. The information at each time step is split into a behavior side and a context side. The behavior side retains the actions initiated by the entity itself, while the context side retains the environmental state at the same time step that is not directly controlled by the entity's current action. The behavior side and the context side are aligned at the time step and stored in pairs, and the samples stored in pairs are accompanied by entity identifiers. The splitting of the behavior side and the context side allows the influence of the context to be deducted from the behavior in subsequent encoding.

[0008] Preferably, the situation invariance adversarial constraint in S2 is applied as follows: after the intrinsic behavioral style representation decoupled from the situation, a situation discriminator is connected, which is then used to predict the situation summary from the intrinsic behavioral style representation. A gradient inversion layer is inserted between the intrinsic behavioral style representation and the situation discriminator, which minimizes the prediction error. The situation conditional encoder is pushed in a direction that makes the situation discriminator unable to predict the situation summary after gradient inversion. The scenario summary is derived from the concatenation of first-order and second-order statistics of the scenario sequence in the time dimension. The adversarial constraints are trained in an alternating manner. First, the scenario conditional encoder is fixed to update the scenario discriminator, and then the scenario discriminator is fixed to update the scenario conditional encoder.

[0009] Preferably, the consistency constraint of the same entity in S2 is applied by using the intrinsic behavioral style representations of different sessions of the same entity as positive sample pairs; The intrinsic behavioral style representations of different entities are used as negative sample pairs. The information noise is used to compare the target to bring the positive sample pairs closer and push the negative sample pairs further away. Cosine similarity is used to measure the closeness between intrinsic behavioral style representations, and temperature coefficient is used to adjust the sharpness of the distinction.

[0010] Preferably, the reversible normalized flow in S3 is composed of several cascaded affine coupling layers. Each affine coupling layer divides the input dimension into two groups. One group remains unchanged, and the other group is affinely transformed according to the scaling and translation calculated by the unchanged group. The dimensions are permuted between adjacent affine coupling layers. The reversible normalized flow is trained with the goal of maximizing the log density of intrinsic behavioral style representations in the training set. The log density of any intrinsic behavioral style representation is equal to the sum of its log density after being mapped to the base distribution by the reversible normalized flow and the absolute value of the logarithm of the Jacobian determinant of the mapping.

[0011] Preferably, the method of rejecting sampling by density band in S3 is as follows: sample candidate points on the base distribution, use the invertibility of the invertible normalized flow to inversely map the candidate points back to the style space to obtain candidate styles, and calculate the logarithmic occurrence density of the candidate styles; Candidate styles are accepted when the log density falls within the density band defined by the lower and upper density thresholds; otherwise, they are discarded and resampled. The lower and upper density thresholds are taken from the quantiles of the log density empirical distribution representing the intrinsic behavioral style of the training set. When it is necessary to bias towards rare behaviors, the candidate points on the base distribution are biased towards values ​​with a medium radius.

[0012] Preferably, the multi-layer feature linear modulation method in S4 is as follows: a set of channel-wise scaling coefficients and translation coefficients are generated by controllable style coding at multiple layers of the temporal policy network, and the features of the corresponding layers are first scaled by channel and then translated by channel. The scaling and translation coefficients are calculated from the controllable style code by a mapping network, and a set is output for each modulated level, so that the controllable style code participates in the determination of local action tendencies in the shallow layer of the temporal policy network and participates in the determination of the overall decision orientation in the deep layer.

[0013] Preferably, the setting method of the persistent style and transient intention dual-layer latent variables in S4 is as follows: the persistent style latent variable remains fixed within a complete interaction and is obtained by linear mapping from the controllable style encoding; the transient intention latent variable is updated step by step by the gated loop unit with each time step. The gated loop unit receives the current context, the previous loop latent state and the persistent style latent variable at each time step and updates the loop latent state. Then, the inference head maps the transient intention latent variable from the loop latent state. Before outputting the action, the features after multi-layer feature linear modulation are concatenated with the instantaneous intention latent variables and then fed into the action head. The action distribution output by the action head is synthesized by the category distribution covering each action type and the continuous distribution describing the action intensity under the selected action type. The joint training takes the log likelihood of the action in real conversation under the given context, recurrent latent state and intrinsic behavioral style representation as the training target.

[0014] Preferably, the style cycle consistency constraint in S5 is applied as follows: the behavior generated by the closed-loop rolling along with the context it experiences is sent back to the context-conditional encoder trained in S2, and only forward calculation is performed here without updating its parameters to obtain the re-encoded style representation, and the vector distance between the re-encoded style representation and the controllable style encoding used during generation is used as the constraint term. The dynamic support determination method for recoded style representation is as follows: input the recoded style representation into the reversible normalized flow to calculate its occurrence density, requiring that the calculated occurrence density is not lower than the lower bound of the support domain. When using it as a regularization term for closed-loop fine-tuning, the part with occurrence density lower than the lower bound of the support domain is penalized according to its insufficiency.

[0015] Preferably, the method of mapping the target behavior profile to the target style code in S6 is as follows: calculate the behavior statistics for each session on the training set, regress the behavior statistics against the style coordinates to obtain the direction corresponding to each behavior tendency in the style space, and then locate the target coordinates for the target behavior profile along the obtained direction as the target style code. Multi-indicator consistency verification includes distribution closeness, style separability, and interactive responsiveness. Distribution closeness is the degree to which the behavior distribution of the simulated intelligent agent group is close to that of the real group. Style separability is the degree to which the simulated intelligent agents generated according to different target behavior profiles are divided into non-overlapping clusters in the style space. Interactive responsiveness is the degree to which the simulated intelligent agent makes a consistent response in accordance with its own style when the environment changes.

[0016] The beneficial effects of this invention are as follows: By using context-conditional encoding and imposing context-invariant adversarial constraints and identity consistency constraints, intrinsic behavioral styles are separated from contextual influences, avoiding misjudging context-driven behaviors as intrinsic styles, resulting in intrinsic behavioral style representations with improved purity and separability. Based on this, the distribution density is modeled using reversible normalized flow and sampling is rejected by density bands in the low-density support domain, enabling the simulated agent to generate rare and reasonable behaviors on demand while restoring the diversity of group behaviors, avoiding the generation of distorted behaviors that deviate from the real feasible domain, thus balancing realism and controllability. By using multi-layer feature linear modulation and persistent style and instantaneous intention dual-layer latent variables, style is deeply injected into the layer-by-layer computation of decision-making and maintains long-term style self-consistency, thus curbing the pattern collapse of group behavior. Furthermore, by leveraging the style cycle consistency in closed-loop rolling and the dynamic support judgment of re-encoded style representation, style drift caused by error accumulation is suppressed, enabling the simulated agent to react stably in interaction, preventing style dilution, and ensuring that the behavior rolled out by rare styles still falls within the real support domain, facilitating controllable configuration and delivery according to the target behavior profile. Attached Figure Description

[0017] Figure 1 This is a structural diagram of a simulated intelligent agent construction method based on behavioral feature mapping according to the present invention; Figure 2 This is a flowchart of a method for constructing a simulated intelligent agent based on behavioral feature mapping according to the present invention; Figure 3 This is a flowchart of a method for constructing a simulated intelligent agent based on behavioral feature mapping according to the present invention. Figure 4 This is a density scatter plot obtained by projecting the style representation of the present invention onto a two-dimensional plane; Figure 5 This is a cloud and rain map showing the particle distribution of the number of comparison actions in a single session according to the present invention. Detailed Implementation

[0018] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0019] At least one embodiment of the present invention discloses a method for constructing a simulated intelligent agent based on behavioral feature mapping, such as... Figure 1 The diagram shows the overall structure of the method of this invention. ① represents the simulation test environment or the system under test, which is the end-user for performing regression and stress testing after the simulated agent is deployed. ② represents the simulated agent construction system, which internally completes dual structuring, context decoupling, support domain sampling, policy mapping, closed-loop fine-tuning, and verification output sequentially from S1 to S6, forming the main algorithm link from logs to a deliverable agent. ③ to ⑥ represent the context-conditional encoder, reversible normalized flow, style-conditional policy network, and context transition model, respectively, serving as the core support modules called by ②. ③ is trained in S2 and reused during re-encoding in S5, and ④ models style density in S3 and... In S5, dynamic support judgment is performed. ⑤ A style-to-decision injection mapping is completed using multi-layer feature linear modulation and persistent style and instantaneous intention dual-layer latent variables. ⑥ In the training phase, it acts as a closed-loop rolling environment. The ellipse in the figure represents the style space, which carries the intrinsic behavioral style representation z decoupled from the context and the controllable style code z* obtained through density band rejection sampling. ⑦ is the real entity behavior log, which is input into ② after session segmentation and behavior-context dual alignment. ⑧ is the target behavior profile, which is mapped to the target style code along the behavioral tendency direction of the style space and then drives the S6 configuration. After the construction is completed, ② will deploy the simulated agent that has passed the multi-index consistency verification to ①. The specific steps are as follows: Figures 2 to 3 As shown, it includes: S1, obtain entity behavior logs, segment them by session, and split and align each time step into behavior side and context side to obtain paired individual behavior sequence and context sequence samples; Specifically, behavioral logs of real entities in the environment are collected; the logs exist in the form of event streams, and each event records the time of occurrence, action type, action object, and the observable environmental state fields at that time; the original logs are aggregated according to entity identifier and time sequence, and then divided into several sessions according to the activity interval threshold, so that each session corresponds to a continuous and semantically complete activity process; the interval threshold is first obtained by statistically analyzing the distribution of time differences between adjacent events in the logs and then set accordingly. For example, in this embodiment, the default value of the interval threshold is taken as the upper quartile of the distribution of time differences between adjacent events; The semantic integrity of a conversation is not only determined by the time interval, but also by the natural boundaries of the activities. For example, a complete retrieval of a decision is considered a semantic unit. If two activities exceed the interval threshold and cross such a natural boundary, they are considered two conversations. Conversely, even if there is a short pause in between, as long as the topic of the activities is coherent, they are considered as one conversation. In this way, the conversations obtained are semantically self-contained, and the style extracted later will not be artificially fragmented. After segmentation, the fields need to be sorted out. This step encodes the action type as a discrete identifier, standardizes continuous quantities such as action intensity and dwell time, and maps the attributes of the action object to a fixed-length attribute vector. After alignment, each session is organized into behavior records arranged by time step, forming a regular form that can be directly read by subsequent processing, and also avoids interference caused by missing fields and inconsistent units in the original log. Field regularization also needs further explanation. The attribute vector of the action object is formed by concatenating the object's built-in categorical attributes and numerical attributes. The categorical attributes are embedded and mapped into a fixed-length vector, and the numerical attributes are incorporated after standardization. The selection of the context field follows a principle, that is, only state variables that were determined before the current time step and are not controlled by the action of this step are included. All results directly generated by the action of this step are classified as the behavior side to avoid mistaking the results for the context. With this processing, the boundary between the behavior side and the context side is clear, and the semantics of the two sequences will not leak into each other. In one embodiment of the present invention, the separation of the behavior sequence and the context sequence begins by splitting the information of each time step into two paths: one is the behavior sequence, which retains only the actions and their intensity issued by the entity itself, reflecting what the entity did; the other is the context sequence, which retains the environmental state at the same time step that is not directly controlled by the entity's current action, such as the set characteristics of the available objects, the strength of external cues, and the availability of resources, reflecting the context in which the entity is at that time. The two sequences are strictly aligned in time steps and stored in pairs, meaning that each behavior record is bound to the context record in which it occurs. This dual structure is a prerequisite for subsequent decoupling. The reason is simple: only by storing the context separately can the influence of the context be deducted from the behavior during the encoding stage. This step outputs paired individual behavior sequences and context sequence samples, along with entity identifiers to support subsequent subject consistency constraints. To support subsequent subject consistency constraints, sessions are grouped together by entity during structuring; multiple sessions under the same entity name are registered together so that pairs of sessions of the same entity can be extracted during training sampling; the entity identifier itself is not included in the modeling of behavior and context, but only serves as an index for organizing samples, which serves both to construct positive samples for consistency constraints and to facilitate the statistical distribution of behavior by entity.

[0020] S2, pairs of individual behavior sequences and situational sequence samples are fed into a situational conditional encoder. Situational invariance adversarial constraints and subject consistency constraints are applied during the encoding process to extract intrinsic behavioral style representations that are decoupled from the situation. Specifically, this step receives paired individual behavior sequences and context sequence samples output by S1. The goal is to extract a representation that can characterize the entity's inherent style without contextual information. The main purpose is to ensure that the encoding only explains the deviation of the entity from the group's conventional response in the same context, rather than explaining the context itself. To this end, a context-conditional encoder is constructed, which simultaneously reads in the behavior sequence and the aligned context sequence, where the context sequence acts as a condition rather than the subject being encoded. The encoder uses a temporal structure to compress the input layer by layer. At the input end, the behavior vector and context vector of each time step are concatenated and fed into a temporal convolutional layer to obtain intermediate features that preserve local temporal patterns. The intermediate features are then passed through an aggregation layer with self-attention to compress the entire conversation into a fixed-length vector. This fixed-length vector is then processed by two parallel linear mappings to give the mean vector and dispersion vector of the style representation, respectively. The data width of the entire path is compressed layer by layer from the number of time steps of the input multiplied by the feature dimension to the aggregated fixed-length vector, and then to the low-dimensional space where the style representation is located. The data flow is continuously narrowing. Here, we'll explain the data flow in the aggregation layer in more detail. The temporal convolutional layer outputs a series of intermediate feature vectors of the same length as the time step. The aggregation layer introduces a learnable query vector, which calculates the attention weights for each of these intermediate features, and then sums them up according to the weights to obtain a fixed-length vector that does not change with the session length. The advantage of doing this is that sessions of varying lengths are compressed to the same dimension, and the weights automatically favor those time steps that better reflect the style, rather than simply averaging across the time dimension. The mean vector and the discrete vector output by the encoder have the same dimension, and together they define the low-dimensional space in which the style representation of the session resides. The dimension of this space is set according to the coarseness of the styles that can be distinguished in the business. Too few dimensions will squeeze different styles together, while too many dimensions will make the density modeling sparse. For example, in this embodiment, the dimension of the style representation is defaulted to a medium value that can cover the main style differences. The style representation is obtained by differentiable reparameters, which means that a standard normal noise scaled dimensionally by the discreteness vector is superimposed on the mean vector to obtain the style representation of the session. In this way, the sampling process can still backpropagate gradients, and the discreteness vector describes the uncertainty range of the style representation in each dimension. Its introduction allows similar behaviors to be mapped to similar rather than identical representations, thus making the style space continuous and interpolable, laying a smooth foundation for density modeling and sampling in S3. It should be noted that the reconstruction goal on which the encoder relies is not achieved independently in this step, but is undertaken by the policy network of S4 as the decoder; that is, the encoder of S2 and the policy decoder of S4 together constitute a conditional sequence autoencoder. The encoder compresses the real conversation into a style representation, and the decoder reproduces the behavior sequence under the conditions of the style representation and the context. The two are trained together. The more accurate the reconstruction, the better the style representation can support the restoration of the behavior of the conversation. The specific form of this goal is given in S4. In the meantime, let me explain why we don't use a regular autoencoder to directly compress the code. A regular autoencoder only focuses on reconstructing the behavior, and it has no reason to remove the context. Context is often prioritized for encoding because it is easy to predict, and the style is diluted. This method explicitly feeds the context as a condition to both the encoder and decoder, so that both can see the context. Therefore, the encoder no longer needs to use style representation to remember the context. The style representation is freed up to specifically carry the individual biases that the context cannot explain. This is the place where the intrinsic style is located. In one embodiment of the invention, conditional coding alone is insufficient to guarantee decoupling, because the encoder may still secretly encode contextual information into the style representation. To address this, a context discriminator is added after the style representation, allowing it to attempt to predict the contextual summary of the current session from the style representation. A gradient reversal layer (GRL) is inserted between the style representation and the discriminator. This creates a game: the context discriminator tries to minimize the prediction error and extract the context from the style; the encoder, after gradient reversal, receives the negative gradient and is pushed in a direction that prevents the discriminator from extracting the context. The two sides optimize alternately, ultimately ensuring that the style representation retains as few contextual clues as possible. The context summary that the discriminator needs to predict is taken from the concatenation of the first and second-order statistics of the context sequence in the time dimension. That is, the time mean and dispersion of each dimension of the context are concatenated into a fixed-length vector. For key context dimensions such as promotion intensity and option size, the time mean is additionally retained. Using such aggregation instead of stepwise contexts reduces noise in the discrimination target on the one hand, and focuses attention on context tendencies that are easy to mistake context for style on the other hand, so that decoupling is truly implemented. To further stabilize the style of the same entity, a consistency constraint is introduced. The approach is to treat style representations of different conversations of the same entity as positive sample pairs and style representations of different entities as negative sample pairs. During training, positive sample pairs are brought closer together and negative sample pairs are pushed apart. This is achieved by using the information noise contrast objective commonly used in contrastive learning, that is, using cosine similarity to measure the proximity between representations and using a temperature coefficient to adjust the sharpness of the distinction. The temperature coefficient is first optimized on the validation set according to the degree of clustering of the same entity. For example, in this embodiment, the default value of the temperature coefficient is taken as the intermediate value that makes the representations of the same entity cluster together and the representations of different entities separate after optimization. Adversarial decoupling and consistency constraints are trained together in an alternating manner. In each round, the encoder is fixed first to update the context discriminator, making it try its best to predict the context summary accurately. Then, the discriminator is fixed to update the encoder. At this time, the gradient is negatively propagated back through the inversion layer, pushing the encoder in the direction of erasing context cues. Positive and negative samples required for consistency of the same entity are constructed in each batch. Different sessions of the same entity are paired as positive samples, and sessions of other entities are used as negative samples. The three objectives are merged according to their respective weights and then propagated back together. The encoder achieves a balance between reconstruction, anti-context, and style clustering. The aforementioned reconstruction objective, context-invariant adversarial constraint, and identity consistency constraint are combined in a weighted manner to form the encoder's overall objective. The adversarial constraint and the consistency constraint are each assigned a weight coefficient to balance reconstruction accuracy and decoupling strength. The weight values ​​are determined by the trade-off between style separability and reconstruction error on the validation set. After training the three constraints together, an intrinsic behavioral style representation decoupled from the context is obtained. It should be noted that the weights of the three objectives are not fully applied at the outset. In practice, the reconstruction should be prioritized first to support the style space, and then the weights of context invariance and identity consistency should be gradually increased to push out context and gather identity entities. This order from loose to tight can avoid decoupling too early, which could cause the style space to collapse. The final value of each weight is still based on the trade-off between style separability and reconstruction error on the validation set.

[0021] S3 models the distribution density of intrinsic behavioral style representations decoupled from context using reversible normalized flow, and rejects sampling in low-density regions that are still within the support domain according to density bands, thus obtaining controllable style codes that cover common and rare behaviors. Specifically, this step receives the context-decoupled intrinsic behavioral style representations output by S2, and aggregates the style representations of all entities in the training set into a sample set to characterize the style distribution of the real group. Here, it is important to emphasize a dependency: whether this step is valid depends on whether S2 has completely removed the context from the style. If context decoupling is not performed, low-density points in the style space may only correspond to some rare context, not rare people, and the sampled points obtained in this way will be false long tails. Only in the credible style space given by S2 can low density be equated with rare people and real existence within the support domain, and only then will the subsequent sampling criteria be meaningful. Since style distributions are usually irregular and multimodal, it is difficult to directly write out their analytical form. Therefore, an invertible normalized flow is used for modeling. The normalized flow is composed of several invertible transformations cascaded together, which maps the complex style distribution to a base distribution with a simple shape step by step. Its advantage is that it can calculate the occurrence density of any style representation. The algorithm is based on the variable substitution relationship, that is, the log density of a style representation is equal to the log density after it is transformed and mapped to the base distribution, plus the absolute value of the logarithm of the Jacobian determinant of the transformation. The latter term describes the scaling of the spatial volume by the transformation. The invertible transformation is implemented by stacked affine coupling layers. Each layer divides the input dimension into two groups. One group remains unchanged, while the other group undergoes an affine transformation based on the scaling and translation calculated from the first group. This construction ensures the invertibility of the transformation and makes the Jacobian matrix a triangular matrix whose determinant is equal to the product of the scaling factors, making it computationally lightweight. The normalized flow aims to maximize the logarithmic density of the style representation in the training set. After training, a style density model that covers the real population is obtained. There's a small structure inside the affine coupling layer that needs explanation: each layer feeds half of the input into a small subnetwork, which calculates the dimensional scaling and translation amounts used for the other half, then performs an affine transformation on the other half, and passes the unchanging half to the next layer; adjacent coupling layers perform a dimensional permutation, so that each dimension has the opportunity to be transformed and also to serve as a condition. After several layers are cascaded, the originally entangled style distribution is gradually straightened into a shape close to the base distribution; the subnetwork itself is just a regular multilayer perceptron, and its size is set according to the style space dimension; The choice of normalized flow over other generative models is constrained by the requirements of this step; the criterion of low-density support domain sampling requires comparing the occurrence density of each candidate style with a threshold, so a model that can directly calculate the density is necessary; normalized flow naturally provides an analytical density, which is perfect for this purpose; while some models that can only generate samples but cannot provide comparable densities cannot determine whether the candidate falls within or outside the support domain, and the long-tail criterion cannot be executed. The training of normalized flow also requires attention to the following: First, numerical stability. The scaling of the sub-network output is usually subjected to a bounded transformation before use to avoid excessive scaling that could lead to density overflow. Second, initialization. The cascaded coupling layers are initially close to an identity transformation. The training gradually shapes the model with iterations, resulting in more stable convergence. These processes do not change the aforementioned logic of density and sampling; they simply ensure that the flow model can be trained reliably and calculated accurately. In one embodiment of the present invention, with a density model, the style space is sampled in a directional manner according to the test target; for common behaviors, sampling can be performed in high-density areas; for rare but reasonable behaviors, the sampling is allowed to fall on areas with low density but still within the data support domain, thereby obtaining styles that are real but rare; to accurately define this area, two thresholds are set: a lower density threshold and an upper density threshold, and only candidate styles with densities falling between the two are accepted. The sampling criterion is expressed as follows: in, Encode the accepted candidate styles; The log-occurrence density of the normalized flow for this candidate style; This is the lower bound threshold for density, used to exclude unreasonable styles that deviate from the support domain; The upper limit threshold for density is used to filter out overly common styles to highlight the long tail; The two thresholds are determined based on the density quantiles of the style representation of the training set. For example, in this embodiment, the lower density threshold is taken as the low quantile of the density distribution of the training set, and the upper density threshold is taken as the value between the middle and low quantiles, so that the sampling is focused on a reasonable long tail. When the test target is common behavior, the acceptance interval can be shifted to the high density area as a whole. The specific sampling process is as follows: First, a candidate point is sampled on the base distribution. Then, the invertibility of the normalized flow is used to inversely map it back to the style space to obtain the candidate style. Next, the log density of the candidate style is calculated using the variable substitution relation. Then, this density is compared with the above criteria. If it falls within the density band, it is accepted; if it falls outside the band, it is discarded and resampled. When you want to favor the long tail, the candidates on one side of the base distribution are biased towards the value of the medium radius, because the medium radius of the base distribution corresponds to the region in the style space that transitions from the second highest density to the lowest density. Then, the density band is used for fine screening. This method of modeling first and then screening according to density constraints avoids the problem that adding noise directly to the style may cause it to go beyond the support domain, so that rare styles always remain in the realistic and feasible range. The support domain boundary is determined by calculating the logarithmic density of each style representation in the training set after training the flow model, resulting in an empirical density distribution. The lower and upper density thresholds are then taken from the quantiles of this distribution. Regions below the lower bound are considered to be outside the support domain because the samples are so sparse that they have almost no real basis. Regions above the upper bound are considered to be overcrowded and belong to common styles. Regions falling between the two are considered to be the reasonable long tail, which is both rare and supported by real samples. This sampling method yields a controllable style encoding that covers both common and rare behaviors. It should be noted that the density band here only serves as the first check in the static style space, ensuring that the selected style has indeed appeared in history. As for whether the behavior of this style rolling out after entering the closed-loop interaction is still reasonable, it needs to be checked by the dynamic support judgment of the re-encoded style representation in S5 as the second check. Only when both checks are combined does it count. Therefore, the controllable style code obtained in this step is strictly speaking a candidate style code, and its rationality in the interaction is confirmed by S5.

[0022] S4, with controllable style coding as condition and context sequence as input, uses a temporal policy network with multi-layer feature linear modulation and persistent style and instantaneous intention dual-layer latent variables, and is jointly trained with context-conditional encoder to obtain style-conditional policy; Specifically, this step receives the controllable style code output by S3 and the context sequence output by S1. The goal is for the agent to perform actions at each time step based on the current context and following a given style. Here, we first clarify the source of the conditions: the generation phase uses the controllable style code from S3 as the condition, while the training phase uses the posterior style representation given by the encoder from the current real session by S2 as the condition. Both scenarios share the same set of policy network parameters, only the source of the conditions differs. The policy network processes the context sequence in a temporal structure. At each time step, it reads the current context, the recurrent hidden state, and the style code, and outputs the action distribution for that step. This action distribution is composed of two parts: one part is the category distribution covering each action type, and the other part is the continuous distribution describing the action intensity under the selected action type. Together, they constitute the complete action distribution for that step. The fundamental training objective of the policy network is to reproduce real actions. During training, action sequences from real conversations are used as supervision, and the style encoding is taken from the style representation given by the S2 encoder for that real conversation. Given the context, recurrent hidden state, and style representation, the network aims to maximize the log-likelihood of real actions, that is, first maximize the probability of the class of real action type, and then maximize the probability of real action intensity under continuous distribution. This conditional likelihood objective is the reconstruction objective on which S2 depends. The encoder and policy decoder thus form a conditional sequence autoencoder and are trained together. The main trunk of the mapping from behavioral features to decisions is closed here, with no gaps in the middle. To ensure that style encoding acts stably on decision-making without being overwhelmed by contextual features, style injection employs Feature-wise Linear Modulation (FiLM). This involves generating a set of channel-wise scaling and translation coefficients at multiple layers of the policy network, scaling and translating the features of that layer channel by channel. These coefficients are calculated from the style encoding by a small mapping network, with one set generated for each modulated layer. This allows style to influence local action tendencies at a shallow level and overall decision orientation at a deep level, deeply participating in the layer-wise computation of decision-making, rather than simply being concatenated at the input and then diluted layer by layer. There is another consideration for multi-layer injection: if the style is only fed once at the input, the network can easily learn to bypass it and guess the action with a high degree of accuracy based on the context alone, thus rendering the style ineffective; by repeatedly injecting the style along multiple layers, and with the subsequent cyclic consistency forced by S5, the network cannot bypass it and can only use the style, thus blocking the problem of condition invalidation. The small mapping network that generates linear modulation coefficients of features assigns a set of parameters to each modulated layer. The input is the same style code, and the output is the channel-wise scaling and translation of the layer. Regardless of whether the conditions come from the posterior style representation in the training stage or the controllable style code in the generation stage, this mapping network works in the same way, only the source of the style code fed in is different. To further explain the process of action head output distribution: The category distribution is generated by the network for all action types, and mapped to the probability of each action type through softmax (normalized exponential function); given the selected action type, the network outputs the continuous distribution parameters (such as mean and variance) corresponding to that type, thereby characterizing the probability distribution of normalized action intensity under that type; during the training phase, the type of the real action is used to extract its probability from the category distribution, and the intensity is substituted into the corresponding continuous distribution to calculate the probability; the log probability values ​​of the two parts are added together as the log likelihood of the behavior at that step, and the accumulation of all time steps is the log likelihood of the entire session; The data width of this path is also processed. Each step of the context sequence is a context vector. After entering the policy network, it is first fused with the recurrent hidden state. The intermediate features are repeatedly injected into the style by linear modulation of the features in several layers. The width is maintained on the hidden dimension of the network. Finally, it is narrowed to the number of categories plus the number of parameters in the continuous distribution when it reaches the action head. The style encoding is amplified from the low-dimensional space of S3 through a small mapping network to the modulation coefficients required by each layer. It is used up after its function and does not enter the recurrent state. In one embodiment of the present invention, in order to maintain style consistency and real-time flexibility in long-term interactions, the hidden state of the agent is divided into two layers; the persistent style latent variable remains fixed within a complete interaction and is directly given by the controllable style encoding, which is responsible for maintaining the consistent behavioral orientation of the same agent. It is applied to each layer of the policy network through the linear modulation of the features mentioned above; the instantaneous intention latent variable evolves with time step and is responsible for characterizing the current short-term action tendency, such as whether to continue comparing or enter the decision-making stage at this moment. Here we supplement the relationship between persistent style latent variables and style coding; persistent style latent variables are not created separately, they are directly obtained from controllable style coding through a linear mapping, and the entire interaction remains unchanged; that is to say, the style coding sampled by S3 is used to generate the modulation coefficients of each layer and is also used as persistent style latent variables through mapping. The two use the same source, so the agent's style orientation is aligned from beginning to end. The instantaneous intent latent variable is not the same as the ordinary recurrent latent state. It is a low-dimensional intent vector obtained by passing a small inference head on top of the recurrent latent state. Its input is the current situation, the previous recurrent latent state, and the persistent style latent variable. It is updated step by step by the gated recurrent unit (GRU) over time. Before taking an action, the features linearly modulated by the features are concatenated with the instantaneous intent vector and then sent to the action head to obtain the action distribution. In this way, the persistent latent variable that determines the style orientation is locked in the whole segment. The same agent will not jump back and forth in style during the interaction. The mobility required to deal with the situational changes is undertaken by the instantaneous intent latent variable. The two have a clear division of labor. The process of updating the transient intent is as follows: at each time step, the gated recurrent unit receives the current context vector, the previous recurrent latent state, and the persistent style latent variable. It calculates the new recurrent latent state through its internal update gate and reset gate. The inference head then maps the low-dimensional transient intent vector from the new recurrent latent state. The persistent style latent variable remains unchanged throughout and only participates as a condition. It is not rewritten by the recurrent update. In this way, the intent follows the context and the style remains stable. The two pieces of information do not overlap. The overall training process consists of three stages. The first stage jointly trains the S2 encoder and the policy decoder for this step, with the loss being the aforementioned behavior likelihood reconstruction term, superimposed with context-invariant adversarial terms, identity consistency terms, and latent variable regularization terms. The second stage freezes the trained encoder, collects its style representations for each session in the training set, and trains the normalized flow of S3 using maximum likelihood. The third stage freezes the encoder and the normalized flow, and performs closed-loop fine-tuning of the policy decoder in the closed-loop environment given in S5. The context transfer model used in this closed-loop environment has been trained on logs before entering this stage. Each of the three stages has its own function, and the product of the previous stage is the input of the next stage. The training relationship does not contain circular dependencies. This step yields a style-conditional policy, which outputs behavioral decisions step by step under a given style and context. Each action includes the action type and intensity.

[0023] S5 allows the style-conditioning strategy to roll in the closed-loop environment served by the context-transfer model, applies style cycle consistency constraints, and applies dynamic support judgment to the re-encoded style representation obtained by the context-conditioning encoder of the closed-loop behavior, and fine-tunes to obtain the simulated agent. Specifically, this step receives the style-conditioning policy obtained from S4 and places it in an interactive environment for closed-loop rolling. That is, every time the agent makes an action, the environment updates its state and provides feedback on the new context. The agent then decides the next step based on the new context. Here, the behavioral decision is generated gradually by the policy during rolling, rather than by a pre-given static sequence. It is important to explain where the closed-loop environment comes from, because without an environment that provides feedback on the context, there is no closed loop. In the testing phase, the environment is the system under test itself. The agent is connected to the sandbox of the recommendation or interaction system under test, and the system under test provides feedback on the next context. Since the training phase cannot rely on online systems, this step uses a context transition model learned from logs to act as the environment. The input of this transition model is the current context and the action taken by the agent in the current step, and the output is a prediction of the next context. During training, the next context that actually appears in the real logs is used as the label, and the deviation between the prediction and the label is used as the loss for supervised learning. It is a supporting component, and its role is simply to allow the policy to roll in a closed loop during training. The closed-loop rolling enables the agent to be reactive and respond to changes in the environment, which overcomes the old problem of not responding when the behavior is replayed. The specific structure of the context transition model is as follows: it reads the current context vector and the agent's current action, concatenates the two, fuses the recent context and action history through a temporal network, and then outputs the prediction of the values ​​of each dimension of the next context; training is supervised by the pair of current context plus action and its next context appearing in real logs, and the deviation between the prediction and the real next context is used as the loss; it only acts as a rolling environment during the training phase, and is replaced by the system under test during the testing phase, and the two have the same external interface; In one embodiment of the present invention, closed-loop rolling will accumulate the agent's own error step by step. If there is no constraint, the state will deviate from the true distribution after several rounds of rolling. It should be noted that the policy is prone to ignoring style in the closed loop and reverting to average behavior, making the style extracted earlier useless. To this end, a style cycle consistency constraint is introduced. The method is to send the behavior sequence generated by the agent's closed-loop rolling, together with the context it has experienced, back to the trained context-conditional encoder S2 to obtain a re-encoded style representation. It is required that it be close to the controllable style encoding used during generation. The degree of closeness between the two is measured by the distance between the vectors and used as a constraint term. This constraint turns whether style is truly expressed into a measurable and optimizable objective; once the agent ignores style and reverts to routine behavior during the roll, the re-encoded style representation will deviate from the given encoding, and the cycle consistency constraint will increase accordingly. During training, this tendency will be suppressed, and behavioral homogenization and pattern collapse will be curbed. This constraint works together with the behavior likelihood of S4 to make the agent both close to reality and retain style. In style cycle consistency, the re-encoding uses the pre-trained contextualized encoder S2. Here, it is only used for forward computation and its parameters are not updated. The reason for reusing the same encoder is to ensure that the re-encoded style representation and the style used in the original generation are in the same metric space, so that the distance between the two can be compared and the determination of whether the style has been preserved can be made more accurate. It's not enough to simply reproduce the style; we also need to see if this closed-loop behavior still resembles what real customers would do. The difficulty here is that rare styles lack corresponding real samples, so we can't use a single authenticity criterion for that style to judge it. On the other hand, if we use the average real behavior of the group as a benchmark, we will also erase the rare styles that we have painstakingly collected, and impulsive customers will be dragged back to ordinary customers. Therefore, the authenticity here must be style-specific, rather than aiming at the average. To address this, a dynamic support decision is added in addition to the recoding. The approach involves feeding the recoded style representation generated from the closed-loop behavior into the pre-trained normalized stream of S3 to calculate its occurrence density, requiring that this density is not lower than the lower bound of the support domain. This means that the style corresponding to this closed-loop behavior must still fall within the support domain that the real customer has previously encountered. Once the behavior drifts out of the real domain and occurs in a behavior that has never occurred before, the density of the recoded style representation will fall out of the support domain, and this decision will fail. It reuses the encoder of S2 and the density model of S3 without introducing any new discriminant components. This decision is used as a regularization term in the closed-loop fine-tuning, and is fed back along with style cycle consistency. When used as a training regularization term, it takes the form of a one-sided penalty. When the density of the encoded style representation is lower than the lower bound of the support domain, a penalty is given for the insufficient amount. When it reaches the lower bound, no penalty is given. In the generation stage, whether the lower bound is reached is used directly as the decision to accept or reject the agent. With this constraint, the policy does not go beyond the support domain of real interaction while maintaining the given rare style. Thus, rareness and rationality are no longer determined only in the static style space, but are jointly determined in the closed-loop interaction behavior space. Impulsive customers are still impulsive, but impulsive like real customers. If there are stricter requirements for the realism of the action level, an additional realism judgment for scrolling transition can be added as an optional enhancement. In general, the above decision is sufficient. Optionally, real distribution correction can be periodically introduced during closed-loop rolling to further suppress state drift. Specifically, when the rolling reaches a set step size threshold, the state distribution of the agent is aligned with the state distribution of the real data in the same context. The difference between the two distributions is used as the correction target to pull the deviated state back to the reliable region before continuing to roll. The step size threshold for triggering correction is taken from the curve of state distribution deviation increasing with the number of steps. For example, in this embodiment, the step size threshold is taken as the number of steps before the distribution deviation begins to increase rapidly. This correction item is optional and can be enabled in situations where the requirements for state drift are more stringent. In the distribution correction, the common distribution divergence metric is used to measure the difference between the two state distributions. The specific value only affects the correction intensity and does not change the process. Rolling and correction alternate at a fixed rhythm. First, the agent is allowed to roll freely for a number of steps to reflect responsiveness. When the step size threshold is reached, a distribution alignment is performed, and then the rolling continues. This process is repeated. This preserves the closed-loop response freedom and prevents the error from accumulating in long rolls. After style cycle consistency constraints, dynamic support judgment of re-encoded style representation, and optional distribution correction, a simulated agent with style self-consistency and stable response after closed-loop fine-tuning is produced.

[0024] S6. Obtain the target behavior profile, map it to the target style code along the behavioral tendency direction of the style space, configure the simulated agent accordingly, and output the simulated agent that can be configured as needed and passes the verification after multi-index consistency verification. Specifically, this step receives the simulated agent output by S5, which is style-consistent and has stable response after closed-loop fine-tuning, and the target behavior profile provided by the tester. The target behavior profile describes the required agent type using several interpretable behavioral tendencies, such as high decision intensity and long comparison period. Since the style space obtained by S2 is decoupled from the context and has separability, a correspondence between behavioral tendencies and style coordinates can be established in this space to map the target behavior profile into one or a set of target style codes, which are then handed over to S4 to generate the corresponding agent. The correspondence is constructed by calculating several interpretable behavioral statistics for each session on the training set, such as decision speed, comparison rounds, and sensitivity to promotions. These statistics are then regressed against style coordinates to obtain the direction corresponding to each behavioral tendency in the style space. Given a target behavioral profile, the target coordinates are located in the style space along these directions, which is the target style encoding. When group generation is required, a cluster of codes is taken from the neighborhood of the coordinate. For common types, the target style encoding is obtained in high-density areas; for rare types required for testing, it is obtained in reasonable long tails by using the low-density support domain sampling of S3. With this directional mapping, the tester does not need to manually adjust the parameters one by one, and can configure simulated agents with specified behavioral tendencies in batches as needed, thus ensuring controllability. In one embodiment of the present invention, the generated agent undergoes a multi-index consistency check before delivery to confirm that its behavior is both close to reality and meets the configuration target. The behavior used for the check is taken from the trajectory generated by these agents rolling in a closed-loop environment. Before performing these three checks, a threshold is set. The newly generated agent rolls in the closed-loop environment in the S5 manner for a period of time, and the occurrence density of its re-encoded style representation is compared with the lower bound of the support domain. Those whose density does not reach the lower bound are directly eliminated, and the remaining ones enter the subsequent checks. The checks are carried out from three aspects: first, distribution closeness, comparing the degree of similarity between the behavior distribution of the simulated agent group and the behavior distribution of the real group; second, style separability, checking whether the agents generated according to different target profiles are distinguishable in behavior and stable; and third, interactive responsiveness, checking whether the agent can make a consistent response that conforms to its own style when the environmental response changes. Each of the three verification indicators has a clear interpretation method; distribution closeness examines whether the distribution of the simulated group and the real group overlaps in several behavioral statistics; style separability examines whether agents generated from different target profiles can be divided into non-overlapping clusters after being projected into the style space; and interactive responsiveness involves injecting different responses into the system in a closed loop and observing whether the agent's response changes with the response without deviating from its own style; all three indicators must be met to pass. Agents that fail validation are regressed to the corresponding step based on their defect type. For example, if the distribution closeness is insufficient, they regress to S5 to strengthen constraints; if the style separability is insufficient, they regress to S3 to adjust the sampling area. Agents that pass validation are the final output of this method. These validated simulated agents can be configured as needed and deployed directly to the simulation test environment. Validated simulated agents are delivered in a configurable form. Each agent is bound to its target style code and its corresponding profile label. During deployment, the tester selects agents for the corresponding profile according to the test plan and can adjust the ratio of agents for each profile to approximate the group composition required for a certain type of test scenario. The entire configuration process revolves around style coding, eliminating the need to retrain the model. Thus, the complete link from behavior logs to controllable simulated agents is closed, with strict connection between the inputs and outputs of each step. The output of the previous step serves as the input for the next step in the calculation. In one embodiment of the present invention, the above method is applied to the testing of an intelligent recommendation and interaction system of an online service platform. It is used to construct a simulated customer intelligent agent to reproduce the browsing, searching, comparing, adding to cart, placing orders and churn of real customers in a controlled environment. The testing team's requirement is that these simulated customers not only need to be realistic individually, but also need to reproduce the differences between real customers in groups, and be able to generate rare but real customer types in a targeted manner, so as to perform regression and stress testing on recommendation strategies and interaction processes. The example first obtains the operation logs of real customers on the platform, and after processing by S1, pairs of behavior sequences and context sequences are obtained; as shown in Table 1, an example of the initial data after structure alignment is given. Table 1, Example of Initial Data In this example, contextual decoupling in S2 plays a crucial role. Taking customer U002 as an example, he quickly adds items to his cart in a strong promotional context. Without contextual decoupling, he would easily be judged as having an impulsive intrinsic style. However, after removing the promotional intensity context, contextual conditional encoding reveals that the customer's behavioral deviation relative to the same group in the same context is not significant. His rapid addition to the cart is mainly driven by the promotion, rather than by his intrinsic style. After training with context-invariant adversarial constraints and identity consistency constraints, U002's style representation is consistent with his representation in a weak promotional context, indicating that a stable intrinsic style has been extracted. like Figure 4As shown, this is a density scatter plot obtained by projecting the customer style representation onto a two-dimensional plane after extraction by S2 and occurrence density estimation by S3 using reversible normalized flow. Each point in the figure corresponds to the style representation of a customer session, and the color intensity of the point represents the occurrence density estimated by reversible normalized flow (see the color bar on the right). The horizontal and vertical axes are the first and second coordinates after dimensionality reduction of the style representation, respectively, and the coordinate plane is not gridded. The dashed circle marks the boundary of the support domain of the style space. The interior of each cluster zone, from dark to light, corresponds to the common styles with high density within the cluster to the reasonable long tail with low density at the cluster edge but still falling within the support domain. Different cluster zones are separated from each other, indicating that the style representation is separable, thus providing a basis for S3 to reject sampling according to density bands in the low-density support domain. in accordance with Figure 4 In the density structure, S3 performs rejection sampling in the low-density support region according to density bands, obtaining rare style codes such as impulsive customers with high decision-making intensity and cautious customers with long comparison cycles. These style codes are injected into the temporal policy network through multi-layer feature linear modulation by S4, and the style orientation of the entire interaction is locked by persistent style latent variables, so that impulsive customers continue to show quick decision-making in multiple rounds of interaction, while cautious customers continue to show repeated comparisons without style jumps in the middle. Then S5 puts these strategies into the closed-loop rolling of the interaction environment, allowing simulated customers to respond to the recommendation system in a way that conforms to their own style, and uses style cycle consistency to maintain the style. At the same time, the dynamic support judgment of the re-encoded style representation confirms that the behavior of these rare customers still falls within the support domain of real customers, preventing the situation of rolling out. Reactivity is specifically manifested in the fact that the same impulsive simulated customer will react quickly but decisively when faced with recommendations of varying strengths from the recommendation system; while cautious customers will still make multiple comparisons under the same stimulus; the difference between the two comes from their respective locked-in persistent styles, while the immediate response to the stimulus comes from the gradual adjustment of the instantaneous intention in the closed loop, so style stability and flexible response coexist. like Figure 5 The figure shows a granular distribution cloud map of the number of comparison actions in a single session. The horizontal axis represents the real customers, the simulated customers generated by this method, and the control group without low-density support domain sampling. The number of comparison actions for the simulated customers and the control group is calculated based on the self-closed-loop interactive scrolling trajectory, while the real customers are taken from the corresponding log sessions. The vertical axis represents the number of comparison actions within a single session. The left half represents the kernel density cloud, and the right half represents the individual point rain. The short horizontal line represents the median, and the thick vertical line represents the interquartile range. The coordinate plane is not gridded. As can be seen from the figure, the main interval of the method group is close to that of the real customers, and there is still coverage in the long tail segment with a relatively high number of comparisons (corresponding to the rare style of cautious equal-length comparison period). The control group without low-density support domain sampling is significantly sparse in this long tail segment. After the target behavior profile mapping and multi-indicator consistency verification of S6, the testing team obtained a batch of simulated customer agents that are both close to reality and can be configured as needed. After integrating them into the test environment of the recommendation and interaction system, they can not only reproduce the mainstream behavior of regular customers, but also perform stress simulations of the system's strategy robustness in long-tail scenarios with rare customer types, thus completing sufficient regression and stress testing without disturbing real users. To further elaborate on the scenario when these simulated customers are put into use, the testing team first generates a batch of simulated customers close to the mainstream based on common profiles to run daily regression of recommendation strategies to confirm that changes have not harmed the mainstream experience. Then, a small number of impulsive and cautious customers are generated based on rare profiles to specifically stress those long-tail paths that are difficult to collect normally, to observe whether risk control and fallback strategies will fail. The entire process does not involve real users and does not consume online traffic, so the testing pace is much faster. In terms of results, this process truly bridges the gap between controllability and realism. In the past, either diversity was sacrificed to achieve realism, or diversity was unleashed but distorted behavior was created. In this example, the style is decoupled first and then sampled by density, so that when the controllable knob is turned to the rare side, the generated product is still a customer backed by real samples, rather than a bizarre behavior created out of thin air.

[0025] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A method for constructing a simulated intelligent agent using behavioral feature mapping, characterized in that, Includes the following steps: S1, obtain entity behavior logs, segment them by session, and split and align each time step into behavior side and context side to obtain paired individual behavior sequence and context sequence samples; S2, pairs of individual behavior sequences and situational sequence samples are fed into the situational conditional encoder. Situational invariance adversarial constraints and subject consistency constraints are applied during the encoding process to extract intrinsic behavioral style representations that are decoupled from the situation. S3 models the distribution density of intrinsic behavioral style representations decoupled from context using reversible normalized flow, and rejects sampling in low-density regions that are still within the support domain according to density bands, thus obtaining controllable style codes that cover common and rare behaviors. S4, with controllable style coding as condition and context sequence as input, uses a temporal policy network with multi-layer feature linear modulation and persistent style and instantaneous intention dual-layer latent variables, and is jointly trained with context-conditional encoder to obtain style-conditional policy; S5 allows the style-conditioning strategy to roll in the closed-loop environment served by the context-transfer model, applies style cycle consistency constraints, and applies dynamic support judgment to the re-encoded style representation obtained by the context-conditioning encoder of the closed-loop behavior, and fine-tunes to obtain the simulated agent. S6. Obtain the target behavior profile, map it to the target style code along the style space behavior tendency direction, configure the simulated agent accordingly, and output the simulated agent that can be configured as needed and passes the verification after multi-index consistency verification.

2. The method for constructing a simulated intelligent agent based on behavioral feature mapping according to claim 1, characterized in that, The interval threshold used for session segmentation in S1 is obtained by statistically analyzing the distribution of time differences between adjacent events in the log. The information at each time step is split into a behavior side and a context side. The behavior side retains the actions initiated by the entity itself, while the context side retains the environmental state at the same time step that is not directly controlled by the entity's current action. The behavior side and the context side are aligned at the time step and stored in pairs, and the samples stored in pairs are accompanied by entity identifiers. The splitting of the behavior side and the context side allows the influence of the context to be deducted from the behavior in subsequent encoding.

3. The method for constructing a simulated intelligent agent based on behavioral feature mapping according to claim 1, characterized in that, The situation invariance adversarial constraint in S2 is applied as follows: after the intrinsic behavioral style representation decoupled from the situation, a situation discriminator is connected, which is then used to predict the situation summary from the intrinsic behavioral style representation. A gradient inversion layer is inserted between the intrinsic behavioral style representation and the situation discriminator, which minimizes the prediction error. The situation conditional encoder is pushed in the direction that makes the situation discriminator unable to predict the situation summary after gradient inversion. The scenario summary is derived from the concatenation of first-order and second-order statistics of the scenario sequence in the time dimension. The adversarial constraint is trained in an alternating manner. First, the scenario conditional encoder is fixed to update the scenario discriminator, and then the scenario discriminator is fixed to update the scenario conditional encoder.

4. The method for constructing a simulated intelligent agent based on behavioral feature mapping according to claim 1, characterized in that, The method for applying the consistency constraint of the same entity in S2 is as follows: the intrinsic behavioral style representations of different sessions of the same entity are used as positive sample pairs; The intrinsic behavioral style representations of different entities are used as negative sample pairs. The information noise is used to compare the target to bring the positive sample pairs closer and push the negative sample pairs further away. Cosine similarity is used to measure the closeness between intrinsic behavioral style representations, and temperature coefficient is used to adjust the sharpness of the distinction.

5. The method for constructing a simulated intelligent agent based on behavioral feature mapping according to claim 1, characterized in that, The reversible normalized flow in S3 is composed of several cascaded affine coupling layers. Each affine coupling layer divides the input dimension into two groups. One group remains unchanged, and the other group is subjected to affine transformation according to the scaling and translation calculated by the unchanged group. The dimensions are permuted between adjacent affine coupling layers. The reversible normalized flow is trained with the goal of maximizing the log density of intrinsic behavioral style representations in the training set. The log density of any intrinsic behavioral style representation is equal to the sum of its log density after being mapped to the base distribution by the reversible normalized flow and the absolute value of the logarithm of the Jacobian determinant of the mapping.

6. The method for constructing a simulated intelligent agent based on behavioral feature mapping according to claim 5, characterized in that, The method of rejecting sampling by density band in S3 is as follows: sample candidate points on the base distribution, use the invertibility of the invertible normalized flow to inversely map the candidate points back to the style space to obtain candidate styles, and calculate the logarithmic occurrence density of candidate styles. Candidate styles are accepted when the log density falls within the density band defined by the lower and upper density thresholds; otherwise, they are discarded and resampled. The lower and upper density thresholds are taken from the quantiles of the log density empirical distribution representing the intrinsic behavioral style of the training set. When it is necessary to bias towards rare behaviors, the candidate points on the base distribution are biased towards values ​​with medium radius.

7. The method for constructing a simulated intelligent agent based on behavioral feature mapping according to claim 1, characterized in that, The multi-layer feature linear modulation method in S4 is as follows: a set of channel-wise scaling and translation coefficients are generated by controllable style coding at multiple layers of the temporal policy network, and the features of the corresponding layers are first scaled by channel and then translated by channel. The scaling and translation coefficients are calculated from the controllable style code by a mapping network, and a set is output for each modulated level, so that the controllable style code participates in the determination of local action tendencies in the shallow layer of the temporal policy network and participates in the determination of the overall decision orientation in the deep layer.

8. The method for constructing a simulated intelligent agent based on behavioral feature mapping according to claim 1, characterized in that, The setting method of the persistent style and transient intention dual-layer latent variables in S4 is as follows: the persistent style latent variable remains fixed within a complete interaction and is obtained by linear mapping from the controllable style code. The transient intention latent variable is updated step by step by the gated loop unit with each time step. The gated loop unit receives the current situation, the previous loop latent state and the persistent style latent variable at each time step and updates the loop latent state. Then the inference head maps the transient intention latent variable from the loop latent state. Before outputting the action, the features after multi-layer feature linear modulation are concatenated with the instantaneous intention latent variables and then fed into the action head. The action distribution output by the action head is synthesized by the category distribution covering each action type and the continuous distribution describing the action intensity under the selected action type. The joint training takes the log likelihood of the action in real conversation under the given context, recurrent latent state and intrinsic behavioral style representation as the training target.

9. The method for constructing a simulated intelligent agent based on behavioral feature mapping according to claim 1, characterized in that, The style cycle consistency constraint in S5 is applied as follows: the behavior generated by the closed-loop rolling along with the context it experiences is sent back to the context-conditional encoder trained in S2, and only forward calculation is performed here without updating its parameters to obtain the re-encoded style representation. The vector distance between the re-encoded style representation and the controllable style encoding used during generation is used as the constraint term. The dynamic support determination method for recoded style representation is as follows: input the recoded style representation into the reversible normalized flow to calculate its occurrence density, and require that the calculated occurrence density is not lower than the lower bound of the support domain. When using it as a regularization term for closed-loop fine-tuning, the part of the occurrence density that is lower than the lower bound of the support domain is penalized according to its insufficiency.

10. The method for constructing a simulated intelligent agent based on behavioral feature mapping according to claim 1, characterized in that, The method of mapping the target behavior profile to the target style code in S6 is as follows: calculate the behavior statistics for each session on the training set, regress the behavior statistics against the style coordinates to obtain the direction corresponding to each behavior tendency in the style space, and then locate the target coordinates for the target behavior profile along the obtained direction as the target style code. Multi-indicator consistency verification includes distribution closeness, style separability, and interactive responsiveness. Distribution closeness is the degree to which the behavior distribution of the simulated intelligent agent group is close to that of the real group. Style separability is the degree to which the simulated intelligent agents generated according to different target behavior profiles are divided into non-overlapping clusters in the style space. Interactive responsiveness is the degree to which the simulated intelligent agent makes a consistent response in accordance with its own style when the environment changes.

Images