Method and device for constructing group user portraits based on agents

Abstract

The invention provides a method and a device for constructing a group user portrait based on an intelligent agent. The method comprises the following steps: acquiring a feature tag and a keyword library of a target group user; receiving the current content recommended by the recommendation system; generating an active strategy by adopting an active strategy network based on the current content, and generating a passive strategy by adopting a passive strategy network under the condition that the active strategy is not generated; executing the passive strategy until the current content is traversed, or searching a target keyword corresponding to the active strategy in a keyword library, searching in a recommendation system by adopting the target keyword, and performing content selection based on a search result; receiving the next content recommended by the recommendation system; calculating a reward value corresponding to the next content and the feature tag in semantic consistency, calculating a penalty value under the condition that the active strategy is executed, and fitting the reward value with the penalty value to obtain a fitting value; and optimizing the strategy network based on the fitting value. And the construction process of the user portrait is obviously accelerated.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and machine learning technology, and in particular to a method for constructing a group user profile based on an intelligent agent, an apparatus for constructing a group user profile based on an intelligent agent, a computer device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the rapid development of short video and content recommendation platforms, the recommendation algorithms behind these platforms have become core technologies affecting user experience and information distribution efficiency. To evaluate the performance of platform recommendation algorithms across different user groups, intelligent agents are typically used to simulate real user behavior and interact within the platform to collect and analyze feedback data.

[0003] To enable intelligent agents to simulate real user behavior, the common approach is as follows: First, based on human experience, a state-action mapping table is established for the target group (e.g., elderly users). This mapping table includes the characteristics of the content displayed on the platform (e.g., opera, games) and their corresponding designated actions (e.g., liking, swiping, viewing duration). Then, the intelligent agent, for each piece of content recommended by the platform, looks up the corresponding characteristics in the mapping table and uses the action corresponding to that characteristic in the mapping table to perform an action on that content.

[0004] However, using a state-action mapping table to enable an agent to simulate the real behavior of a specified user group is problematic because the mapping table itself is a static and deterministic behavioral strategy. This forces the agent to rigidly execute predetermined actions, requiring extensive interaction between the agent and the platform to build a profile of the specified user group. This extensive interaction takes a considerable amount of time, ultimately reducing the time required for the agent to construct the profile of the specified user group within the platform. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for constructing user profiles for a group based on intelligent agents, so as to shorten the cycle of constructing a profile of a specified user group by an intelligent agent in a platform.

[0006] To address the aforementioned technical problems, this application provides the following technical solutions: The first aspect of this application provides a method for constructing a user profile of a target user group based on an intelligent agent. The method is applied to an intelligent agent interacting with a recommendation system. The intelligent agent includes a policy network, which comprises a passive policy network and an active policy network. The method includes: acquiring feature tags and a keyword library of the target user group; receiving current content recommended by the recommendation system; generating an active policy based on the current content using the active policy network, and if no active policy is generated, generating a passive policy using the passive policy network, wherein the passive policy indicates the operation to be performed on the current content, and the active policy indicates the operation to be performed in the recommendation system; executing the passive policy until the current content is scrolled over, or searching for the target keyword corresponding to the active policy in the keyword library, and using the target keyword to search in the recommendation system and selecting content based on the search results; receiving the next content recommended by the recommendation system; calculating the reward value corresponding to the semantic consistency between the next content and the feature tags, and, if the active policy has been executed, calculating the penalty value, and fitting the reward value and the penalty value to obtain a fitted value; optimizing the policy network based on the fitted value until the intelligent agent constructs a user profile of the target user group in the recommendation system.

[0007] Compared to existing technologies, the method for constructing user profiles based on intelligent agents provided in the first aspect of this application introduces a policy network (active / passive policy network) with a hierarchical decision-making mechanism, enabling the intelligent agent to flexibly simulate the complex interaction patterns of real users (i.e., including both passive consumption behavior and active exploration behavior based on search). It effectively utilizes the high-value signal of active retrieval, and can quickly guide the feedback of the recommendation system to converge towards the preferences of the target group, thereby significantly accelerating the user profile construction process. This solves the problems of low interaction efficiency and excessively long profile construction cycle caused by traditional static behavior strategies, and provides a high-quality data foundation for subsequent algorithm evaluation.

[0008] In other embodiments provided in this application, the agent further includes a value network; and a policy network optimized based on fitted values, including: generating a first foreground value for the current content using the value network, and generating a second foreground value for the next content using the value network, wherein the foreground value is used to characterize the expectation of the decision network to complete the construction of the target group user profile based on the current and subsequent interactions; generating an advantage value based on the fitted value, the first foreground value, and the second foreground value, wherein the advantage value is used to characterize the magnitude of the foreground value of the next content compared to the current content in achieving the target group user profile; based on Optimize the policy network, where L(θ) represents the policy network loss function, r t (θ) represents the importance sampling ratio. Indicates the advantage value.

[0009] By introducing a value network and a dominance function, policy network optimization is based on long-term global benefits rather than immediate rewards. Combined with the stable update mechanism of the Proximal Policy Optimization (PPO) algorithm, the efficiency and stability of agent learning are significantly improved, and the convergence of target user profiles is accelerated.

[0010] In other embodiments provided in this application, an advantage value is generated based on the fitted value, the first foreground value, and the second foreground value, including: based on A t =R t +γ*V(s {t+1} )-V(s t ), calculate the dominance value, where A t R represents the dominance value. t V(s) represents the fitted value, γ represents the discount factor, and V(s) represents the value fitted. {t+1} V(s) represents the second foreground value. t ) represents the first foreground value.

[0011] By calculating the advantage value in the form of temporal difference, it is possible to combine the current fitting reward with the value network's prediction of the future state to achieve an efficient and unbiased estimation of the long-term impact of a single step action. This avoids dependence on the complete trajectory, reduces variance, and makes policy optimization more stable and efficient, thereby significantly improving the learning speed of the agent in exploration and the convergence efficiency of the target profile.

[0012] In other embodiments provided in this application, the method further includes: based on Target=R t +γ*V old (s {t+1} ), calculate the target value, where Target represents the target value, R t V represents the fitted value, γ represents the discount factor, and V old (s {t+1} The second foreground value is represented by the target value; the first foreground value is subtracted from the target value to obtain the error; the value network is optimized based on the error.

[0013] By calculating the target value required for updating the value network through temporal difference objective, and effectively fixing part of the optimization objective using historical network parameters (Vold), the self-circulation bias in value estimation is reduced, thereby improving the stability and accuracy of value function learning. This enables the value network to more reliably predict long-term returns, provides a higher quality benchmark signal for the policy network, and ultimately enhances the convergence and efficiency of the overall training process.

[0014] In other embodiments provided in this application, the proactive strategy includes a target semantic vector, which is used to represent the optimal exploration direction to reach the target group of users; searching for the target keyword corresponding to the proactive strategy in the keyword library includes: searching for the keyword with the highest similarity to the target semantic vector in the keyword library, and determining the found keyword as the target keyword.

[0015] In other embodiments provided in this application, the method further includes: based on Minimize the semantic regularization loss to constrain the target semantic vector output by the active policy network to approximate the semantic distribution of keywords in the keyword corpus, where L semantic z represents the semantic regularization loss. target This represents the target semantic vector output by the active policy network, where K represents the keyword library, and e k This indicates the keyword K in the keyword library. k A vector of keywords.

[0016] The active policy output is abstracted into a target semantic vector, and actions are executed by searching for the most similar keywords in a pre-defined keyword library. This allows the agent's exploration behavior to move beyond direct selection from a fixed keyword list and instead explore a continuous semantic space, enhancing the policy's expressive power and generalization. Simultaneously, a matching mechanism with the keyword library ensures that the generated search instructions are executable and platform-compatible, balancing exploration efficiency with interactive realism.

[0017] In other embodiments provided in this application, the current content includes visual information, audio information, tag information, and comment information of the content; the passive policy is generated using a passive policy network, which includes: inputting the feature vectors corresponding to the visual information, audio information, tag information, and comment information of the current content, as well as the historical interaction sequence before the current content, into the passive policy network to obtain the passive policy output by the passive policy network.

[0018] By fusing multimodal features of content (visual, audio, tags, comments) with historical interaction sequences as state input, passive policy networks can make decisions based on more comprehensive and three-dimensional environmental information. This not only enhances the agent's deep understanding of content semantics but also enables it to provide accurate feedback based on long-term behavioral patterns. As a result, the realism of passive consumption behavior and the rationality of decision-making are significantly improved, providing a more reliable data foundation for building accurate and stable user profiles.

[0019] In other embodiments provided in this application, the recommendation system includes a virtual user simulator, which is used to simulate a real content recommendation platform. After the agent constructs a profile of the target group of users in the recommendation system, the method further includes: running the agent in the real content recommendation platform and collecting information flow data fed back by the real content recommendation platform; calculating the profile forming efficiency, information cocoon index, content concentration, and target group ad loading rate of the real content recommendation platform based on the information flow data; determining the level corresponding to the profile forming efficiency, the level corresponding to the information cocoon index, and the levels corresponding to the content concentration and target group ad loading rate; and generating an evaluation report of the real content recommendation platform according to the actual meaning of the determined levels.

[0020] By introducing a virtual user simulator for low-cost pre-training, followed by deployment and evaluation on a real platform, a complete "simulation training - real application" process is formed. This not only significantly reduces the training costs and risks associated with direct interaction with the real platform, but also allows for a systematic and quantitative evaluation of the platform's performance in multiple dimensions, such as profile convergence speed, information diversity, content distribution, and commercial deployment, based on the collected real data. This provides an objective and comprehensive basis for platform optimization and regulation.

[0021] In other embodiments provided in this application, the profile forming efficiency is the reciprocal of the total number of interaction steps required for the semantic consistency of M consecutive content items to be greater than a threshold; the profile forming efficiency, information cocoon index, content concentration, and target group ad loading rate of the real content recommendation platform are calculated based on information flow data, including: when the profile of the agent in the real content recommendation platform is stable, for multiple consecutive recommended content items, the category of each content item is determined; the probability of each category is calculated; the entropy is calculated based on the probability of each category, and the entropy is determined as the information cocoon index; and / or, the cumulative sum of the probabilities of each category is calculated after sorting them from high to low, the Gini coefficient is calculated based on the cumulative sum, and the Gini coefficient is determined as the information cocoon index.

[0022] Defining profile building efficiency as the reciprocal of the number of convergence steps and using statistical indicators such as entropy or Gini coefficient to quantify the information cocoon effect provides an objective and calculable metric for evaluating recommendation systems. This allows for the precise quantification and comparison of profile building efficiency and algorithmic bias, enabling a scientific and intuitive revelation of the performance of recommendation strategies in terms of user guidance and content diversity. It also provides reliable data support and an evaluation paradigm for algorithm optimization and supervision.

[0023] A second aspect of this application provides an apparatus for constructing a user profile of a group based on an intelligent agent. The apparatus is applied to an intelligent agent interacting with a recommendation system. The intelligent agent includes a policy network, which includes a passive policy network and an active policy network. The apparatus includes: an acquisition module for acquiring feature tags and a keyword library of target group users; a first receiving module for receiving current content recommended by the recommendation system; and a generation module for first generating an active policy based on the current content using the active policy network, and then generating a passive policy using the passive policy network if no active policy is generated. The passive policy indicates the operation to be performed on the current content, and the active policy indicates the operation to be performed in the recommendation system. The system performs the following operations: a search module; an execution module, which executes a passive strategy until the current content is scrolled over, or searches for the target keyword corresponding to the active strategy in the keyword library, searches the recommendation system using the target keyword, and selects content based on the search results; a second receiving module, which receives the next content recommended by the recommendation system; a calculation module, which calculates the reward value corresponding to the semantic consistency between the next content and the feature label, and calculates the penalty value when the active strategy has been executed, and fits the reward value and the penalty value to obtain the fitted value; and an optimization module, which optimizes the strategy network based on the fitted value until the agent constructs a profile of the target group of users in the recommendation system.

[0024] A third aspect of this application provides a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method of the first aspect.

[0025] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of the first aspect.

[0026] The fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the method of the first aspect.

[0027] The apparatus for constructing a group user profile based on an intelligent agent provided in the second aspect of this application, the computer device provided in the third aspect, the computer-readable storage medium provided in the fourth aspect, and the computer program product provided in the fifth aspect have the same or similar beneficial effects as the method for constructing a group user profile based on an intelligent agent provided in the first aspect. Attached Figure Description

[0028] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, with the same or corresponding reference numerals denoteing the same or corresponding parts, wherein: Figure 1This is a schematic diagram of the scenario architecture for the method of constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 1 ; Figure 2 This is a flowchart illustrating the method for constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 1 ; Figure 3 This is a schematic diagram of the scenario architecture for the method of constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 2 ; Figure 4 This is a flowchart illustrating the method for constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 2 ; Figure 5 This is a schematic diagram of the structure of the device for constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 1 ; Figure 6 This is a schematic diagram of the structure of the device for constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 2 ; Figure 7 This is a schematic diagram of the structure of the computer device in the embodiments of this application. Detailed Implementation

[0029] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0030] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0031] Currently, the rigid behavior strategies of intelligent agents based on static state-action mapping tables lead to inefficient interaction with the platform and excessively long user profile construction cycles.

[0032] The root cause of the aforementioned technical problems lies in its reliance on a pre-set, fixed state-action mapping table to drive agent behavior. This static strategy lacks adaptability to dynamic feedback from the recommendation system and an active exploration mechanism. The agent cannot autonomously adjust its behavior strategy based on changes in recommended content during interaction, nor can it quickly guide the recommendation direction through strong signals such as active retrieval. Consequently, it can only gradually approach the target user profile through a large number of slow, passive interactions, severely limiting the efficiency and realism of profile construction.

[0033] In view of this, embodiments of this application provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for constructing group user profiles based on intelligent agents. By constructing a hybrid hierarchical decision-making mechanism including an active policy network and a passive policy network, the intelligent agent can dynamically simulate the complex behavioral patterns of real users. This includes both passive feedback to recommended content (such as liking or watching) and exploratory behavior guided by active retrieval (such as keyword search). Based on the current recommended content and historical interaction states, the intelligent agent first evaluates whether it is necessary to initiate a retrieval to accelerate profile fitting through the active policy network; if not, it generates a consumption action for the current content through the passive policy network. After each interaction, the system rewards the subsequent recommended content based on semantic consistency with the target profile and calculates penalties based on the action execution cost, thereby driving continuous optimization of the policy network. In this way, the intelligent agent can autonomously learn strategies to efficiently approximate the target user profile through continuous interaction with the recommendation system, thereby significantly improving the speed of profile construction and the realism of data collection.

[0034] It should be noted that all components, data, and related processing methods involved in this application are authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0035] First, the scenario architecture of the method for constructing a group user profile based on an intelligent agent provided in the embodiments of this application will be described.

[0036] Figure 1 This is a schematic diagram of the scenario architecture for the method of constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 1 See Figure 1 As shown, the architecture may include: an agent 11 and a recommender system 12.

[0037] Among them, agent 11 includes a policy network. The policy network includes a passive policy network and an active policy network.

[0038] Passive policy networks are used to simulate the direct consumption feedback behavior of real users after receiving content. A passive policy network is a decision function based on a deep neural network. Its input is the current state (features of the current recommended content), and its output is a specific action instruction (passive policy) for the current content. Its goal is to learn a policy that ensures the user behavior sequence generated by its decisions most effectively guides the future content distribution of the recommendation system towards the target user profile.

[0039] Active policy networks simulate the proactive retrieval behavior of real users to satisfy specific information needs or explore interests. The function of an active policy network is not to directly manipulate the current content, but rather to determine, in the current interaction state, whether and how to proactively shape the subsequent content flow of the recommendation system through active input (such as searching). Its goal is to learn to issue efficient search commands at the appropriate time, thereby overcoming the limitations of passively receiving content and significantly accelerating the process of adjusting the recommendation system to reflect the target user profile.

[0040] Recommendation system 12 refers to the external object that the intelligent agent 11 interacts with and obtains content feedback from to build user profiles.

[0041] In practical applications, the recommendation system 12 can be a virtual user simulator or a real content recommendation platform.

[0042] The virtual user simulator is an internally built agent model used to simulate the recommendation logic of a real platform. It is used for large-scale pre-training and rapid iteration of the agent's core policies before deployment to a real platform.

[0043] Authentic content recommendation platforms refer to external, commercial short video or news feed platforms. It is a black-box system where intelligent agents ultimately operate to collect authentic data, build accurate profiles, and conduct a final evaluation of the platform's actual recommendation strategy.

[0044] When this architecture is running, agent 11 receives the current content recommended by recommendation system 12. The active policy network first determines whether to search; if not, the passive policy network determines the action to take on the content (such as liking, swiping, or setting viewing time). Next, agent 11 executes the action or sends search keywords to recommendation system 12. Recommendation system 12 returns the next content based on the received action. Agent 11 calculates the matching degree between this content and the feature tags of the target group users as a reward; if a search has been performed, a penalty is deducted. Finally, the reward and penalty are used to optimize its own policy network, repeating this process until a user profile is built.

[0045] Next, the method for constructing a group user profile based on an intelligent agent, as provided in the embodiments of this application, will be described in detail.

[0046] Figure 2 This is a flowchart illustrating the method for constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 1 See Figure 2 As shown, the method may include: S21: Obtain the feature tags and keyword library of the target group users.

[0047] The target user group refers to a specific set of users who share common demographic characteristics, behavioral patterns, interests, or consumption intentions, such as elderly users, college student game enthusiasts, etc.

[0048] Feature tags are structured semantic identifiers used to abstractly describe the core attributes, interests, preferences, or behavioral patterns of a target user group. For example, feature tags for the elderly user group include: age ≥60, retired, opera, square dancing, health recipes, traditional news, high viewing time, low like rate, active in the morning, etc.

[0049] A keyword database is a structured collection of query terms that targets a specific group and covers their typical search intentions. For example, a keyword database for elderly users might include: Tai Chi instructional videos, dietary precautions for high blood pressure, excerpts from classic Huangmei Opera, and introductory tutorials for square dancing, etc.

[0050] Feature labels are static attribute identifiers used to define and measure the profile of a target group, while keyword libraries are dynamic query tools used to enable agents to actively explore behaviors; the two are not the same.

[0051] In practice, data mining and analysis can be conducted based on historical behavioral data from the platform. For example, topic modeling or cluster analysis can be performed on the interactive content of labeled user groups to extract high-frequency interest topics as feature tags, and the associated typical search queries and comment keywords can be compiled into a keyword library. Alternatively, domain knowledge can be combined with manual definition and enhancement. For example, through market research reports, sociological feature classifications, or expert experience, the feature dimensions of the target group can be directly set, and a list of high-intent keywords matching them can be manually constructed. Then, synonyms can be expanded through word vector models to improve the coverage of the library.

[0052] S22: Receive the current content recommended by the recommendation system.

[0053] The current content here refers to the information items received by the agent in any interaction loop and fed back by the recommendation system in real time. It can be the first recommendation received after the agent starts, or the Nth recommendation in an intermediate state after multiple rounds of interaction. Its specific form is usually multimedia content such as videos, images and text, and products, as well as their associated metadata (such as titles, tags, and categories).

[0054] When an agent first starts up and establishes a connection with the recommendation system, the system typically employs a cold start strategy for initial content recommendations due to a lack of historical interaction data. This process may be based on pre-set basic profile attributes (such as simulated device type and geographic information) during agent initialization, or by directly accessing the platform's global popular content and diversity exploration stream for push notifications.

[0055] In subsequent interactions, the recommendation system enters a dynamic learning and adjustment phase. Each time the agent performs an action on previously recommended content (such as viewing, liking, or initiating a search), the recommendation system continuously inputs these actions as feedback signals into its ranking model. Based on algorithms such as reinforcement learning, collaborative filtering, or deep neural networks, the model updates its estimation of the agent's preferences and re-selects content from the content pool that better matches its inferred interests for the next round of recommendations.

[0056] These recommended contents are the current content.

[0057] S23: Based on the current content, an active policy network is first used to generate an active policy. If no active policy is generated, a passive policy network is then used to generate a passive policy. The passive policy is used to indicate the operation to be performed on the current content, and the active policy is used to indicate the operation to be performed in the recommendation system.

[0058] This is a hierarchical decision-making mechanism of "proactive exploration first, passive consumption as a safety net," which enables intelligent agents to proactively guide the direction of recommendations during interactions to accelerate profile convergence, while preserving natural consumption behavior to ensure the authenticity of interactions and the integrity of data.

[0059] For example, suppose the agent's current goal is to build a profile of "elderly opera enthusiasts." When the recommendation system pushes a video titled "Analysis of Classic Peking Opera Excerpts," the active policy network determines whether active exploration is needed to strengthen the profile. If the active policy network outputs "active exploration is needed," it generates an abstract intent pointing to the search (such as keywords related to "comprehensive collection of local operas"). If the active policy network determines that active search is unnecessary (e.g., the current content already highly matches the profile), the passive policy network generates specific action instructions for the video (such as watching the entire video and liking it).

[0060] An active strategy refers to a decision instruction generated by an agent to proactively guide the content direction of a recommendation system. For example, when an agent simulates an "outdoor sports enthusiast," the active strategy might ultimately map to the search keyword "hiking gear review 2024."

[0061] When generating the proactive strategy, the proactive strategy network makes decisions based on the current state (including the multimodal features of the currently recommended content, historical interaction sequences, and target profile features). Specifically, the state information is first encoded into a high-level representation, and then a binary classification or probability output layer determines whether proactive exploration needs to be initiated. If the output is "needed," a target semantic vector is further generated (e.g., mapped to a continuous vector through a fully connected layer). This vector is then used for nearest neighbor retrieval in a pre-defined keyword database to match the most similar specific keywords as the executable search instruction.

[0062] Passive strategies refer to the consumption behavior decisions made by an agent directly in response to the content currently pushed by the recommendation system. For example, when an agent simulates a "classical music lover," if the recommendation system recommends a symphony video, the passive strategy might be "watch the whole thing and like it"; if it recommends a short pop music video, it might generate instructions such as "scroll away quickly" or "watch only the first 3 seconds."

[0063] When generating a passive policy, the passive policy network performs end-to-end behavior prediction based on the current state (including multimodal features of the currently recommended content, user historical interaction sequences, and real-time context). Specifically, it first integrates multimodal features such as visual, text, and audio data, and encodes them into a state vector by combining historical behavior sequences (such as the actions and durations of the previous 20 interactions). Then, it outputs the probability distribution of discrete actions (such as liking, commenting, and swiping) and the predicted values ​​of continuous parameters (such as the proportion of viewing time) through a multi-layer neural network. Finally, it generates executable actions by sampling based on probability or taking the maximum value.

[0064] S24: Execute the passive strategy until the current content is scrolled over, or search for the target keyword corresponding to the active strategy in the keyword library, and use the target keyword to search in the recommendation system and select content based on the search results.

[0065] If a passive policy is generated, the agent will directly execute the operation specified by the policy in the recommendation system interface (such as liking, swiping, or setting the viewing duration for the current video).

[0066] If an active policy is generated, the agent first performs similarity matching on the abstract intent of the policy output in a pre-set keyword library to retrieve the most relevant specific keywords. Then, it uses these keywords as the query input to the search box of the recommendation system and triggers the search, thereby actively guiding the direction of subsequent content recommendations.

[0067] S25: Receive the next content recommended by the recommendation system.

[0068] Once the agent's actions are received and processed by the recommendation system (such as recording a like or executing a search request), the system re-runs its ranking algorithm based on the updated user state, selecting and pushing the next batch of content most likely to attract the user from the content pool (i.e., the next content). The agent then receives the next content. At this point, a closed-loop interaction chain of "action-feedback" is formed, providing a data foundation for subsequent reward calculations and strategy optimization.

[0069] S26: Calculate the reward value corresponding to the semantic consistency between the next content and the feature label, and calculate the penalty value if the active policy has been executed. Fit the reward value and the penalty value to obtain the fitted value.

[0070] When calculating the reward value, firstly, the next content and the feature label set are encoded into high-dimensional semantic vectors. Then, the cosine similarity or dot product similarity between the two is calculated as the basic reward score. To enhance the discriminative power, a weighted sum can be performed based on the association strength between the content and specific labels (for example, in the category of "elderly group," the "health preservation" label has a higher weight than the "games" label). Finally, the result is normalized to obtain a reward value in the range of [0,1] or [-1,1] to reflect the fit between the content and the target profile.

[0071] The penalty value quantifies the execution cost of proactive strategies, aiming to constrain the agent's abuse of search behavior to simulate the operational costs of real users. When the agent performs a proactive search, a fixed or dynamic negative penalty value is assigned to the search operation according to a pre-set action cost table. Furthermore, the penalty value can be adjusted based on search frequency; for example, the penalty value increases with continuous searches to simulate user fatigue. Alternatively, it can be dynamically penalized based on the relevance of keywords to the target profile, with higher penalties for searching irrelevant keywords. The final penalty value is typically a negative constant or a negative function of variables.

[0072] When finally fitting the reward and penalty values, linear superposition or weighted fusion can be used. The reward and penalty values ​​are combined into a unified instantaneous reward signal (fitted value). That is, fitted value = reward value + penalty value. In complex designs, weighting coefficients can be introduced to balance the influence of the two. That is, fitted value = α * reward value + β * penalty value, where α and β are dynamically adjusted according to the training phase.

[0073] S27: Optimize the policy network based on the fitted values ​​until the agent constructs a profile of the target group of users in the recommendation system.

[0074] When optimizing the policy network, a policy gradient reinforcement learning framework can be followed, specifically implemented using algorithms such as Proximal Policy Optimization (PPO). After each round of interaction, the agent calculates the policy gradient based on the obtained fitted values. The optimizer (such as Adam) calculates the gradient based on the loss function of the policy network and backpropagates it to update the policy network parameters, making it more inclined to choose action sequences that can obtain higher fitted values ​​when facing similar states in the future, thereby systematically improving the efficiency of profile construction.

[0075] This completes one "interaction-learning" cycle. In each cycle, the agent receives the current content from the recommendation system and sequentially performs steps such as state representation, hierarchical decision-making, action execution, receiving feedback (next content), calculating the reward fit value, and network optimization. The optimized policy network will generate new decisions based on the updated parameters in the next round of interaction, and so on. As the cycle progresses, the policy network gradually learns how to quickly guide the content direction through active searching and how to finely adjust the feedback through passive consumption, so that the semantic consistency between the content stream output by the recommendation system and the target feature labels is continuously enhanced until the conditions for profile construction are met, and the cycle terminates.

[0076] In determining when a profile is fully constructed, the semantic similarity between the content stream and the target feature tags is typically based on whether this semantic similarity reaches and remains stable above a preset threshold across multiple rounds of interaction. Specifically, when the average semantic similarity between the recommended content received by the agent in N consecutive (e.g., N=50) recommendations and the target feature tags exceeds a threshold τ (e.g., τ=0.85), and the fluctuation range is less than a certain tolerance (e.g., standard deviation <0.05), the profile is considered to have been stably constructed.

[0077] As can be seen from the above, the method for constructing user profiles based on intelligent agents provided in this application introduces a policy network (active / passive policy network) with a hierarchical decision-making mechanism, enabling the intelligent agent to flexibly simulate the complex interaction patterns of real users (i.e., including both passive consumption behavior and active exploration behavior based on search). It effectively utilizes the high-value signal of active retrieval, and can quickly guide the feedback of the recommendation system to converge towards the preferences of the target group, thereby significantly accelerating the user profile construction process. This solves the problems of low interaction efficiency and excessively long profile construction cycle caused by traditional static behavior strategies, and provides a high-quality data foundation for subsequent algorithm evaluation.

[0078] Furthermore, as a response to Figure 2 In a refinement and extension of the method shown, this application embodiment also provides a method for constructing a group user profile based on an intelligent agent.

[0079] Figure 3 This is a schematic diagram of the scenario architecture for the method of constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 2 See Figure 3 As shown, the architecture may include: an intelligent agent 11, a virtual user simulator 121, and a real content recommendation platform 122.

[0080] Among them, agent 11 includes a policy network and a value network.

[0081] A value network's function is to evaluate the expected long-term cumulative reward obtained by following the current policy in a given state. Specifically, the value network takes the environmental state (such as a fusion representation of current recommended content and historical interactions) as input and maps it to a scalar value through a deep neural network. This value represents the expected total reward from that state until the task ends (such as completing profile construction). The value network provides a benchmark reference for the policy network's decision-making, thereby helping the policy network to perform gradient updates more stably and efficiently, reducing variance in the learning process and accelerating convergence.

[0082] The virtual user simulator 121 is used in the offline training and rapid iteration phase of the intelligent agent 11. By simulating the core feedback logic of the real content recommendation platform 122, it provides the intelligent agent 11 with a low-cost, highly controllable, and accelerated synthetic training environment. This allows the intelligent agent 11 to initially learn the basic behavioral patterns of the target user profile without directly interacting with the real content recommendation platform 122, significantly reducing training time and economic costs, and avoiding frequent attempts to violate platform security policy restrictions.

[0083] The real-content recommendation platform 122 is used in the online verification and final evaluation phase of agent 11. After agent 11 has been trained to initial convergence in the simulation environment, it is deployed to the real-content recommendation platform 122. By interacting with the actual recommendation system, it collects information flow data in a real environment and calculates core evaluation indicators such as profile formation efficiency and information cocoon index based on this data. This allows for an objective and reliable multi-dimensional quantitative evaluation of the real performance of the recommendation algorithm and provides a practical basis for platform optimization, algorithm auditing, or supervision.

[0084] During operation, agent 11 generates action decisions (active search or passive consumption) based on current content and historical states through its policy network. The value network evaluates the current state and predicts long-term returns to assist in policy optimization. During the training phase, agent 11 primarily interacts with a virtual user simulator 121. After receiving actions, the virtual user simulator 121 simulates the platform's feedback to generate the next content, which agent 11 uses to update its policy and value networks. During the evaluation phase, agent 11 interacts with a real content recommendation platform 122. After receiving actions, the real content recommendation platform 122 returns the next real content; agent 11 collects data and calculates evaluation metrics, but does not update network parameters.

[0085] Figure 4 This is a flowchart illustrating the method for constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 2 See Figure 4 As shown, the method may include: S41: Obtain the feature tags and keyword library of the target group users.

[0086] S42: Receive the current content recommended by the recommendation system.

[0087] S43: Input the feature vectors corresponding to the visual information, audio information, tag information, and comment information of the current content, as well as the historical interaction sequence before the current content, into the active policy network. If no active policy is generated, input the feature vectors corresponding to the visual information, audio information, tag information, and comment information of the current content, as well as the historical interaction sequence before the current content, into the passive policy network to obtain the passive policy output by the passive policy network.

[0088] Visual information refers to the visual features carried in content through images or video frames, such as the color of food, the dynamic scenes of the cooking process, and the layout of the scene in a food video.

[0089] Audio information refers to sound signals such as speech, music, and ambient sounds that accompany the content, such as the instructor's voice instructions and the rhythm of background music in a fitness tutorial.

[0090] Tag information refers to category identifiers or topic keywords that are manually or automatically labeled and associated with the content. For example, a travel video may be tagged with "island scenery", "independent travel", "travel guide" etc.

[0091] Comment information refers to the text of user comments on the content, such as keywords like "high cost performance" and "poor battery life" that frequently appear in the comments below a tech product review video.

[0092] The corresponding feature vectors are as follows: visual feature vectors are extracted from video frames or thumbnails through forward propagation using a pre-trained convolutional neural network (such as ResNet). Audio feature vectors are extracted by performing Mel-spectrum transformation on audio tracks using an audio classification model (such as VGGish) to extract deep feature vectors. Tag feature vectors encode text tags into semantic feature vectors using a word embedding model (such as Word2Vec or BERT). Comment feature vectors are generated by aggregating comment text (such as average pooling) and then converting it into a holistic semantic feature vector using a sentence encoder (such as Sentence-BERT).

[0093] Historical interaction sequences refer to a set of previous state-action pairs and their feedbacks recorded chronologically by the agent during its interaction with the recommendation system. For example, suppose the agent simulates a "fitness enthusiast," its historical interaction sequence might include: in the first 10 interactions, performing "watch and like" the "bodyweight training tutorial" video, receiving a high reward of 0.9; performing "quickly swipe away" the "dessert ad" video, receiving a negative reward of -0.3; and most recently, initiating a keyword search for "protein powder review" and receiving new content "muscle-building diet guide."

[0094] After the feature vectors and historical interaction sequences are input into the policy network, the network first fuses and encodes the multimodal features (visual, audio, label, and comment vectors) and historical interaction sequences through multi-layer fully connected or attention mechanisms to extract high-level semantic representations. These representations are then mapped to action probability distributions or specific parameter values ​​through the output layer of the policy network (e.g., softmax layers for discrete actions and linear layers for continuous parameters). For active policy networks, the output is typically a binary decision (whether to search) and an associated search intent vector. For passive policy networks, the output is an instruction to operate on the current content (e.g., liking, viewing time percentage, etc.).

[0095] The proactive strategy includes a target semantic vector, which represents the optimal exploration direction to reach the target user group. The target semantic vector is a continuous vector generated by the proactive strategy network. In the high-dimensional semantic space, it points to the exploration direction that is most likely to accelerate the convergence of the user profile. By matching its similarity with a preset keyword library, it can be transformed into a specific search query that the platform can execute.

[0096] S44: Execute a passive strategy until the current content is scrolled over, or search for the keyword with the highest similarity to the target semantic vector in the keyword library, determine the found keyword as the target keyword, and use the target keyword to search in the recommendation system and select content based on the search results.

[0097] If the output is an active strategy, it indicates that an active search is required. Before an active search, the target keywords need to be determined. During the query and determination process, the cosine similarity between the target semantic vector and the corresponding vector of each keyword in the keyword library is first calculated. Then, the keyword with the highest similarity is selected as the target keyword. Finally, the agent uses this target keyword as the search query input to the recommendation system's search interface, triggering the platform's content retrieval and ranking mechanism. This retrieves a batch of highly relevant content as the subsequent recommendation stream, achieving the goal of quickly guiding the system's output direction through active exploration.

[0098] S45: Receive the next content recommended by the recommendation system.

[0099] S46: Calculate the reward value corresponding to the semantic consistency between the next content and the feature label, and calculate the penalty value if the active policy has been executed. Fit the reward value and the penalty value to obtain the fitted value.

[0100] S47: Use a value network to generate a first foreground value for the current content, and use a value network to generate a second foreground value for the next content.

[0101] Among them, the prospect value is used to characterize the expectation of the decision network to complete the construction of the target group user profile based on the current and subsequent interactions.

[0102] In the context of reinforcement learning, the foreground value specifically refers to the state value function, which represents the mathematical expectation of the cumulative future discount reward that can be obtained starting from a specific state (such as the complete environmental state when receiving the current or next recommended content) while following the current policy. This value not only includes the immediate reward of the current step but also encompasses the contribution of all subsequent interaction steps to the ultimate goal of completing the user profile. Therefore, it quantifies the potential value or quality of a state in achieving the long-term goal. In the user profile building task, a high foreground value means that starting from this state, the agent has a high probability of efficiently and accurately converging to the target user profile. A low foreground value indicates that the state may deviate from the goal or require more interaction costs.

[0103] When generating the first foreground value, the multimodal features of the current content and the historical interaction sequence are fused and encoded into a state vector, which is then input into the value network. The network outputs a scalar value through forward propagation. This value is the expected cumulative discount reward that can be obtained from the current state to the end of the task according to the current policy, i.e., the first foreground value.

[0104] When generating the second foreground value, the next content fed back by the recommendation system after the agent performs an action (passive operation or active search) and its updated historical sequence are encoded into a new state vector in the same way, input into the same value network for forward propagation, and the output scalar is the expected cumulative discount reward starting from this new state, i.e., the second foreground value.

[0105] S48: Generate dominance values ​​based on fitted values, first foreground values, and second foreground values.

[0106] The advantage value is used to characterize the potential of the next piece of content to reach the target user profile compared to the current content.

[0107] In reinforcement learning, the advantage value is defined as the additional value gain brought by a specific action. It is calculated by comparing the expected gain of the new state after performing the action (second foreground value) with the expected gain of the original state (first foreground value) and combining it with the immediate reward (fit value).

[0108] Specifically, the advantage value reflects the long-term gains from taking an action (such as searching or liking) in the current state, compared to the expected improvement from maintaining the original strategy, when transitioning to a new state. A positive value indicates that the action helps achieve the target profile faster and better, while a negative value indicates that the action may deviate from the target or be less efficient.

[0109] Specifically, step S48 above may include: based on A t =R t +γ*V(s {t+1} )-V(s t), calculate the advantage value.

[0110] Among them, A t R represents the dominance value. t V(s) represents the fitted value, γ represents the discount factor, and V(s) represents the value fitted. {t+1} V(s) represents the second foreground value. t ) represents the first foreground value.

[0111] Except for the time-difference form of A t =R t +γ*V(s {t+1} )-V(s t In addition to generalized advantage estimation (GAE), the advantage value can also be calculated using the Monte Carlo method. This method combines multi-step time-series differencing to balance the bias and variance of the estimation. Alternatively, the advantage value can be directly calculated using the Monte Carlo method, yielding the difference between the cumulative reward of the complete trajectory and the actual value function estimate.

[0112] S49: Base Optimize the policy network until the agent constructs a profile of the target group of users in the recommendation system.

[0113] Where L(θ) represents the policy network loss function, r t (θ) represents the importance sampling ratio. Indicates the advantage value.

[0114] After each round of interaction, the agent generates an action based on the current policy, obtains a reward fit value and a new state, and calculates the advantage value. Then, the policy network parameters are updated according to the PPO loss function, making it more inclined to choose actions with high advantage values ​​in subsequent interactions. The updated policy network continues to interact with the environment, collect new data, and repeat the optimization steps. This process is repeated continuously, with the policy constantly improving itself, until the content stream received by the agent stably matches the characteristics of the target user profile, at which point the profile is considered complete, and the loop terminates.

[0115] To ensure the practical feasibility of the target semantic vector generated by the active policy network and prevent the vector from deviating from the effective semantic range of the keyword library, thus causing search failure, a semantic regularization loss is added to the training loss function. This forces the target semantic vector output by the network to be as close as possible in vector space to the vector representation of a keyword in the predefined keyword library, thereby ensuring that the search intent proposed by the agent can be accurately mapped to a specific query term that the platform can recognize.

[0116] Specifically, step S49 above may include: based on Optimize the policy network.

[0117] Where λ represents the weighting factor, L semantic This represents the semantic regularization loss.

[0118] Based on the standard PPO objective function, a semantic regularization loss term is added. This primarily maximizes the long-term cumulative reward (i.e., efficiently constructing a profile) through the advantage function, and secondly ensures that the target semantic vector output by the active policy network falls within the effective semantic range of the keyword library (i.e., executability). This aims to prevent the network from outputting meaningless search intentions, thereby guaranteeing the effectiveness and realism of the active exploration behavior and making the training process more stable and reliable.

[0119] Specifically, step S49 above may include: based on Minimize the semantic regularization loss to constrain the target semantic vector output by the active policy network to approximate the semantic distribution of keywords in the keyword corpus. Among them, L semantic z represents the semantic regularization loss. target This represents the target semantic vector output by the active policy network, where K represents the keyword library, and e k This indicates the keyword K in the keyword library. k A vector of keywords.

[0120] In other words, among all keyword vectors in the keyword library, the one with the closest Euclidean distance is found, and the square of this minimum distance is used as the loss value. By minimizing this loss, the network is constrained during training, causing its output intent vector to be pulled into the effective semantic space spanned by the keyword library. This ensures that the search intent proposed by the agent can always be mapped to specific keywords that the platform can understand and retrieve, avoiding the generation of invalid or unexecutable query instructions.

[0121] In each round of policy network optimization, in order to improve the accuracy of the value network's prediction of future returns and thus provide a more reliable advantage value estimate for the policy network, a stable target value can be generated using the old parameter value network (V_old). This target value is then compared with the current value network (V)'s prediction of the prospect value for the same state. The mean squared error is calculated as the loss, and the value network parameters are updated through gradient descent to make its prediction closer to the true long-term return expectation.

[0122] S410: Based on Target=R t +γ*V old (s {t+1} ), calculate the target value, where Target represents the target value, R t V represents the fitted value, γ represents the discount factor, and V old (s {t+1} The second foreground value is represented by the target value; the first foreground value is subtracted from the target value to obtain the error; the value network is optimized based on the error.

[0123] When optimizing a value network based on error, the difference between the target value and the current value network's prediction of the state is first calculated as the temporal difference error. Then, the square of this error is used as the loss function of the value network. Finally, the gradient of the loss function with respect to the network parameters is calculated using the gradient descent algorithm, and the parameters in the value network are updated through backpropagation. This allows the value network's prediction of the state value to gradually approach the target value generated based on the old network, thereby improving the accuracy and stability of its long-term return estimation.

[0124] The training of both the policy network and the value network in the above-mentioned intelligent agent is carried out in a virtual user simulator. The virtual user simulator is used to simulate a real content recommendation platform. After the intelligent agent completes the construction of the target group user profile in the virtual user simulator (the intelligent agent can already imitate the target group users very well), it can be deployed to the real content recommendation platform to evaluate the recommendation algorithm of the real content recommendation platform through the intelligent agent.

[0125] It should be noted that the number of agents can be flexibly configured according to the evaluation needs. A single agent can be deployed to conduct in-depth simulation and evaluation for a specific user group, or multiple heterogeneous agents (i.e., different types, such as simulating the elderly, college students, new mothers, etc.) can be deployed in parallel to achieve multi-dimensional comparison and stress testing of the recommendation system's performance on different groups.

[0126] S411: Employ an intelligent agent to run within a real content recommendation platform and collect information flow data from the platform; calculate the profile formation efficiency, information cocoon index, content concentration, and target group ad loading rate of the real content recommendation platform based on the information flow data; determine the corresponding levels for profile formation efficiency, information cocoon index, content concentration, and target group ad loading rate; and generate an evaluation report for the real content recommendation platform based on the practical significance of the determined levels.

[0127] At this point, the agent, acting as a "simulated user" account, interacts realistically with the platform. Throughout this process, the agent continuously executes its learned hybrid strategy (active search and passive consumption) and fully records all the information stream data returned in real-time by the platform based on each of its actions, including the original logs such as the identification document (ID) of the recommended content, its type, tags, exposure duration, and whether it was clicked.

[0128] User profile building efficiency is a quantitative metric that measures how quickly an agent can successfully build a stable and accurate user profile of the target group from its initial state (cold start) during interaction with a recommendation system. It is typically measured as the reciprocal of the total number of interaction steps required for the semantic consistency of M consecutive pieces of content to exceed a threshold. That is, first, a threshold standard for determining whether a profile is "formed" is set (e.g., the average semantic consistency of M consecutive recommended content with the target feature tags exceeds a threshold τ). Then, the total number of interaction steps required for the agent to first meet this standard from the start of interaction is recorded. Tconverge Finally, calculate EPC =1 / Tconverge The larger this value, the faster the agent builds its profile, and the more efficient the recommendation system's understanding and response to that group.

[0129] The information cocoon index is a metric that quantifies the diversity of content provided by a recommendation system to a specific user group. It is used to assess whether the algorithm exhibits an "information cocoon" effect, where it excessively reinforces content that users are already interested in, leading to an extremely narrow range of content. Specifically, it can be determined using entropy, the Gini coefficient, or a combination thereof.

[0130] Specifically, when the profile of the intelligent agent in the real content recommendation platform is stable, for multiple consecutive recommended contents, the category of each content is determined; the probability of each category is calculated; the entropy is calculated based on the probability of each category, and the entropy is determined as the information cocoon index; and / or, the cumulative sum of the probabilities of each category is calculated after sorting them from high to low, the Gini coefficient is calculated based on the cumulative sum, and the Gini coefficient is determined as the information cocoon index.

[0131] Profile stability refers to the state in which, during the interaction between the agent and the recommendation system, the semantic matching degree between the content stream received and the target group's feature tags has reached and is maintained at a high level of confidence. The specific criterion is usually set as follows: in a certain number of consecutive recommendations, the average semantic consistency score between the content and the target feature tags exceeds a preset threshold, and its fluctuation is below a certain tolerance range.

[0132] Categories refer to standardized classification tags used by recommendation systems to divide information items (such as videos, articles, and products) in their content pool according to theme, domain, or function. They are the basic metadata used by the platform for content management and interest matching. For example, on short video platforms, categories include "funny," "food tutorials," "sports events," "beauty and skincare," and "news and current events"; on e-commerce platforms, categories include "mobile phones and digital products," "home appliances," "clothing and shoes," and "food and beverages." Each recommended content item is typically assigned one or more category tags. By statistically analyzing the distribution of these categories in the information stream, the concentration and diversity of the recommendation results can be quantified.

[0133] When calculating the probability of each category, first count the frequency of each category within a selected observation window (e.g., N consecutive recommended content items). Then, divide the frequency of each category by the total number of content items N to obtain the probability of that category appearing in the recommendation stream.

[0134] When determining the information cocoon index based on entropy, the category probability distribution is substituted into the information entropy formula. H =−∑ i =1 Cpi log2 pi Perform calculations, where C This represents the total number of categories that appeared. (Calculation result) H This is known as the information cocoon index. The lower the entropy value, the more concentrated the content categories are, the worse the diversity, and the stronger the information cocoon effect. The higher the entropy value, the more even the content distribution and the better the diversity.

[0135] When determining the information cocoon index based on the Gini coefficient, first classify each category according to its probability. pi Sort from highest to lowest to obtain the sequence. p (1)≥ p (2)≥...≥ p ( C ), and calculate its cumulative sum. Sj =∑ k =1 jp ( k Then, using the Gini coefficient formula... G =1−∑ i =1 CC 2 i -1⋅ p ( i Perform the calculation. Calculation results. G This is the information cocoon index based on the Gini coefficient. A higher coefficient indicates a more unequal distribution of content, with content concentrated in a few categories, and a more significant information cocoon effect. A lower coefficient indicates a more uniform distribution of content.

[0136] Content concentration is typically calculated by the cumulative probability percentage of the top-K most frequent categories in the recommendation feed. For example, the total occurrence ratio of the top 3 categories. The higher this value, the more concentrated the content.

[0137] The target audience ad load rate is calculated in the recommendation feed as the percentage of ad content strongly related to the target audience's characteristics out of the total recommended content. For example, when simulating the "elderly group," the proportion of health product and insurance ads would be this metric.

[0138] The evaluation indicators obtained at this point can be output directly, or they can be further processed before output.

[0139] In the levels corresponding to profile formation efficiency, information cocoon index, content concentration, and target group ad load rate, the levels are discrete segmented evaluations of quantitative assessment indicators. They are usually divided into a limited number of levels, such as "excellent, good, average, poor" or "high, medium, low," with each level corresponding to a numerical range. The purpose is to transform continuous indicator values ​​into intuitive and easy-to-understand qualitative evaluations, facilitating quick comprehension and decision-making.

[0140] The profile formation efficiency level indicates that the higher the level, the faster and more accurately the system can identify and meet the interests of the target user, and the better the cold start performance.

[0141] The Information Cocoon Index is a rating system. The higher the rating, the worse the content diversity and the greater the risk of users becoming trapped in information asymmetry.

[0142] Content concentration level: the higher the level, the more singular the theme of the recommended content and the less exploratory it is.

[0143] For the platform, a high ad load rate level for the target audience may indicate precise commercial targeting, while for users, an excessively high level may represent strong intrusion.

[0144] The core of generating the evaluation report is to transform the qualitative interpretation of each indicator's rating into a comprehensive diagnosis and recommendations for platform performance. The report integrates all ratings and their practical significance to form a structured conclusion. First, it outlines the platform's overall performance rating in areas such as user profile building efficiency, content diversity, theme concentration, and the accuracy of commercial placement. Then, it analyzes the strengths and risks item by item, and finally, based on the meaning of the ratings, proposes targeted improvement directions. For example, if a platform receives a rating of "Excellent Efficiency, High Cocoon Index, High Ad Loading Rate" in the "Elderly Users" assessment, the report conclusion could be: "The platform can quickly serve elderly users, but there is a risk of information narrowing, and ad overload. It is recommended to optimize the content exploration mechanism and adjust the ad frequency." This concludes the description of the method for constructing a group user profile based on an intelligent agent, as provided in the embodiments of this application.

[0145] Based on the same inventive concept, embodiments of this application also provide an apparatus for constructing a group user profile based on an intelligent agent.

[0146] The device for constructing user profiles based on intelligent agents is applied to intelligent agents that interact with recommendation systems. The intelligent agent includes a policy network, which includes a passive policy network and an active policy network.

[0147] Figure 5 This is a schematic diagram of the structure of the device for constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 1 See Figure 5 As shown, the device may include: The acquisition module 51 is used to acquire the feature tags and keyword library of the target group users.

[0148] The first receiving module 52 is used to receive the current content recommended by the recommendation system.

[0149] The generation module 53 is used to first generate an active policy based on the current content using an active policy network, and then generate a passive policy using a passive policy network if no active policy is generated. The passive policy is used to indicate the operation to be performed on the current content, and the active policy is used to indicate the operation to be performed in the recommendation system.

[0150] The execution module 54 is used to execute a passive strategy until the current content is scrolled over, or to search for the target keyword corresponding to the active strategy in the keyword library, and to use the target keyword to search in the recommendation system and select content based on the search results.

[0151] The second receiving module 55 is used to receive the next content recommended by the recommendation system.

[0152] The calculation module 56 is used to calculate the reward value corresponding to the semantic consistency between the next content and the feature label, and to calculate the penalty value when the active policy has been executed, and to fit the reward value and the penalty value to obtain the fitted value.

[0153] Optimization module 57 is used to optimize the policy network based on the fitted values ​​until the agent constructs a profile of the target group of users in the recommendation system.

[0154] Furthermore, as a response to Figure 5 In addition to the refinement and expansion of the illustrated device, this application embodiment also provides a device for constructing a group user profile based on an intelligent agent.

[0155] Figure 6 This is a schematic diagram of the structure of the device for constructing a group user profile based on an intelligent agent in the embodiments of this application. Figure 2 See Figure 6 As shown, the device may include: The acquisition module 61 is used to acquire the feature tags and keyword library of the target group users.

[0156] The first receiving module 62 is used to receive the current content recommended by the recommendation system.

[0157] The generation module 63 is used to first generate an active policy based on the current content using an active policy network, and then generate a passive policy using a passive policy network if no active policy is generated. The passive policy is used to indicate the operation to be performed on the current content, and the active policy is used to indicate the operation to be performed in the recommendation system.

[0158] Given that the current content includes visual information, audio information, tag information, and comment information, the generation module 63 is specifically used to input the feature vectors corresponding to the visual information, audio information, tag information, and comment information of the current content, as well as the historical interaction sequence before the current content, into the passive policy network to obtain the passive policy output by the passive policy network.

[0159] In the case where the active strategy includes a target semantic vector, which is used to represent the optimal exploration direction to reach the target group of users, the generation module 63 is specifically used to find the keyword with the highest similarity to the target semantic vector in the keyword library, and to determine the found keyword as the target keyword.

[0160] The execution module 64 is used to execute a passive strategy until the current content is scrolled over, or to search for the target keyword corresponding to the active strategy in the keyword library, and to use the target keyword to search in the recommendation system and select content based on the search results.

[0161] The second receiving module 65 is used to receive the next content recommended by the recommendation system.

[0162] The calculation module 66 is used to calculate the reward value corresponding to the semantic consistency between the next content and the feature label, and to calculate the penalty value when the active policy has been executed, and to fit the reward value and the penalty value to obtain the fitted value.

[0163] Optimization module 67 is used to optimize the policy network based on the fitted values ​​until the agent constructs a profile of the target group of users in the recommendation system.

[0164] When the agent also includes a value network, the optimization module 67 is specifically used to generate a first prospect value for the current content using the value network, and a second prospect value for the next content using the value network. The prospect value represents the expectation of the decision network to complete the construction of the target group user profile based on current and subsequent interactions. Based on the fitted value, the first prospect value, and the second prospect value, an advantage value is generated. The advantage value represents the magnitude of the potential for the next content to achieve the target group user profile compared to the current content. Optimize the policy network, where L(θ) represents the policy network loss function, r t (θ) represents the importance sampling ratio. Indicates the advantage value.

[0165] Optimization module 67, specifically used for A-based t =R t +γ*V(s {t+1} )-V(s t ), calculate the dominance value, where A t R represents the dominance value.t V(s) represents the fitted value, γ represents the discount factor, and V(s) represents the value fitted. {t+1} V(s) represents the second foreground value. t ) represents the first foreground value.

[0166] Optimization module 67 is also used for Target=R t +γ*V old (s {t+1} ), calculate the target value, where Target represents the target value, R t V represents the fitted value, γ represents the discount factor, and V old (s {t+1} The second foreground value is represented by the target value; the first foreground value is subtracted from the target value to obtain the error; the value network is optimized based on the error.

[0167] Optimization module 67 is also used for base Minimize the semantic regularization loss to constrain the target semantic vector output by the active policy network to approximate the semantic distribution of keywords in the keyword corpus, where L semantic z represents the semantic regularization loss. target This represents the target semantic vector output by the active policy network, where K represents the keyword library, and e k This indicates the keyword K in the keyword library. k A vector of keywords.

[0168] In a recommendation system that includes a virtual user simulator to simulate a real content recommendation platform, evaluation module 68 is used to employ an intelligent agent to operate within the real content recommendation platform and collect information flow data fed back by the real content recommendation platform. Based on the information flow data, it calculates the profile formation efficiency, information cocoon index, content concentration, and target group ad loading rate of the real content recommendation platform; determines the level corresponding to profile formation efficiency, the level corresponding to information cocoon index, and the levels corresponding to content concentration and target group ad loading rate; and generates an evaluation report of the real content recommendation platform based on the practical significance of the determined levels.

[0169] The efficiency of image creation is the reciprocal of the total number of interaction steps required for the semantic consistency of M consecutive content items to be greater than a threshold.

[0170] Evaluation module 68 is specifically used to determine the category of each content for subsequent recommended content when the profile of the intelligent agent in the real content recommendation platform is stable; calculate the probability of each category; calculate the entropy based on the probability of each category and determine the entropy as the information cocoon index; and / or calculate the cumulative sum of the probabilities of each category sorted from high to low, calculate the Gini coefficient based on the cumulative sum and determine the Gini coefficient as the information cocoon index.

[0171] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0172] Based on the same inventive concept, this application also provides a computer device.

[0173] Figure 7 This is a schematic diagram of the structure of the computer device in an embodiment of this application. See also... Figure 7 As shown, the computer device may include: a memory 71, a processor 72, and a computer program stored on the memory 71, wherein the processor 72 executes the computer program to implement the methods described in the foregoing embodiments.

[0174] It should be noted that the description of the above computer device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the computer device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0175] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the methods described in the foregoing embodiments.

[0176] It should be noted that the description of the above computer-readable storage medium embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the computer-readable storage medium embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0177] Based on the same inventive concept, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the methods described in the foregoing embodiments.

[0178] It should be noted that the descriptions of the above computer program product embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the computer program product embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0179] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for constructing user profiles for a group based on intelligent agents, characterized in that, The method is applied to an agent interacting with a recommendation system, the agent including a policy network, the policy network including a passive policy network and an active policy network, the method comprising: Obtain the characteristic tags and keyword database of the target user group; Receive the current content recommended by the recommendation system; Based on the current content, an active policy is first generated using the active policy network. If no active policy is generated, a passive policy is then generated using the passive policy network. The passive policy is used to indicate the operation to be performed on the current content, and the active policy is used to indicate the operation to be performed in the recommendation system. The passive strategy is executed until the current content is scrolled over, or the target keyword corresponding to the active strategy is searched in the keyword library, and the target keyword is used to search in the recommendation system and to select content based on the search results; Receive the next content recommended by the recommendation system; Calculate the reward value corresponding to the semantic consistency between the next content and the feature label, and calculate the penalty value if the active policy has been executed, and fit the reward value and the penalty value to obtain the fitted value; The policy network is optimized based on the fitted values ​​until the agent constructs a profile of the target group of users in the recommendation system.

2. The method according to claim 1, characterized in that, The agent further includes a value network; optimizing the policy network based on the fitted value includes: The value network is used to generate a first prospect value for the current content and a second prospect value for the next content, wherein the prospect value is used to characterize the expectation of the decision network to complete the construction of the target group user profile based on the current and subsequent interactions; Based on the fitted value, the first foreground value, and the second foreground value, an advantage value is generated, wherein the advantage value is used to characterize the foreground size of the next content compared to the current content in achieving the target group user profile; based on Optimize the policy network, where L(θ) represents the policy network loss function, r t (θ) represents the importance sampling ratio. Indicates the advantage value.

3. The method according to claim 2, characterized in that, The step of generating a dominance value based on the fitted value, the first foreground value, and the second foreground value includes: Based on A t =R t +γ*V(s {t+1} )-V(s t ), calculate the aforementioned advantage value, Among them, A t R represents the dominance value. t V(s) represents the fitted value, γ represents the discount factor, and V(s) represents the value fitted. {t+1} V(s) represents the second foreground value. t ) represents the first foreground value.

4. The method according to claim 2, characterized in that, The method further includes: Based on Target=R t +γ*V old (s {t+1} ), calculate the target value, where Target represents the target value, R t V represents the fitted value, γ represents the discount factor, and V old (s {t+1} Indicates the second foreground value; The error is obtained by subtracting the first foreground value from the target value; The value network is optimized based on the error.

5. The method according to claim 2, characterized in that, The proactive strategy includes a target semantic vector, which represents the optimal exploration direction to reach the target user group; the step of searching for the target keyword corresponding to the proactive strategy in the keyword library includes: The keyword with the highest similarity to the target semantic vector is searched in the keyword library, and the found keyword is determined as the target keyword.

6. The method according to claim 5, characterized in that, The method further includes: based on Minimize the semantic regularization loss to constrain the target semantic vector output by the active policy network to approximate the semantic distribution of keywords in the keyword library. Among them, L semantic z represents the semantic regularization loss. target This represents the target semantic vector output by the active policy network, where K represents the keyword library, and e k This indicates the keyword K in the keyword library. k A vector of keywords.

7. The method according to any one of claims 1 to 4, characterized in that, The current content includes visual information, audio information, tag information, and comment information; The process of generating a passive policy using the passive policy network includes: The passive policy network is input with the feature vectors corresponding to the visual information, audio information, tag information, and comment information of the current content, as well as the historical interaction sequence before the current content, to obtain the passive policy output by the passive policy network.

8. The method according to any one of claims 1 to 4, characterized in that, The recommendation system includes a virtual user simulator, which is used to simulate a real content recommendation platform. After the agent constructs a profile of the target group of users in the recommendation system, the method further includes: The intelligent agent is used to run in a real content recommendation platform and collects information flow data fed back by the real content recommendation platform; Based on the information flow data, the profile formation efficiency, information cocoon index, content concentration, and target group ad loading rate of the real content recommendation platform are calculated. Determine the level corresponding to the profile forming efficiency, the level corresponding to the information cocoon index, and the level corresponding to the content concentration and the target group ad loading rate; Based on the actual meaning corresponding to the determined levels, an evaluation report of the real content recommendation platform is generated.

9. The method according to claim 8, characterized in that, The profile forming efficiency is the reciprocal of the total number of interaction steps required for the semantic consistency of M consecutive content items to be greater than a threshold; the calculation of the profile forming efficiency, information cocoon index, content concentration, and target group ad loading rate of the real content recommendation platform based on the information flow data includes: When the profile of the intelligent agent in the real content recommendation platform is stable, the category of each content is determined for multiple consecutive recommended contents. Calculate the probability of each category; Calculate the entropy based on the probability of each category, and determine the entropy as the information cocoon index; and / or calculate the cumulative sum of the probabilities of each category sorted from high to low, calculate the Gini coefficient based on the cumulative sum, and determine the Gini coefficient as the information cocoon index.

10. A device for constructing user profiles for a group based on intelligent agents, characterized in that, The device is applied to an agent that interacts with a recommendation system. The agent includes a policy network, which includes a passive policy network and an active policy network. The device includes: The acquisition module is used to acquire the feature tags and keyword library of the target group of users; The first receiving module is used to receive the current content recommended by the recommendation system; The generation module is used to first generate an active policy based on the current content using the active policy network, and then generate a passive policy using the passive policy network if the active policy is not generated. The passive policy is used to indicate the operation to be performed on the current content, and the active policy is used to indicate the operation to be performed in the recommendation system. An execution module is used to execute the passive strategy until the current content is scrolled over, or to search for the target keyword corresponding to the active strategy in the keyword library, and to use the target keyword to search in the recommendation system and select content based on the search results; The second receiving module is used to receive the next content recommended by the recommendation system; The calculation module is used to calculate the reward value corresponding to the semantic consistency between the next content and the feature label, and to calculate the penalty value when the active policy has been executed, and to fit the reward value and the penalty value to obtain the fitted value; An optimization module is used to optimize the policy network based on the fitted values ​​until the agent constructs a profile of the target group of users in the recommendation system.

Images