Methods, apparatus, devices, and products for determining the number of target users
By acquiring information about the broadcaster and the environment to generate state features, and using machine learning models to predict the broadcaster's service capacity, the problem of uneven traffic distribution in the live streaming recommendation system is solved, thereby improving the user and broadcaster experience and unlocking long-tail value.
Patent Information
- Application Number
- CN202510626312.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-15
- Publication Date
- 2026-11-17
AI Technical Summary
The Matthew effect exists in the distribution of traffic in the live streaming recommendation system, which leads to the concentration of traffic in the hands of top streamers. Small and medium-sized streamers find it difficult to break through the traffic barrier, and the service capacity of streamers is not effectively modeled, resulting in a decrease in user experience as the number of viewers increases.
By acquiring information about the broadcaster and the environment, state features are generated, and machine learning models are used to determine the number of target users. Based on deep learning networks and reinforcement learning models, the service capacity of the broadcaster is predicted, and the number of users is adjusted to optimize traffic allocation.
It improved the accuracy and generalization of target user number prediction, mitigated the Matthew effect, enhanced the experience for both broadcasters and users, and efficiently tapped into long-tail value.
Smart Images

Figure CN123123595A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computers, and more specifically to methods, apparatus, devices, and products for determining the number of target users. Background Technology
[0002] With the rapid development of live streaming services, the live streaming function of social media platforms has become an indispensable core module in the digital ecosystem. Live streaming platforms are not only carriers of entertainment and social interaction, but have also evolved into super gateways connecting content, goods, and users. However, the contradiction between the massive amount of live streaming content and users' personalized needs is becoming increasingly prominent. Live streaming recommendation systems, as a key technology for matching users with streamers, quickly and efficiently help users find streamers of interest from a vast amount of information, thereby ensuring a positive user experience when watching live streams on the platform. Summary of the Invention
[0003] In a first aspect of the embodiments of this disclosure, a method for determining the number of target users is provided. The method includes acquiring broadcaster information and environmental information of a live stream. The method also includes generating state features of the live stream based on the broadcaster information and environmental information. Furthermore, the method includes determining the number of target users of the live stream based on the state features using a machine learning model.
[0004] In a second aspect of the embodiments of this disclosure, an apparatus for determining the number of target users is provided. The apparatus includes an information acquisition module configured to acquire broadcaster information and environmental information of a live stream. The apparatus also includes a feature extraction module configured to generate state features of the live stream based on the broadcaster information and environmental information. Furthermore, the apparatus includes a target number determination module configured to determine the number of target users of the live stream using a machine learning model based on the state features.
[0005] In a third aspect of the embodiments of this disclosure, an electronic device is provided. The electronic device includes a processor and a memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, cause the electronic device to perform the method according to the first aspect.
[0006] In a fourth aspect of the embodiments of this disclosure, a computer program product is provided having stored thereon computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method of the first aspect.
[0007] The summary section is provided to present the chosen concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or principal features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description
[0008] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:
[0009] Figure 1 A schematic diagram of an example environment in which several embodiments of the present disclosure may be implemented is shown;
[0010] Figure 2 A flowchart is shown illustrating a method for determining the number of target users according to some embodiments of the present disclosure;
[0011] Figure 3 This illustration shows a schematic diagram illustrating how user experience varies with the number of users during live streaming according to some embodiments of the present disclosure;
[0012] Figure 4 A schematic diagram illustrating the process of generating state features according to some embodiments of the present disclosure is shown;
[0013] Figure 5 A schematic diagram illustrating the process of training a reinforcement learning model according to some embodiments of the present disclosure is shown;
[0014] Figure 6 A schematic diagram illustrating a live streaming traffic allocation process according to some embodiments of the present disclosure is shown;
[0015] Figure 7 A block diagram of an apparatus for determining a target number of users according to some embodiments of the present disclosure is shown; and
[0016] Figure 8 A block diagram of a device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation
[0017] It is understood that all user-related data involved in this technical solution should be obtained and used only after authorization from the user. This means that if it is necessary to use a user's personal information in this technical solution, the user's explicit consent and authorization are required before obtaining this data; otherwise, no related data collection and use will be carried out. It should also be understood that when implementing this technical solution, relevant laws and regulations should be strictly followed in the process of data collection, use, and storage, and necessary technical measures should be taken to protect user data security and ensure the secure use of data.
[0018] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.
[0019] For example, upon receiving a user's proactive request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.
[0020] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0021] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.
[0022] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0023] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects unless explicitly stated. Other explicit and implicit definitions may also be included below.
[0024] As mentioned above, live streaming recommendation systems play a crucial role in the personalized distribution of live streaming content. Extensive research in the field of recommendation systems analyzes users' historical interactive behaviors in live streaming rooms (such as viewing time, likes, comments, and unfollowing) and basic user and live streaming room characteristics. By combining real-time data mining and machine learning techniques, these systems explicitly model and learn user interests, recommending live streaming candidates that can potentially lead to higher conversion rates.
[0025] In theory, recommendation systems can help users explore unfamiliar live streaming content through personalized distribution and provide more traffic to long-tail streamers (such as those with fewer interactive users but whose content focuses on specific areas like skincare, gaming, and music). However, in reality, traditional recommendation methods often face severe popularity biases in traffic distribution, leading to a high concentration of traffic in favor of top streamers, while smaller streamers struggle to overcome traffic barriers. Many studies have analyzed the Matthew effect in content distribution within recommendation systems, primarily attributing it to the natural distribution of data samples and the explicit modeling and overuse of user history by recommendation systems. Over-recommending popular content or streamers can have adverse effects on users, streamers, platforms, and the entire recommendation ecosystem.
[0026] In live streaming scenarios, due to the limited labor and energy of streamers, when the recommendation system allocates too much traffic to popular live streams, resulting in an influx of viewers, the average user experience significantly decreases as the number of viewers increases because the streamer cannot provide timely and expected responses and interactive services to all viewers. In other words, there is a limit to the number of users a streamer can simultaneously provide high-quality service to. High-quality service refers to the streamer's ability to provide timely and effective responses, answering all questions, and other behaviors that improve the experience for both the streamer and the viewers. According to this disclosure, the number of users a streamer can simultaneously provide high-quality service to is defined as the streamer's service capacity (or "streamer capacity").
[0027] Recommendation systems for live streaming typically neglect modeling the effective load (i.e., the streamer's service capacity) on the streamer's side, leaving the system unaware of the streamer's current remaining service capacity. When a streamer has reached their service capacity, a continuous influx of users may experience a degraded experience due to the lack of immediate feedback from the streamer, which in turn negatively impacts the streamer's experience, creating a negative feedback loop and a compounding Matthew effect on the system ecosystem. However, effectively learning the streamer's service capacity is also a challenging task. Due to the randomness of system traffic distribution, there is a lack of end-to-end learning labels that can effectively characterize the streamer's service capacity. Furthermore, constrained by the popularity bias of live stream exposure, simply evaluating the streamer's service capacity through the target posterior average can lead to the underestimation of some potentially high-quality long-tail streamers.
[0028] To address this, this disclosure proposes a method for determining the number of target users. This method accurately learns and evaluates the service capacity of a live streamer based on historical information, thereby determining the target number of users indicating the service capacity. According to embodiments of this disclosure, relevant information from the live stream is first acquired, such as information related to the live streamer and environmental information. Subsequently, based on the acquired information, state features of the live stream are generated. Finally, a machine learning model determines the target number of users for the live stream based on the state features, such as the service capacity of the live streamer. This method improves the accuracy and generalization of predicting the number of target users, effectively alleviates the Matthew effect in live streaming, and efficiently mines the long-tail value in the live streaming system, thereby simultaneously improving the experience of both the live streamer and the users.
[0029] The embodiments of this disclosure will now be described in further detail with reference to the accompanying drawings. Figure 1 A schematic diagram of an example environment 100 in which various embodiments of the present disclosure may be implemented is shown. Reference Figure 1 As shown, a live stream (which can be viewed as a live streaming room) consists of a streamer and one or more users. That is, there is a one-to-many matching relationship between streamers and users; one streamer can serve multiple users simultaneously. Furthermore, streamers and users can interact and communicate through the live streaming platform. In a live stream, the streamer showcases their work, including but not limited to product demonstrations, sports / concert performances, and personal talent displays. The streamer can also interact with users and answer their questions. Users watch the live stream and can interact with the streamer or other users through bullet comments and other means, such as liking and asking questions. The streamer's enthusiasm is closely related to the number of viewers in the live stream. A sufficient number of users who can interact well with the streamer (including liking and commenting) significantly increases the streamer's enthusiasm, while a consistently low number of viewers will dampen the streamer's motivation.
[0030] Continue to refer to Figure 1 The system collects live-stream related information from the live stream 110, including streamer information 120 and environmental information 130. Streamer information 120 includes, for example, streamer profile information. Environmental information 130 includes, for example, contextual information at the time of sampling, such as timestamps, location, and device, but is not limited to these. Then, the streamer information 120 and environmental information 130 are input into a machine learning model 140, which processes the information and outputs the target user count 150. In this embodiment, the machine learning model 140 is based on a deep learning network, including a feature extraction network and a policy network. The feature extraction network processes the collected information to generate features representing the current state of the live stream 110, serving as state features. The policy network processes the state features to generate the target user count 150. The machine learning model 140 can be generated through offline training. It should be noted that... Figure 1The machine learning model 140 shown is for illustrative purposes only, and this disclosure does not limit the specific structure of the input layer, hidden layer and output layer of the machine learning model 140.
[0031] As mentioned above, the number of target users indicates the livestream service capacity of Live110, which is related to the user experience of the target users. According to embodiments of this disclosure, target users are determined based on user-livestream interaction behavior. In some embodiments, interaction behavior is categorized, and users with higher levels of interaction behavior are identified as target users. Of course, target users can also be determined based on interaction frequency, etc. In short, users with high engagement with the livestream are considered target users. For simplicity, "user" as used below refers to "target user".
[0032] Furthermore, user experience can be determined through user interaction information with the streamer. For example, the more frequent and extensive a user's interaction with the streamer in live stream 110, the higher their user experience. In some embodiments, user interaction information includes the average viewing time of users in live stream 110. Assume that live stream 110 currently contains three target users: user A, user B, and user C. Environment 100 obtains the cumulative viewing time of the three users in live stream 110, assuming it is 3 hours, 5 hours, and 4 hours respectively. Then, it calculates the average viewing time as (3+5+4) / 3 = 4 hours. Therefore, it can be determined that in live stream 110, when the number of target users is 3, the average viewing time is 4 hours.
[0033] According to embodiments of this disclosure, after determining the target number of users 150, the number of users in the live stream 110 can be adjusted based on the target number of users 150. For example, when the number of users exceeds the target number of users, the excess users are promptly diverted to other live streams that are of interest to other users and have a smaller number of users (e.g., the number of users is less than the target number of users) to ensure the experience of both the broadcaster and the users in the live stream 110.
[0034] This method effectively improves the user and broadcaster experience on live streaming platforms. This is because the target user number 150, determined by machine learning model 140 based on broadcaster information 120 and environmental information 130 in live stream 110, has high accuracy and generalization. Furthermore, since the processing does not overly rely on historical data, this method can also determine the target user number for new broadcasters on the platform, thus offering higher timeliness and being equally friendly to new broadcasters. In addition, based on the target user number determined by this solution, users exceeding the target number can be diverted to other live streams they are interested in with fewer users, effectively mitigating the Matthew effect in live streaming and efficiently mining the long-tail value of the live streaming system, thereby simultaneously improving the user and broadcaster experience.
[0035] The following will combine Figures 2 to 8 The method according to embodiments of this disclosure is described in detail. For ease of understanding, the specific data mentioned in the following description are exemplary and not intended to limit the scope of this disclosure. It is understood that the embodiments described below may also include additional actions not shown and / or actions shown may be omitted, and the scope of this disclosure is not limited in this respect.
[0036] Figure 2 A flowchart of a method 200 for determining a target number of users according to some embodiments of the present disclosure is shown. Method 200 can be performed by a device for determining the target number of users, which may be, for example, a standalone device or system. This device can be implemented in software and / or hardware. The method 200 will now be illustrated schematically using a device for determining the target number of users as an example. Method 200 includes blocks 202, 204, and 206.
[0037] like Figure 2 As shown in box 202, the system obtains the streamer's information and the live stream's environmental information. For example, streamer information includes various details related to the streamer, used to create a profile of the streamer. Environmental information may include information describing the live stream environment, such as the time and location of entering and leaving the live stream, the number of users, background design, lighting, network environment, and audience interaction. This information reflects the streamer's role in the live stream business scenario, as well as the relationship between the streamer and users.
[0038] In box 204, live stream state features are generated based on broadcaster information and environmental information. For example, since the collected information may contain redundant information, feature extraction from broadcaster and environmental information can reduce data dimensionality and remove irrelevant information, thereby simplifying the data structure and improving the processing speed and efficiency of subsequent machine learning models. Simultaneously, extracting key features can improve the prediction accuracy and generalization ability of subsequent machine learning models. In some embodiments, features of broadcaster and environmental information can be extracted separately, and then state features are generated based on the extracted features. Figure 1 The aforementioned feature extraction network can generate state features based on broadcaster information and environmental information.
[0039] In box 206, a machine learning model determines the target number of users for the live stream based on state features. For example, the machine learning model can be based on a deep learning network to process the input state features to predict the service capacity of the streamer in the live stream scenario indicated by the state features, i.e., the target number of users. This disclosure does not limit the specific deep learning network used to implement the machine learning model; for example, an existing deep learning network or a combination thereof can be used and trained offline to generate the machine learning model. In some embodiments, combined with... Figure 1 The machine learning model mentioned here specifically refers to the aforementioned policy network. The state features are input into the policy network, processed, and the output is the number of target users.
[0040] Figure 3 A schematic diagram illustrating how user experience varies with the number of users during live streaming according to some embodiments of this disclosure is shown. (Reference) Figure 3 The horizontal axis represents the number of users in the live stream, and the vertical axis represents the average viewing time of these users (representing "user experience"). By recording the average viewing time of users with different numbers of users, a curve showing how user experience changes with the number of users is generated. By analyzing the general trend of the curve, it can be seen that such an inflection point in user experience does indeed exist (see reference). Figure 3 Midpoint "A" represents the overall trend of the average viewing time per user (i.e., average user experience), which first increases and then decreases with the number of users in the live stream. In other words, when the number of users in the live stream exceeds the streamer's service capacity, the streamer becomes overwhelmed, resulting in a deterioration in the average user experience. According to embodiments of this disclosure, the number of users corresponding to this user experience "inflection point" is the target number of users.
[0041] This method, using machine learning models to determine the target user count based on streamer and environmental information, exhibits high accuracy, generalization, and timeliness. Based on the determined target user count, the streamer's service capacity can be estimated, allowing users exceeding the target number to be diverted to other live streams of interest with fewer users, thus improving the experience for both users and streamers.
[0042] superior Figure 1 and Figure 2 Example environments and methods in which embodiments of this disclosure can be implemented are described. The following will be combined with… Figure 4 To describe the processing steps involved in the method. Figure 4 A schematic diagram of a process 400 for generating state features according to some embodiments of the present disclosure is shown. In conjunction with the foregoing description, a feature extraction network 430 generates state features 440 based on broadcast information 410 and environmental information 420. (See reference...) Figure 4 In some embodiments, the feature extraction network 430 includes an embedding layer 450, a feature concatenation layer 460, and a fully connected network 470 that are coupled to each other. The process of generating state features by the feature extraction network 430 is as follows.
[0043] First, generate the first feature F1 of the broadcaster information 410 and the second feature F2 of the environment information 420. (Continue to refer to...) Figure 4 The acquired anchor information 410 and environment information 420 are input into the embedding layer 450, and the embedding layer 450 is used to perform densification processing on the anchor information 410 and environment information 420 to generate the first feature F1 and the second feature F2.
[0044] Secondly, based on the first feature F1 and the second feature F2, state features are generated. (Continue to refer to...) Figure 4 The first feature F1 and the second feature F2 are concatenated using feature concatenation 460 to generate the third feature F3. Then, the third feature F3 is processed by a fully connected network 470 through high-dimensional feature cross-processing to generate state features. In some embodiments, the fully connected network 470 can be implemented using a multilayer perceptron (MLP).
[0045] The state features generated through process 400 simplify the data structure and improve the processing speed and efficiency of subsequent machine learning models. Furthermore, the feature extraction network 430 extracts key features from the broadcaster information 410 and environmental information 420, which enhances the prediction accuracy and generalization ability of subsequent machine learning models.
[0046] Next, the feature states are input into a machine learning model to determine the probability distribution of the number of users in the live stream, and sampling is performed based on the probability distribution to determine the target number of users. In some embodiments, the machine learning model that processes the feature states is the aforementioned policy network. A policy network is a method that directly learns the optimal action policy that an agent should take in a specific environment through a deep learning model. In this embodiment, the environmental state input to the policy network is the state features of the live stream; its output action a i The probability distribution of action a i =i (i∈[1,K] represents assigning a to the broadcaster) i There are 1 user, where parameter K can be preset to represent the number of selectable actions; the agent performs action a. i The reward r obtained after making the decision is a. i The average viewing time for each user. In summary, the core of the policy network is to guide the agent's actions (determining the target number of users) by learning the optimal policy (maximizing the average viewing time).
[0047] According to the embodiments of this disclosure, since the live streaming scenario has strong randomness, an offline training method is adopted to generate the above-mentioned policy network by training a reinforcement learning model. Figure 5 A schematic diagram of a process 500 for training a reinforcement learning model according to some embodiments of the present disclosure is shown. (See reference...) Figure 5 Live510 can be viewed as multiple live streaming scenarios on a live streaming platform, including multiple streamers and multiple users. It should be understood that Live510 is merely an example, and this disclosure does not limit the number of streamers and users in a live streaming scenario. The following is combined with... Figure 5 This section provides a detailed explanation of the process of training a reinforcement learning model.
[0048] First, historical live stream traffic is obtained from live stream 510, and training dataset 520 is generated based on this historical traffic. (Continue to refer to...) Figure 5 The training dataset 520 includes a broadcaster profile 522, context information 524, and target user count and user experience as label data 526. It should be understood that the information in the training dataset 520 is all historical information, i.e., information from historical live streams. In some embodiments, the training dataset 520 is generated as follows: First, broadcaster information samples (broadcaster profile 522) and environment information samples (context information 524) from historical live stream traffic are extracted; simultaneously, the historical target user count and historical user experience from historical live stream traffic are extracted as label data 526. Then, the training dataset 520 is generated based on the broadcaster information samples, environment information samples, and label data.
[0049] Secondly, based on training dataset 520, a reinforcement learning model 530 was trained. (Continue to refer to...) Figure 5 The reinforcement learning model 530 includes a feature extraction network 532, an offline reinforcement learning network 534, and a policy network 536, all coupled together. The offline reinforcement learning network 534 determines the fitting relationship between the historical number of target users and the historical user experience, while the policy network determines the probability distribution of the target user number. In some embodiments, the offline reinforcement learning network 534 is based on a conservative Q-learning algorithm and has N output heads representing the predicted user experience when the broadcaster serves a corresponding number of users. The policy network 536 predicts the broadcaster's service capacity, outputting the mean and variance of a Gaussian distribution. This allows for the reconstruction of the estimated service capacity distribution (i.e., the aforementioned probability distribution), and sampling from this distribution yields the final estimated service capacity (i.e., the target user number).
[0050] Continue to refer to Figure 5 The feature extraction network 530 generates historical state features of historical live streams based on anchor information samples and environment information samples in the training dataset 520. For details on the generation process of historical state features, please refer to the previous text. Figure 4 The relevant descriptions are omitted here. Next, we construct the training dataset D = {(s, a, r)}. j} Where s represents historical state features, a represents the number of historical target users, and r represents historical user experience. Next, the offline reinforcement learning network 534 and policy network 536 generate prediction data based on the historical state features. The prediction data includes: the predicted user experience 540 (also called the first prediction data) generated by the offline reinforcement learning network 534 based on the historical state features for each preset number of users; and the predicted number of target users 550 (also called the second prediction data) generated by the policy network 536 based on the historical state features and the predicted user experience. Then, based on the above prediction data and label data 526, the loss 560 of the reinforcement learning model 530 is determined.
[0051] In some embodiments, the loss 560 is determined as follows: First, based on the predicted user experience 540 and the label data 526, the loss (also known as the first loss) of the offline reinforcement learning network 534 is determined. The samples in the recommender system exhibit a significant long-tail effect, with a severe imbalance in traffic distribution between popular and unpopular streamers: popular streamers receive more user exposure, while unpopular streamers often fail to be seen by enough users. In such a significantly skewed sample space, the overestimation of the value function caused by distribution shift becomes more pronounced. Therefore, in this embodiment, formula (1) is used to determine the loss of the offline reinforcement learning network 534 to alleviate the overestimation problem:
[0052]
[0053] Where α is the regularity coefficient. 'r' represents the predicted user experience, and 'r' represents the labeled data. Referring to formula (1), the loss of the offline reinforcement learning network 534 is based on the Bellman error of the traditional deep Q-network with the addition of a CQL regularization term. The CQL regularization term prevents the Q-value from being overestimated by appropriately encouraging the Q-value already present in the dataset while suppressing the Q-value of out-of-distribution (OOD).
[0054] Next, based on the predicted user experience 540, the predicted target user number 550, and the label data 526, the loss of the policy network 536 (also known as the second loss) is determined. Since the service environment of the recommender system has strong randomness, it is difficult to effectively learn the distribution of behavioral policies that generate offline training samples. Therefore, this embodiment does not explicitly model the behavioral policies. Instead, inspired by the AWAC algorithm, it attempts to learn a policy distribution that maximizes the expected reward near the behavioral policies. Based on the advantage function prediction value, the distribution of the policy network is estimated using maximum likelihood. In this embodiment, formula (2) is used to determine the loss of the policy network 536:
[0055]
[0056] Where, π θ (a|s) represents the probability that the broadcaster's service capacity is a in state s, where π θ The advantage function represents the predicted number of target users. Used to measure the policy distribution π θ The value of action 'a' relative to the average action is determined, and the mean of the actual reward feedback in state 's' is directly incorporated into the calculation of the advantage function. As an approximate calculation of state value, in order to improve generalization while maintaining a certain level of exploration capability, the policy π is modeled as a Gaussian distributed stochastic policy, and the Gaussian distribution parameters are learned by a feedforward neural network.
[0057] Then, based on losses L1 and L2, the final loss is determined. (Continue to refer to...) Figure 5 Based on the determined loss, the offline reinforcement learning network 534 and the policy network 536 are adjusted through backpropagation. In some embodiments, the parameters of the offline reinforcement learning network 534 are adjusted based on loss L1, and the parameters of the policy network 536 are adjusted based on loss L2, until convergence. Finally, the adjusted policy network 536 (i.e., the policy network 536 obtained after the above training convergence) is the machine learning model deployed in the online inference task (i.e., the machine learning model in box 206).
[0058] To further illustrate the process of the reinforcement learning model 530, the following pseudocode examples of training the reinforcement learning model 530 according to some embodiments of the present disclosure are shown.
[0059] Initialize the parameters of the offline reinforcement learning network and the policy network. θ
[0060]
[0061] This method uses the generalized features of broadcasters as input to learn personalized broadcaster service capacity. When training a reinforcement learning model that includes offline reinforcement learning networks and policy networks, it effectively mitigates errors introduced by distribution bias, improving the model's prediction accuracy and generalization ability. This method comprehensively enhances the accuracy of broadcaster service capacity prediction, thereby improving the experience for both broadcasters and users.
[0062] Figure 6 A schematic diagram of a live streaming traffic allocation process 600 according to some embodiments of the present disclosure is shown. Process 600 can be configured to be executed in a live streaming recommendation system. Based on the aforementioned method 200, process 600 takes a streamer-user bipartite graph abstracted from historical interaction information between streamers and users as input, and uses the number of target users determined by method 200 as the capacity of each streamer node in the bipartite graph. Then, based on bipartite graph matching and optimal subgraph algorithms, it solves the maximum weight degree constrained subgraph problem of the bipartite graph, thereby obtaining a bilateral traffic allocation scheme that achieves optimal global streamer-user relationship matching.
[0063] refer to Figure 6 In the live streaming scenario 610 of the live streaming platform, there are multiple streamers and multiple users. In box 620, the target number of users for each streamer in live streaming scenario 610 is determined. This can be determined based on the aforementioned method 200, or other methods, which will not be elaborated here. In box 630, based on the interaction information between streamers and users in live streaming scenario 610, and using the target number of users as a constraint, an initial streamer-user bipartite graph is generated. In some embodiments, streamers and users are respectively regarded as two mutually exclusive vertex sets in the bipartite graph. The edges connecting streamer vertices and user vertices are used to characterize the historical interaction behavior of the relationship pair. The edge weight is set to the user's viewing time with the streamer. In this embodiment, the streamer-user bipartite graph is defined as... Let X be the set of broadcasters and Y be the set of users; edge weight w: E→R is used to characterize the user's experience with the broadcaster, for example, using the viewing time of each user as the edge weight; c: X→Z +The capacity of each broadcaster is defined as the maximum service capacity of each broadcaster's endpoint. Based on a broadcaster-user bipartite graph, considering the degree constraints of user and broadcaster nodes, the goal is to find an optimal (maximum edge weight sum) subgraph (a subset of edges) that satisfies the conditions to represent the traffic allocation between broadcasters and users. This can be formalized as an integer linear programming problem:
[0064] for
[0065]
[0066] Here, f(e) indicates whether edge e is selected into the final result edge set.
[0067] In box 640, determine whether the above planning problem is solvable. If solvable, proceed to box 652. If not solvable, proceed to box 654. In box 652, solve the above planning problem based on the bipartite graph maximum weight degree constrained subgraph algorithm (i.e., algorithm (1)). The execution process of algorithm (1) is shown below.
[0068] Algorithm (1)
[0069] a. Specify a lower limit of 1 and an upper limit of 1 (target number of users), where the upper limit on the user side is a specific constant;
[0070] b. Apply the MWDCS (maximum weight degree-constrained subgraph problem) algorithm to solve for the maximum weight subgraph that satisfies the upper and lower degree limits.
[0071] However, considering the special nature of live streaming scenarios, it is necessary to ensure that there is a reasonable subgraph as the output under any circumstances. The broadcaster-user bipartite graph is a naturally large-scale sparse graph, and the upper limit constraint of the degree of the broadcaster node is usually small, which means that the above planning problem may be unsolvable in most cases, that is, some nodes cannot simultaneously satisfy the upper and lower limits of node capacity. Therefore, in box 654, a subgraph with the maximum edge weight is solved in the minimum isolated vertex subgraph that satisfies the capacity limit on the broadcaster side (i.e., algorithm (2)). According to some embodiments, the solution process of algorithm (2) includes boxes 656 and 658. In box 656, the LeastIsolated-vertexMaximum Weighted Subgraph of One-side-capacity Bipartite graph (LIMWS) algorithm with one-side capacity limit of bipartite graph is used to solve the exact solution, and its execution process is shown below.
[0072] Algorithm (2) - Exact Solution
[0073] a. Make two copies of the original image, G1 and G2;
[0074] b. Calculate W = 2N∑ x∈X c(x);
[0075] c. Connect two copies of the same point with a -W edge, and the newly added edge set is G. W ;
[0076] d. Obtain graph G' = G1 + G W +G2;
[0077] e. Find the maximum weighted degree restricted subgraph P = MWDSC(G') for the new graph G';
[0078] f. Based on the intersection of P and G1, the final subgraph is obtained.
[0079] Under the condition of positive edge weights, the above is equivalent to the problem of minimizing isolated vertices on the right side while only restricting the capacity on the left side. An approximate solution can be efficiently found using the maximum weighted maximum matching algorithm. Specifically, in box 658: the optimal solution is found using the Least Isolated-vertex Approximate Maximum Weighted subgraph of cOne-side-capacity Bipartite graph (LIAMWS) algorithm, which uses a one-side capacity constraint on the bipartite graph. Its execution process is shown below.
[0080] Algorithm (2) - Optimization Solution
[0081] a. First, set the user-side capacity to 1, then split the broadcaster into points according to the service capacity, and convert it into a bipartite graph G' with a capacity of 1;
[0082] b. Find the maximum weight matching in the bipartite graph G' (compare the maximum weight matching), M = MWMM(G');
[0083] c. Transform the obtained matching M back into the subgraph G of the original graph. M ;
[0084] d. For each broadcast node x: r(x) ← c(x) ← d GM (x);
[0085] e. For each broadcaster node x: Sort Ex in descending order;
[0086] f. For each edge e: Based on the intersection of P and G1, obtain the final subgraph.
[0087] g. If
[0088] D x +=e,
[0089] r(x)--
[0090] h. Return
[0091] In summary, the optimal subgraph 660 can be solved based on any one of boxes 652, 656, and 658. In box 670, based on the optimal subgraph, a traffic allocation result is generated. The streamer and user connected by each edge in the subgraph are the streamer-user relationships that need to be preserved. Based on the traffic allocation result, it is directly used to decide on the traffic scheduling of the live streaming service line (680).
[0092] In other embodiments, considering the potential drawbacks of the above allocation method, such as: 1) over-reliance on historical information, making it impossible to generalize and predict new relationship pairs; and 2) weakening other valuable objectives, which detracts from the overall efficiency of the online recommendation system, a deep neural network is used to perform binary classification learning on the allocation decisions of broadcaster-user relationship pairs. Specifically, the broadcaster-user relationships indicated by the optimal subgraph 660 are compared with those in the initial broadcaster-user bipartite graph. The retained edges are used as positive examples, and the removed edges are used as negative examples to generate a training dataset for training a binary classification network (e.g., a Softmax classifier). Then, the predicted scores output by the binary classification network are superimposed on the online recommendation chain to achieve soft control over the allocation of broadcaster-user traffic.
[0093] This method first constructs a bipartite graph based on the interaction signals between users and broadcasters, using viewing time as the edge weight to characterize the bilateral relationship between the user and broadcaster sides in a live streaming scenario. The estimated broadcaster service capacity is used as the in-degree constraint for broadcaster nodes. A bipartite graph matching algorithm is then introduced to solve the maximum weight maximum cardinality subgraph problem with bilateral constraints, thereby obtaining a globally optimal user-broadcaster traffic allocation scheme that effectively improves the experience for both users and broadcasters on the live streaming platform.
[0094] Figure 7 A block diagram of an apparatus 700 for determining a target number of users according to some embodiments of the present disclosure is shown. Figure 7 As shown, the device 700 includes an information acquisition module 710, configured to acquire the broadcaster's information and the broadcast environment information. The device 700 also includes a feature extraction module 720, configured to generate live broadcast state features based on the broadcaster's information and the environment information. Furthermore, the device 700 includes a target quantity determination module 730, configured to determine the target number of users for the live broadcast using a machine learning model based on the state features.
[0095] In some embodiments, the feature extraction module 720 includes a first generation module configured to generate a first feature of the broadcaster information and a second feature of the environment information, respectively; and a second generation module configured to generate a state feature based on the first feature and the second feature.
[0096] In some embodiments, the first generation module is configured to use the embedding layer to densify the anchor information and the environmental information respectively, and generate a first feature and a second feature.
[0097] In some embodiments, the second generation module is configured to generate a third feature by splicing the first feature and the second feature; and to generate a state feature by performing high-dimensional feature cross-processing on the third feature using a fully connected network.
[0098] In some embodiments, the machine learning model is based on a deep learning network, and the target number determination module 730 includes a distribution determination module configured to input state features into the machine learning model to determine the probability distribution of the number of users in the live stream; and an adoption module configured to sample based on the probability distribution to determine the target number of users.
[0099] In some embodiments, the number of target users indicates the capacity of the broadcaster service in relation to the user experience of the live stream, wherein the user experience includes the average viewing time of the target users in the live stream.
[0100] In some embodiments, the apparatus 700 further includes: a training data generation module configured to generate a training dataset based on historical live streaming traffic; and an offline training module configured to train a reinforcement learning model based on the training dataset.
[0101] In some embodiments, the training data generation module includes: a sample extraction module configured to extract anchor information samples and environment information samples from historical live stream traffic; a label extraction module configured to extract the historical target user count and historical user experience from historical live stream traffic as label data; and a data generation module configured to generate a training dataset based on the anchor information samples, environment information samples, and label data.
[0102] In some embodiments, the reinforcement learning model includes a feature extraction network, an offline reinforcement learning network, and a policy network that are coupled together, wherein the offline reinforcement learning network is used to determine the fitting relationship between the number of historical target users and the historical user experience, and the policy network is used to determine the probability distribution of the number of target users.
[0103] In some embodiments, the offline training module, the third generation module, is configured to generate historical state features of the historical live broadcast based on anchor information samples and environmental information samples by the feature extraction network; the prediction module is configured to generate prediction data based on the historical state features by the offline reinforcement learning network and the policy network; the loss determination module is configured to determine the loss of the reinforcement learning model based on the prediction data and the label data; the adjustment module is configured to adjust the offline reinforcement learning network and the policy network through backpropagation based on the loss; and the fourth generation module is configured to generate a machine learning model based on the adjusted policy network.
[0104] In some embodiments, the prediction module includes: a first prediction module configured to generate predicted user experience for each preset number of users based on historical state features by an offline reinforcement learning network, as first prediction data; and a second prediction module configured to generate predicted target number of users based on historical state features and the first prediction data by a policy network, as second prediction data.
[0105] In some embodiments, the loss determination module includes: a first loss module configured to determine a first loss of the reinforcement learning model based on first predicted data and label data; a second loss module configured to determine a second loss of the reinforcement learning model based on the first predicted data, the second predicted data, and label data; and a third loss module configured to determine a loss based on the first loss and the second loss.
[0106] This device effectively enhances the user and broadcaster experience on live streaming platforms. This is because the target user count, determined by the machine learning model based on broadcaster and environmental information, demonstrates high accuracy and generalization. Furthermore, since the process doesn't overly rely on historical data, it can also determine the target user count for new broadcasters, resulting in higher timeliness and making it equally suitable for them. Additionally, based on the target user count determined by this solution, users exceeding the target number can be diverted to other live streams they are interested in with fewer users, thus simultaneously improving the experience for both users and broadcasters.
[0107] Figure 8 A block diagram of a device 800 capable of implementing various embodiments of the present disclosure is shown. (See diagram for example.) Figure 8As shown, device 800 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 802 or loaded from storage unit 808 into random access memory (RAM) 803. The RAM 803 can also store various programs and data required for the operation of device 800. The CPU / GPU 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804. Although not shown in... Figure 8 As shown, device 800 may also include a coprocessor.
[0108] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0109] The various methods or processes described above can be executed by CPU / GPU 801. For example, in some embodiments, the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by CPU / GPU 801, one or more steps or actions in the methods or processes described above can be performed.
[0110] In some embodiments, the methods and processes described above can be implemented as a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.
[0111] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.
[0112] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network, to an external computer or external storage device. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.
[0113] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages and conventional procedural programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.
[0114] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.
[0115] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0117] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A method for determining the number of target users, comprising: Obtain the information of the live streamer and the environment information of the live stream; Based on the broadcaster information and the environment information, the status characteristics of the live broadcast are generated; as well as The number of target users for the live stream is determined by a machine learning model based on the state characteristics.
2. The method according to claim 1, wherein generating the state features of the live stream includes: A first feature of the broadcaster information and a second feature of the environment information are generated respectively; as well as The state feature is generated based on the first feature and the second feature.
3. The method according to claim 2, wherein generating the first feature of the broadcaster information and the second feature of the environment information respectively includes: The anchor information and the environmental information are densified using an embedding layer to generate the first feature and the second feature.
4. The method according to claim 2, wherein generating the state feature comprises: The third feature is generated by concatenating the first feature and the second feature; as well as The state feature is generated by performing high-dimensional feature cross-processing on the third feature using a fully connected network.
5. The method according to claim 1, wherein the machine learning model is based on a deep learning network, and the determination of the target number of users for the live stream by the machine learning model based on the state features includes: The state features are input into a machine learning model to determine the probability distribution of the number of users in the live stream. as well as The number of target users is determined by sampling based on the probability distribution.
6. The method of claim 1, wherein the number of target users indicates the broadcaster service capacity related to the user experience of the live stream, wherein the user experience includes the average viewing time of the target users in the live stream.
7. The method according to claim 1, further comprising: A training dataset is generated based on historical live stream traffic. as well as A reinforcement learning model is trained based on the training dataset.
8. The method of claim 7, wherein generating the training dataset comprises: Extract streamer information samples and environment information samples from the historical live stream traffic; Extract the number of historical target users and historical user experience from the historical live stream traffic, and use them as tag data; as well as The training dataset is generated based on the broadcaster information sample, the environment information sample, and the tag data.
9. The method according to claim 8, wherein the reinforcement learning model comprises a feature extraction network, an offline reinforcement learning network, and a policy network coupled together, wherein the offline reinforcement learning network is used to determine the fitting relationship between the historical target user number and the historical user experience, and the policy network is used to determine the probability distribution of the target user number.
10. The method of claim 9, wherein training the reinforcement learning model comprises: The feature extraction network generates historical state features of the historical live broadcast based on the broadcaster information sample and the environmental information sample. The offline reinforcement learning network and the policy network generate prediction data based on the historical state features; Based on the predicted data and the labeled data, the loss of the reinforcement learning model is determined; Based on the loss, the offline reinforcement learning network and the policy network are adjusted through backpropagation; as well as The machine learning model is generated based on the adjusted policy network.
11. The method of claim 10, wherein generating prediction data by the offline reinforcement learning network and the policy network based on the historical state features comprises: The offline reinforcement learning network generates predicted user experience for each preset number of users based on the historical state features, which serves as the first prediction data. as well as The policy network generates a predicted number of target users based on the historical state features and the first prediction data, which serves as the second prediction data.
12. The method of claim 11, wherein determining the loss of the reinforcement learning model based on the predicted data and the labeled data comprises: Based on the first predicted data and the label data, the first loss of the reinforcement learning model is determined; Based on the first prediction data, the second prediction data, and the label data, the second loss of the reinforcement learning model is determined; as well as The loss is determined based on the first loss and the second loss.
13. An apparatus for determining the number of target users, comprising: The information acquisition module is configured to acquire the broadcaster information and the environmental information of the live broadcast. The feature extraction module is configured to generate the status features of the live stream based on the broadcaster information and the environment information; as well as The target number determination module is configured to determine the target number of users for the live stream based on the state characteristics using a machine learning model.
14. An electronic device, comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1 to 12.
15. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of claims 1 to 12.
