Creating homogeneous conversations using machine learning (ML) models

Abstract

An electronic device and method for creating homogeneous sessions using a machine learning (ML) model are disclosed. The electronic device receives interaction information from a group of users for a set of items. The electronic device receives a text description for each item. The electronic device determines an embedding vector associated with each item. The electronic device clusters the determined embedding vectors. The electronic device applies a reinforcement learning model to a set of labels. The electronic device determines a set of session slots associated with the received interaction information from the group of users. The electronic device applies a sequence model to the set of session slots associated with the received interaction information from the group of users. The electronic device determines a set of recommended items for each user in the group. The electronic device presents the determined set of recommended items.

Description

Technical Field

[0001] Various embodiments of this disclosure relate to the determination of sessions. More specifically, various embodiments of this disclosure relate to the creation of homogeneous sessions using machine learning (ML) models. Background Technology

[0002] Advances in artificial intelligence have led to the development of different types of machine learning (ML) models, such as recommendation models. A typical recommendation model recommends a set of items to a user based on user interactions extracted from one or more sessions associated with the user. A session can begin at a point in time when the user can start interacting with at least one item through a platform (e.g., an over-the-top platform). Furthermore, each session can be associated with a fixed duration. In the example, the fixed duration could be thirty minutes. In this paper, the first session can correspond to the time interval between a first point in time when the user can start interacting with at least one item through the platform and a second point in time. The second point in time can be thirty minutes after the first point in time. Additionally, the second session can include a duration of thirty minutes after the end of the first session. Such sessions can be received and provided to the recommendation model. The recommendation model can recommend one or more items to the user based on the user interactions associated with the provided sessions. However, each user may exhibit different behavioral patterns associated with interacting with items on the platform. For example, teenage users may exhibit inconsistent behavior and therefore may spend significantly less time on various items than middle-aged users. For instance, teenage users may switch from one item to another within very short time spans (e.g., seconds). However, middle-aged users may exhibit stable behavioral patterns and may spend a minute analyzing items. Therefore, the number of interactions per individual may differ within a fixed duration. Consequently, the size of a session may vary from person to person. Thus, a session associated with a fixed duration may include similar items for some users and a mix of items for others.

[0003] By comparing the described system with some aspects of this disclosure, as described in the remainder of this application and with reference to the accompanying drawings, the limitations and disadvantages of conventional and traditional methods will become apparent to those skilled in the art. Summary of the Invention

[0004] As described more fully in the claims, an electronic device and method for creating homogeneous sessions using a machine learning (ML) model are provided, substantially as shown in and / or described in connection with at least one of the figures in the accompanying drawings.

[0005] These and other features and advantages of this disclosure will become apparent from the following detailed description and review of the accompanying drawings, in which the same reference numerals always denote the same parts. Attached Figure Description

[0006] Figure 1 This is a block diagram illustrating an exemplary network environment for creating homogeneous sessions using a machine learning (ML) model, according to embodiments of the present disclosure.

[0007] Figure 2 This illustrates an embodiment according to the present disclosure. Figure 1 A block diagram of an exemplary electronic device.

[0008] Figure 3 This is a diagram illustrating an exemplary processing pipeline for creating homogeneous sessions using a machine learning (ML) model, according to embodiments of the present disclosure.

[0009] Figure 4 This is a diagram illustrating an exemplary scenario for determining a set of session slots according to embodiments of the present disclosure.

[0010] Figure 5 This is a diagram illustrating an exemplary scenario for creating homogeneous sessions using a machine learning (ML) model according to embodiments of the present disclosure.

[0011] Figure 6 This is a diagram illustrating an exemplary scenario for recommending items using an application based on a hierarchical recurrent neural network (HRNN) according to embodiments of the present disclosure.

[0012] Figure 7 This is a flowchart illustrating operations of an exemplary method for creating homogeneous sessions using a machine learning (ML) model according to embodiments of the present disclosure. Detailed Implementation

[0013] The implementations described below can be found in electronic devices and methods for creating homogeneous sessions using machine learning (ML) models. An exemplary aspect of this disclosure provides an electronic device that receives interaction information from a group of users for a set of items. The electronic device can receive a text description of each item in this set of items. The electronic device can determine an embedding vector associated with each item in this set of items based on the received text description. The electronic device can cluster the determined embedding vectors associated with each item in this set of items into a set of labels. The electronic device can apply a reinforcement learning model to this set of labels based on the clustering of the determined embedding vectors. The electronic device can determine a set of session slots associated with the received interaction information from this group of users based on the application of the reinforcement learning model. The electronic device can apply a sequence model to this set of session slots associated with the received interaction information from this group of users. The electronic device can determine a set of recommended items for users in this group of users based on the application of the sequence model. The electronic device can present the determined set of recommended items.

[0014] Typically, a recommendation model can recommend a set of items to a user based on user interactions extracted from one or more sessions associated with the user. A session can begin at a point in time when the user starts interacting with at least one item through the platform. Furthermore, each session can be associated with a fixed duration. In the example, the fixed duration could be thirty minutes. In this paper, the first session can correspond to the time interval between a first point in time when the user can start interacting with at least one item through the platform and a second point in time. The second point in time can be thirty minutes after the first point in time. Furthermore, the second session can include a duration of thirty minutes after the second point in time. Such sessions can be received and provided to the recommendation model. The recommendation model can recommend one or more items to use based on the user interactions associated with the provided sessions. However, each user may exhibit different behavioral patterns associated with interacting with items on the platform. For example, teenage users may exhibit inconsistent behavior and therefore may spend significantly less time on various items than middle-aged users. For instance, a teenage user might switch from one item to another in a very short time span (e.g., a few seconds). However, a middle-aged user might exhibit a stable behavioral pattern and might spend a minute analyzing items. Therefore, the number of interactions for each individual may differ within a fixed duration. Therefore, the size of a session can vary from person to person. Thus, a session associated with a fixed duration may include similar items for some users and mixed items for others.

[0015] To address the aforementioned issues, the disclosed electronic device can employ a machine learning (ML) model to create homogeneous sessions. The electronic device can use a reinforcement learning model to determine a set of session slots associated with the interaction information received from a group of users. The length of each session slot in the determined set can be customized based on the interaction patterns of the users associated with the corresponding session slot. That is, the sizes of two or more session slots in this set can be different. In the example, this group of users can include teenage users and middle-aged users. Teenage users may have unstable interaction behavior, while middle-aged users may have stable interaction behavior. That is, teenage users may switch from one item to another faster than middle-aged users. Therefore, the size of the determined set of session slots associated with teenage users can be smaller than the size of the determined set of session slots associated with middle-aged users. Furthermore, the size of the determined set of session slots can vary based on the mood of the users associated with this set of session slots. The determined set of session slots can be fed as input to a sequential model to determine a set of recommended items for users in this group. This set of recommended items (such as movies) can be presented on an over-the-top (OTT) platform. Since each session slot can correspond to a single type of item (e.g., action movies or comedy movies), recommendations based on such homogeneous slots may be more accurate and relevant for a particular user. In other words, cohesive session slots with items in a single dominant category can help recommend items that are likely to be more popular with the user.

[0016] Figure 1 This is a block diagram illustrating an exemplary network environment for creating homogeneous sessions using a machine learning (ML) model, according to embodiments of the present disclosure. References Figure 1 The diagram illustrates a network environment 100. Network environment 100 may include an electronic device 102, a server 104, a database 106, and a communication network 108. Electronic device 102 can communicate with server 104 via one or more networks (such as communication network 108). Electronic device 102 may include a reinforcement learning model 102A, a sequential model 102B, and a K-means clustering model 102C. Figure 1 The diagram also shows interactive information 110, a set of items 112, and a set of text descriptions 114 that can be stored in database 106. The set of items 112 may include items 112A, 112B, ..., and 112N. Similarly, the set of text descriptions 114 may include text descriptions 114A, 114B, ..., and 114N. Text description 114A may be associated with item 112A, text description 114B may be associated with item 112B, ..., and text description 114N may be associated with item 112N.

[0017] Figure 1 The N items and N text descriptions shown are merely examples. Without departing from the scope of this disclosure, database 106 may include only two items and text descriptions, or more than N items and text descriptions. For the sake of brevity, Figure 1 Only N items and text descriptions are shown in this document. However, in some embodiments, more than N items and text descriptions may exist without limiting the scope of this disclosure. Figure 1 The image also shows a user 116 who can be associated with or operate the electronic device 102.

[0018] Electronic device 102 may include suitable logic, circuitry, interfaces, and / or code, which may be configured to receive interaction information 110 from a group of users for a set of items 112. Electronic device 102 may receive a text description (such as text description 114A) for each item in the set of items 112 (such as item 112A). Electronic device 102 may determine an embedding vector associated with each item in the set of items 112 (such as item 112A) based on the received text description (e.g., text description 114A). Electronic device 102 may cluster the determined embedding vectors associated with each item in the set of items 112 into a set of labels. Electronic device 102 may apply a reinforcement learning model 102A to the set of labels based on the clustering of the determined embedding vectors. Electronic device 102 may determine a set of session slots associated with the received interaction information 110 from the group of users based on the application of reinforcement learning model 102A. Electronic device 102 may apply a sequence model 102B to the set of session slots associated with the received interaction information 110 from the group of users. Electronic device 102 can determine a set of recommended items for a user (e.g., user 116) within a group of users based on the application of sequence model 102B. Electronic device 102 can then present the determined set of recommended items.

[0019] Examples of electronic devices 102 may include, but are not limited to, computing devices, smartphones, cellular phones, mobile phones, gaming devices, mainframes, servers, computer workstations, devices with machine learning (ML) capabilities (enabled or hosted with, for example, computing resources, storage resources, network resources and / or one or more machine learning models) and / or consumer electronics (CE) devices.

[0020] Server 104 may include suitable logic, circuitry, interfaces, and / or code, which may be configured to receive interaction information 110 from a group of users for a set of items 112. Server 104 may receive a text description (such as text description 114A) for each item in the set of items 112 (such as item 112A). Server 104 may determine an embedding vector associated with each item in the set of items 112 (such as item 112A) based on the received text description (e.g., text description 114A). Server 104 may cluster the determined embedding vectors associated with each item in the set of items 112 into a set of labels. Server 104 may apply a reinforcement learning model 102A to the set of labels based on the clustering of the determined embedding vectors. Server 104 may determine a set of session slots associated with the received interaction information 110 from the group of users based on the application of reinforcement learning model 102A. Server 104 may apply a sequence model 102B to the set of session slots associated with the received interaction information 110 from the group of users. Server 104 can determine a set of recommended items for user 116 within a group of users based on the application of the sequential model 102B. Server 104 can then present the determined set of recommended items.

[0021] Server 104 can be implemented as a cloud server and can perform operations through web applications, cloud applications, HTTP requests, repository operations, file transfers, etc. Other example implementations of Server 104 may include, but are not limited to, database servers, file servers, web servers, media servers, application servers, mainframe servers, machine learning servers (enabled or hosted with, for example, computing resources, storage resources, and network resources), or cloud computing servers.

[0022] In at least one embodiment, server 104 can be implemented as multiple distributed cloud-based resources using several techniques well known to those skilled in the art. Those skilled in the art will understand that the scope of this disclosure is not limited to implementing server 104 and electronic device 102 as two separate entities. In some embodiments, without departing from the scope of this disclosure, the functionality of server 104 can be wholly or at least partially integrated into electronic device 102. In some embodiments, server 104 can host database 106. Alternatively, server 104 can be decoupled from database 106 but communicatively coupled to database 106.

[0023] Database 106 may include suitable logic, interfaces, and / or code, and may be configured to store interaction information 110, a set of items 112, and text descriptions 114. Database 106 may be exported from data in a relational or non-relational database, or from a set of comma-separated value (CSV) files in a conventional or large data storage device. Database 106 may be stored or cached on a device such as a server (e.g., server 104) or electronic device 102. The device storing database 106 may be configured to receive queries from electronic device 102 regarding interaction information 110, a set of items 112, and / or text descriptions 114. In response, the device storing database 106 may be configured to retrieve the queried interaction information 110, a set of items 112, and / or text descriptions 114 based on the received query and provide them to electronic device 102.

[0024] In some embodiments, database 106 may be hosted on multiple servers located in the same or different locations. The database 106 may be operated using hardware including processors, microprocessors (e.g., for performing or controlling one or more operations), field-programmable gate arrays (FPGAs), or application-specific integrated circuits (ASICs). In some other cases, database 106 may be implemented using software.

[0025] Communication network 108 may include a communication medium through which electronic devices 102 and server 104 can communicate with each other. Communication network 108 may be either a wired or wireless connection. Examples of communication network 108 may include, but are not limited to, the Internet, cloud networks, cellular or wireless mobile networks (such as LTE and 5G New Radio (NR)), satellite communication systems (e.g., using low Earth orbit satellites), Wi-Fi networks, personal area networks (PANs), local area networks (LANs), or metropolitan area networks (MANs). Various devices in network environment 100 may be configured to connect to communication network 108 according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of the following: Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Li-Fi, 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocols, and Bluetooth (BT) communication protocols.

[0026] The reinforcement learning model 102A can be a machine learning (ML) model, specifically a classifier model, which can be trained to recognize the relationship between inputs (such as features in the training dataset) and output labels (such as a set of session slots). The reinforcement learning model 102A can be defined by its hyperparameters, such as the number of weights, cost function, input size, number of layers, etc. The parameters of the reinforcement learning model 102A can be tuned, and the weights can be updated to move towards the global minimum of the cost function of the reinforcement learning model 102A. After several epochs of training on the features in the training dataset, the reinforcement learning model 102A can be trained to output a classification result for a set of inputs.

[0027] Reinforcement learning model 102A may include electronic data, which may be implemented as a software component of an application, for example, executable on electronic device 102. Reinforcement learning model 102A may rely on libraries, external scripts, or other logic / instructions for execution by a processing device. Reinforcement learning model 102A may include code and routines configured to enable a computing device to perform one or more operations, such as determining a set of session slots. Additionally or alternatively, reinforcement learning model 102A may be implemented using hardware including a processor, a microprocessor (e.g., for performing one or more operations or controlling the execution of one or more operations), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). Alternatively, in some embodiments, reinforcement learning model 102A may be implemented using a combination of hardware and software.

[0028] In this embodiment, the reinforcement learning model 102A may be an neural network (NN) model. The neural network may be a system or computational network of artificial neurons arranged as nodes in multiple layers. The multiple layers of the neural network may include an input layer, one or more hidden layers, and an output layer. Each of the multiple layers may include one or more nodes (or, for example, artificial neurons represented by circles). The outputs of all nodes in the input layer may be coupled to at least one node in the hidden layer. Similarly, the input of each hidden layer may be coupled to the output of at least one node in other layers of the neural network. The output of each hidden layer may be coupled to the input of at least one node in other layers of the neural network. Nodes in the final layer may receive input from at least one hidden layer to output a result. The number of layers and the number of nodes in each layer may be determined by the hyperparameters of the neural network. Such hyperparameters may be set before, during, or after training the neural network on a training dataset.

[0029] Each node in a neural network can correspond to a mathematical function (e.g., a sigmoid function or a rectified linear unit) with a set of parameters that are adjustable during the network's training. This set of parameters can include, for example, weight parameters, regularization parameters, etc. Each node can use the mathematical function to compute an output based on one or more inputs from nodes in other layers of the neural network (e.g., previous layers). All nodes or some nodes in a neural network can correspond to the same or different mathematical functions.

[0030] In training a neural network, one or more parameters of each node can be updated based on whether the output of the final layer for a given input (from the training dataset) matches the correct result of the loss function based on the neural network. This process can be repeated for the same or different inputs until the minimum of the loss function is achieved and the training error is minimized. Several methods for training are known in the art, such as gradient descent, stochastic gradient descent, batch gradient descent, gradient boosting, metaheuristics, etc.

[0031] The neural network may include electronic data, which may be implemented as a software component of an application, for example, executable on electronic device 102. The neural network may rely on libraries, external scripts, or other logic / instructions for execution by a processing device. The neural network may include code and routines configured to enable a computing device to perform one or more operations to determine a set of session time slots. Additionally or alternatively, the neural network may be implemented using hardware including a processor, a microprocessor (e.g., for performing one or more operations or controlling the execution of one or more operations), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). Alternatively, in some embodiments, the neural network may be implemented using a combination of hardware and software.

[0032] The sequential model 102B can be a system or computational network of artificial neurons arranged as nodes in a set of NN layers. The sequential model 102B can be applied to a set of session slots associated with interaction information received from a set of users. Based on the application of the sequential model 102B, a set of recommended items can be determined for users within the set of users. In embodiments, the sequential model 102B can be at least one of the following: a hierarchical recurrent neural network (HRNN), a recurrent neural network (RNN), a short-term attention / memory prioritization (STAMP) model, a neural attention recommender machine (NARM) model, or a transformer model. An HRNN can be a stack of RNNs that can be used to model sequential data. An RNN can be a type of neural network that can be associated with memory. In this document, the output of a previous step of an RNN can be fed into the input layer at the current step of the RNN. Further details relating to the sequential model 102B can be similar to those of the reinforcement learning model 102A without departing from the scope of this disclosure. Therefore, such details are omitted for the sake of brevity.

[0033] K-means clustering model 102C can be a machine learning model that can be applied to the embedding vectors associated with each item in a set of items 112 based on received text descriptions (e.g., text description 114A). K-means clustering model 102C can cluster the determined embedding vectors associated with each item in the set of items 112 into a set of labels. Further details relating to K-means clustering model 102C may be similar to those of reinforcement learning model 102A without departing from the scope of this disclosure. Therefore, such details are omitted for the sake of brevity.

[0034] A set of items 112 may include media content that can be associated with a platform, such as an Internet Direct (OTT) streaming platform, including video content, audio content, and closed-circuit caption text. In some cases, a set of items 112 may include images, text files, etc., associated with a product for sale. Furthermore, a set of items 112 may be stored in a database 106. An electronic device 102 may query items, such as item 112A. The database 106 may provide the queried item 112A to the electronic device 102.

[0035] A set of text descriptions 114 can be associated with a set of items 112. For example, text description 114A can be associated with item 112A. Text description 114B can be associated with item 112B, and so on. In an embodiment, the text description (e.g., text description 114B) that can be associated with each item (e.g., item 112B) can include at least one of the following: movie plot, genre of the item, one or more actors of the item, one or more directors of the item, synopsis of the item, or script of the item.

[0036] In operation, electronic device 102 can receive interaction information 110 from a group of users regarding a set of items 112. Interaction information 110 may include information about one or more items that a user in the group interacts with. If the set of items 112 includes media content associated with an Internet Direct (OTT) platform, interaction information 110 associated with user 116 may indicate one or more items in the set of items 112 that user 116 can consume through the OTT platform. Furthermore, interaction information 110 associated with user 116 may indicate a rating that user 116 might provide for each item in the set of items 112. If the set of items 112 includes media content associated with an e-commerce platform, interaction information 110 associated with user 116 may indicate one or more items in the set of items 112 that user 116 might wish to list, purchase, or rate. In the example, the group of users can include a first user, a second user, and a third user. In this document, interaction information 110 may indicate a first subset of a set of items 112 that a first user may interact with, a second subset of a set of items 112 that a second user may interact with, and a third subset of a set of items 112 that a third user may interact with. For example, in Figure 3 Further details relating to the reception of interactive information 110 are described (at 302).

[0037] Electronic device 102 can receive a text description (e.g., text description 114A) for each item in a set of items 112 (e.g., item 112A). Text description 114A may include details associated with item 112A. In one example, item 112A may be a product for sale on an e-commerce platform. Text description 114A may indicate the specifications of item 112A, the color of item 112A, the attributes of item 112A, the size of item 112A, the price of item 112A, the purpose of item 112A, etc. In another example, item 112A may be media content on an OTT platform. Text description 114A may indicate the actors of item 112A, the genre of item 112A, a summary of item 112A, etc. For example, in... Figure 3 Further details regarding the reception of the text description are described in section 304.

[0038] Electronic device 102 can determine an embedding vector associated with each item (e.g., item 112A) in a set of items 112 based on a received text description (e.g., text description 114A). The embedding vector associated with item 112A can be a multi-dimensional vector, which can be determined based on extracting multiple features associated with item 112A from the received text description 114A. For example, in Figure 3Further details regarding the determination of the embedding vector are described in section 306.

[0039] Electronic device 102 can cluster the identified embedding vectors associated with each item in a set of items 112A (e.g., item 112A) into a set of labels. It can be understood that clustering a dataset can divide the data points of the dataset into groups. In the example, electronic device 102 can use a K-means clustering model 102C to cluster the identified embedding vectors. Electronic device 102 can assign labels to each item in the set of items 112 based on the identified clustering of the embedding vectors associated with the corresponding item (e.g., item 112A). For example, in... Figure 3 Further details regarding the clustering of the determined embedding vectors are described in section 308.

[0040] Electronic device 102 can apply reinforcement learning model 102A to a set of labels based on clustering of determined embedding vectors. In this document, a set of labels and interaction information 110 can be provided as input to reinforcement learning model 102A. In some embodiments, the interaction information 110 can be modified based on assigning a set of labels to a set of items 112. The modified interaction information can be provided as feedback input to reinforcement learning model 102A. For example, in Figure 3 Further details regarding the application of reinforcement learning model 102A are described (at 310).

[0041] Electronic device 102 can determine a set of session slots associated with received user interaction information 110 based on the application of reinforcement learning model 102A. Reinforcement learning model 102A can analyze a set of labels and the received interaction information 110 to determine a set of session slots. The size of the session slots associated with user 116 can correspond to user 116's interaction behavior patterns, user 116's emotions, etc. Furthermore, the set of session slots associated with received user interaction information 110 can be determined according to a set of rules. For example, in... Figure 3 Further details regarding the determination of a set of session slots are described in section 312.

[0042] Electronic device 102 can apply sequence model 102B to a set of session slots associated with a determined set of received user interaction information 110. Sequence model 102B can be a recommendation model. A set of session slots associated with the received interaction information 110 can be provided as input to sequence model 102B. For example, in… Figure 3 Further details regarding the application of the sequential model 102B are described in section 314.

[0043] Electronic device 102 can determine a set of recommended items for user 116 within a group of users based on the application of sequence model 102B. Sequence model 102B can analyze a set of session slots associated with received interaction information 110 from a group of users to determine a set of recommended items based on user 116's preferences. For example, in Figure 3 Further details regarding the determination of a set of recommended items are described in section 316.

[0044] Electronic device 102 can present a determined set of recommended items. In one embodiment, the determined set of recommended items can be presented on a display device associated with electronic device 102. In another embodiment, the determined set of recommended items can be sent to an OTT platform. Hereinafter, the determined set of recommended items can be presented via an OTT platform. User 116 can consume items from the determined set of recommended items based on user selection of corresponding items. In another embodiment, the determined set of recommended items can be sent to an e-commerce platform. Hereinafter, the determined set of recommended items can be presented via an e-commerce platform. User 116 browses the determined set of recommended items and can purchase one or more items from the determined set of recommended items from the e-commerce platform.

[0045] Figure 2 This illustrates an embodiment according to the present disclosure. Figure 1 A block diagram of an exemplary electronic device. Combined with... Figure 1 To explain using elements Figure 2 . refer to Figure 2 An exemplary electronic device 102 is illustrated. Electronic device 102 may include circuitry 202, memory 204, input / output (I / O) device 206, network interface 208, reinforcement learning model 102A, sequential model 102B, and K-means clustering model 102C. Memory 204 may store interaction information 110, a set of items 112, and a set of text descriptions 114. Input / output (I / O) device 206 may include a display device 210.

[0046] Circuit 202 may include suitable logic, circuitry, and / or interfaces that can be configured to execute program instructions associated with different operations to be performed by electronic device 102. Operations may include receiving interactive information, receiving text descriptions, determining embedding vectors, clustering embedding vectors, applying reinforcement learning models, determining session slots, applying neural network models, determining a set of recommended items, and presenting recommended items. Circuit 202 may include one or more processing units that can be implemented as individual processors. In embodiments, one or more processing units may be implemented as integrated processors or processor clusters that collectively perform the functions of one or more dedicated processing units. Circuit 202 can be implemented based on a variety of processor technologies known in the art. Examples of implementations of circuit 202 may include x86-based processors, graphics processing units (GPUs), reduced instruction set computing (RISC) processors, application-specific integrated circuit (ASIC) processors, complex instruction set computing (CISC) processors, microcontrollers, central processing units (CPUs), and / or other control circuitry.

[0047] Memory 204 may include suitable logic, circuitry, interfaces, and / or code, which may be configured to store one or more instructions to be executed by circuitry 202. The one or more instructions stored in memory 204 may be configured to perform different operations of circuitry 202 (and / or electronic device 102). Memory 204 may also be configured to store interactive information 110, a set of items 112, and a set of text descriptions 114. Examples of implementations of memory 204 may include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), hard disk drive (HDD), solid-state drive (SSD), CPU cache, and / or secure digital card (SD card).

[0048] I / O device 206 may include suitable logic, circuitry, interfaces, and / or code that can be configured to receive input and provide output based on the received input. For example, input may include user input indicating a request for a set of recommended items that user 116 can interact with, consume, or purchase through an OTT platform or e-commerce platform. Output may include the determined set of recommended items. I / O device 206 may include display device 210. Examples of I / O device 206 may include, but are not limited to, a display (e.g., a touchscreen), a keyboard, a mouse, a joystick, a microphone, or a speaker. Examples of I / O device 206 may also include Braille I / O devices, such as Braille keyboards and Braille readers.

[0049] Network interface 208 may include suitable logic, circuitry, interfaces, and / or code that can be configured to facilitate communication between electronic device 102 and server 104 via communication network 108. Network interface 208 can be implemented using various known technologies to support wired or wireless communication between electronic device 102 and communication network 108. Network interface 208 may include, but is not limited to, antennas, radio frequency (RF) transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, encoder-decoder (CODEC) chipsets, subscriber identity module (SIM) cards, or local buffer circuitry.

[0050] Network interface 208 can be configured to communicate wirelessly with networks such as the Internet, intranets, wireless networks, cellular telephone networks, wireless local area networks (LANs), or metropolitan area networks (MANs). Wireless communication can be configured to use one or more of several communication standards, protocols, and technologies, such as Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), Fifth Generation (5G) New Radio (NR), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, or IEEE 802.11n), Voice over Internet Protocol (VoIP), Li-Fi, Wi-MAX, email protocols, instant messaging, and Short Message Service (SMS).

[0051] Display device 210 may include suitable logic, circuitry, and interfaces that can be configured to display or present a determined set of recommended items. Display device 210 may be a touchscreen that allows a user (e.g., user 116) to provide user input via display device 210. The touchscreen may be at least one of a resistive touchscreen, a capacitive touchscreen, or a thermal touchscreen. Display device 210 may be implemented using several known technologies, such as, but not limited to, at least one of: liquid crystal display (LCD), light-emitting diode (LED) display, plasma display, or organic LED (OLED) display technology, or other display devices. According to embodiments, display device 210 may refer to a display screen of a head-mounted device (HMD), smart glasses device, see-through display, projection-based display, electrochromic display, or transparent display. For example, in Figure 3 The various operations of circuit 202 for creating homogeneous sessions using machine learning (ML) models are further described.

[0052] Figure 3This is a diagram illustrating an exemplary processing pipeline for creating homogeneous sessions using a machine learning (ML) model, according to embodiments of this disclosure. Combined with... Figure 1 and Figure 2 To explain using elements Figure 3 . refer to Figure 3 An exemplary processing pipeline 300 is illustrated, showing exemplary operations 302 to 318 for implementing the creation of homogeneous sessions using a machine learning (ML) model. Exemplary operations 302 to 318 can be performed by any computing system, for example, by... Figure 1 Electronic device 102 or by Figure 2 Circuit 202 is executed. Figure 3 It also includes a reinforcement learning model 102A, a sequential model 102B, a K-means clustering model 102C, interaction information 110, a set of items 112, a set of text descriptions 114, an embedding vector 306A, a set of labels 308A, a set of session slots 312A, and a set of recommended items 316A.

[0053] At 302, the operation of receiving interactive information can be performed. Circuit 202 can be configured to receive interactive information 110 from a group of users for a set of items 112. Users in the group can interact with one or more items in the set of items 112 based on their respective interests. In one example, an interaction between a user (such as user 116) and an item (such as item 112A) (e.g., a movie or product) can indicate whether the user has watched or purchased item 112A. In another example, an interaction between a user (such as user 116) and an item (such as item 112A) (e.g., a movie or product) can indicate a rating provided by the user for item 112A. Therefore, the interactive information 110 associated with user 116 can indicate one or more items in the set of items 112 that user 116 may interact with.

[0054] In this embodiment, a set of sessions associated with each user in a group of users can be received. Typically, the size of each session in the set of sessions associated with each user in the group can be fixed. In this example, the duration of each session can be thirty minutes. Interaction information 110 can be determined based on the received set of sessions.

[0055] In an embodiment, the received interaction information 110 may correspond to a user interaction matrix between a group of users and a set of items 112. Entries in the cells of the user interaction matrix may correspond to interactions between a user and the item associated with the corresponding cell in the user interaction matrix. It is understood that the user interaction matrix may be constructed based on a set of sessions associated with each user in the group. In the example, the group of users may include user "P" and user "Q". User "P" may interact with items 112A, 112D, and 112M in the set of items 112 during a first session. User "P" may interact with items 112C, 112E, and 112F in the set of items 112 during a second session. User "Q" may interact with items 112A, 112E, and 112F in the set of items 112 during a first session. User "Q" may interact with items 112C, 112M, and 112D in the set of items 112 during a second session. Therefore, the number of sessions associated with each user in the group may be "2". To determine the user interaction matrix, users "P" and "Q" can be arranged along the "Y" direction, and a set of items 112 can be arranged along the "X" direction. Therefore, the user interaction matrix has 2 rows and N columns. Each cell of the user interaction matrix corresponds to an interaction between a user and an item. That is, the first cell of the user interaction matrix corresponds to the interaction between user "P" and item 112A. Since user "P" interacts with item 112A, the entry in the first cell can be "1". Similarly, the second cell of the user interaction matrix corresponds to the interaction between user "P" and item 112B. Since user "P" does not interact with item 112A, the entry in the first cell can be "0". Similarly, the entries in the other cells of the user interaction matrix can be determined.

[0056] At point 304, the operation of receiving text description information can be performed. Circuit 202 can be configured to receive a text description of each item in a set of items 112. In this document, a set of text descriptions 114 associated with the set of items 112 can be received. The text description of an item (e.g., item 112A) (e.g., text description 114A) can provide a textual description or narrative of the corresponding item. In the example, the text description of an item (e.g., item 112A) (e.g., text description 114A) can be received from an OTT platform.

[0057] In embodiments, the text description (e.g., text description 114A) of each project (e.g., project 112A) may include at least one of the following: a movie plot, the genre of the project, one or more actors of the project, one or more directors of the project, a synopsis of the project, or a script of the project. If the project is a specific film, the movie plot may provide information about one or more events that may occur in the film (e.g., project 112A). The genre of the project (e.g., project 112A) may indicate the type of the project (e.g., project 112A). In the example, the project (e.g., project 112A) may be media content. In this document, the genre of the project (e.g., project 112A) may be an action film, a romance film, a science fiction film, a horror film, a musical film, etc. The synopsis of the project may briefly describe the project. The script of the project may be the project's script.

[0058] At 306, the embedding vector determination operation can be performed. Circuit 202 can be configured to determine an embedding vector 306A associated with each item in a set of items 112 based on received text descriptions (such as text description 114A). The received text descriptions associated with items (such as item 112A) (such as text description 114A) can be analyzed to determine a set of features associated with the corresponding item. The embedding vector associated with item 112A can be a multi-dimensional vector, which can be determined based on multiple features associated with item 112A extracted from the received text description 114A. In the example, if item 112A is a product or service sold through an e-commerce platform, the multiple features associated with item 112A could be the specifications of item 112A, the color of item 112A, the attributes of item 112A, the size of item 112A, the price of item 112A, the purpose of item 112A, etc. In another example, if Project 112A is media content hosted on an OTT platform, several characteristics associated with Project 112A could be the actors in Project 112A, the genre of Project 112A, the length of Project 112A, etc.

[0059] At 308, an embedding vector clustering operation can be performed. Circuit 202 can be configured to cluster the identified embedding vectors 306A associated with each item in a set of items 112 into a set of labels 308A. It is understood that clustering can be a technique that groups an unlabeled dataset into a set of clusters. The identified embedding vectors 306A associated with each item in a set of items 112 can be clustered into a set of labels 308A. In this document, a set of labels can be a group to which the identified embedding vectors 306A associated with each item in a set of items 112 can be clustered.

[0060] In an embodiment, the clustering of the determined embedding vectors 306A can be based on a K-means clustering model 102C. The K-means clustering model 102C can be a machine learning model that employs an unsupervised learning algorithm, such as the elbow method, to cluster the determined embedding vectors 306A associated with each item in a set of items 112 into a set of labels 308A.

[0061] In an embodiment, circuit 202 can be configured to determine a set of clusters associated with the set of items 112 based on the determined embedding vector 306A associated with each item in the set of items 112. Hereinafter, the elbow method can be applied to the determined embedding vector 306A associated with each item in the set of items 112. Subsequently, a set of clusters associated with the set of items 112 can be determined. In the example, a number of clusters, "k", can be determined.

[0062] Circuit 202 can also be configured to determine centroids associated with each cluster in the determined set of clusters. When determining the set of clusters, the centroids associated with each cluster in the determined set of clusters can be determined randomly. In the example, a number of "k" centroids can be determined based on a set of embedding vectors associated with a set of items 112. In another example, the number of "k" centroids can be determined in a way that differs from the determined set of embedding vectors.

[0063] Circuit 202 can also be configured to determine the distance of each item to a determined centroid associated with each cluster in a determined set of clusters. In this document, the distance can be Euclidean distance, Manhattan distance, Minkowski distance, or Hamming distance. For example, the Euclidean distance of item 112A to each centroid of “k” numbers of centroids can be determined. Similarly, the Euclidean distance of item 112B to each centroid of “k” numbers of centroids can be determined.

[0064] Circuit 202 can also be configured to determine the nearest cluster to each item based on the determined distances. In this paper, the determined distances of each item to the determined centroids associated with each cluster in the determined set of clusters can be analyzed to determine the nearest cluster for each item. In the example, the number of centroids can be four. The Euclidean distance between item 112A and the first centroid can be “2.7” units, the Euclidean distance between item 112A and the second centroid can be “1.1” units, the Euclidean distance between item 112A and the third centroid can be “3.2” units, and the Euclidean distance between item 112A and the fourth centroid can be “2.9” units. Therefore, the second centroid can be the closest to item 112A. Similarly, the nearest centroid to each other item in the set of items 112 can be determined.

[0065] Circuit 202 can also be configured to assign a label to each item in a set of items based on the determined nearest cluster, wherein clustering the determined embedding vectors into a set of labels 308A can also be based on assigning a label to each item. The label can be the name of the cluster. In the example, the second centroid can be closest to item 112A. Therefore, the label assigned to item 112A could be "second".

[0066] At 310, the operation of applying the reinforcement learning model can be performed. Circuit 202 can be configured to apply reinforcement learning model 102A to a set of labels 308A based on the clustering of the determined embedding vectors 306A. In this paper, the determined set of labels 308A can be provided as input to reinforcement learning model 102A. For example, in Figure 4 Further details regarding the application of reinforcement learning model 102A are provided.

[0067] At 312, a set of session slots can be determined. Circuit 202 can be configured to determine a set of session slots 312A associated with received user interaction information 110 based on the application of reinforcement learning model 102A. In an embodiment, when determining a set of labels 308A, the interaction information 110, together with the determined set of labels 308A, can be provided as input to reinforcement learning model 102A. That is, the interaction information 110 (such as a user interaction matrix) can be modified by replacing each item in the set of items 112 with the determined labels associated with the corresponding items. The modified interaction information can be provided as input to reinforcement learning model 102A. Reinforcement learning model 102A can analyze the modified interaction information to determine a set of session slots 312A. In this document, the length of a session slot can differ from other session slots in the set of session slots 312A. That is, the duration of each session slot for each user can be different and can be based on the user's emotions, choices, and interaction behavior patterns. Furthermore, each session slot in the determined set of session slots 312A can be a homogeneous session slot. That is, the number of dominant items in each session slot in the determined set of session slots 312A can be 1.

[0068] In an embodiment, reinforcement learning model 102A may include a controller model configured to determine a set of session slots (e.g., a set of session slots 312A) associated with each user in a set of users based on a set of rules. Reinforcement learning model 102A may also include an evaluator model configured to reward or penalize the controller model based on a set of rules. The controller model may be a machine learning model that can be applied to a set of labels 308A. The controller model may analyze interaction information 110 and a set of labels 308A to determine a set of session slots 312A associated with each user based on a set of rules. The evaluator model may be a machine learning model that can be applied to the determined set of session slots 312A. The evaluator model may evaluate the determined set of session slots 312A to determine whether the determined set of session slots 312A corresponds to a set of rules. If the determined set of session slots 312A corresponds to a set of rules, the evaluator model may reward the controller model. If the determined set of session slots 312A violates at least one rule in a set of rules, the evaluator model can penalize the controller model. In such a case, the controller model can analyze the interaction information 110, a set of labels 308A, and the determined set of session slots 312A to determine the optimal set of session slots.

[0069] In an embodiment, a set of rules may include at least one of the following: a first rule corresponding to a session slot being a set of user item interaction timestamps; a second rule corresponding to the dominance of one item in a set of items 112 in a session slot; a third rule corresponding to the difference in the dominance of items between two consecutive session slots; a fourth rule corresponding to the minimum number of items in a session slot being equal to 2; or a fifth rule corresponding to the flexibility in the size of the session slot.

[0070] The first rule states that a session slot can be a set of user item interaction timestamps. That is, a given session slot can indicate one or more items that a user interacted with at one or more timestamps. For example, a session slot can indicate that from timestamp "T0" to timestamp "T1", user 116 may have interacted with item 112A; from timestamp "T1" to timestamp "T2", user 116 may have interacted with item 112B; and from timestamp "T2" to timestamp "T3", user 116 may have interacted with item 112D.

[0071] The second rule states that only one item from a set of items 112 can be dominant in a session slot. In the example, a comedy genre item can be dominant in a session slot. For instance, a session slot could indicate that a user might interact with items 112A, 112B, 112D, 112L, and 112N. Items 112A, 112D, 112L, and 112N could be a music film starring actor "John". Item 112B could be a comedy film starring actor "Greg". Therefore, the number of dominant items in a session slot could be "1".

[0072] The third rule states that the dominant items for two consecutive session slots can be different. In the example, a set of session slots 312A may include a first session slot and a second session slot. The dominant item for the first session slot may be an action movie, and the dominant item for the second session slot may be a comedy movie.

[0073] The fourth rule states that the minimum number of items in a session slot can be equal to two. A session slot can indicate user 116's behavioral patterns, preferences, and choices. For example, a session slot could indicate that the user might like a comedy movie starring actor "X". To infer user 116's preferences, a session slot can include at least two items.

[0074] The fifth rule states that the size of a session slot can be flexible. Typically, the size of a session slot can be fixed. For example, a regular session slot could be "30" minutes. However, the size of the session slot in this disclosure can be customized based on the received interaction information 110. In the example, items 112A, 112B, 112C, 112D, 112E, and 112F could be a comedy genre starring actress "Mary". Items 112G, 112H, 112I, 112J, 112K, 112L, 112M, and 112N could be a horror genre starring actor "George". Furthermore, the received interaction information 110 may state that a first user (e.g., user 116) may sequentially interact with items 112A, 112B, 112C, 112D, 112F, 112G, 112J, 112K, and 112L from a set of items 112. The received interaction information 110 may also state that a second user (e.g., user 116) may sequentially interact with items 112A, 112B, 112D, 112E, 112G, 112H, 112M, 112N, 112F, and 112C. The first user may initially watch comedy items and then may watch horror items. Since a change in item type may occur when the first user switches from comedy to horror, the first session associated with the first user may include items 112A, 112B, 112C, 112D, and 112F. The second session associated with the first user may include items 112J, 112K, and 112L. With each item being "1" minute, the length of the first session associated with the first user may be "5" minutes, and the length of the second session associated with the first user may be "3" minutes. Similarly, the second user may initially watch a comedy item, then a horror item, and finally switch back to a comedy item. Therefore, the first session associated with the second user may include items 112A, 112B, 112D, and 112E. The second session associated with the second user may include items 112G, 112H, 112M, and 112N. The third session associated with the second user may include items 112F and 112C. Therefore, the length of the first session associated with the first user may be "4" minutes, the length of the second session associated with the first user may be "4" minutes, and the length of the third session associated with the first user may be "2" minutes. Therefore, the size of a session slot can be flexibly determined based on factors such as the user's interaction behavior, the user's emotions, and the user's preferences.

[0075] In an embodiment, circuit 202 can also be configured to apply a controller model to a set of tags 308A. Circuit 202 can also be configured to determine a session slot index based on the application of the controller model. Determining a set of session slots 312A can also be based on the determined session index. Circuit 202 can also be configured to apply an evaluator model to each session slot in the determined set of session slots 312A. Circuit 202 can also be configured to determine an evaluation result based on the application of the evaluator model. The controller model can be configured to optimize the set of session slots 312A based on the determined evaluation result. Based on a set of tags 308A and received interaction information 110, modified interaction information associated with each user in a set of users can be determined. The modified interaction information can include a sequentially arranged set of items. Each item in the sequentially arranged set of items can be labeled according to a set of tags 308A. The modified interaction information can be provided as input to the controller model for policy creation. Based on the application of the controller model, a session slot index can be determined. The session slot index can be an index of a session slot in a set of session slots 312A. When determining the session slot index, the session slot associated with the session slot index determined for the user can be identified. In the example, the session slot index could be "3". In this paper, a third session slot can be determined for user 116. Similarly, each other session slot in a set of session slots 312A can be determined. An evaluator model can then be applied to each session slot in the set of session slots 312A. The evaluator model can analyze each session slot in the set of session slots 312A to determine an evaluation result. Based on the evaluation result, a reward or penalty controller model can be applied.

[0076] In an embodiment, circuit 202 may also be configured to determine the number of dominant items in each session slot of a set of session slots 312A. Circuit 202 may also be configured to determine whether the number of dominant items (i.e., items of the dominant category) in each session slot of a set of session slots 312A is 1. Circuit 202 may also be configured to determine an evaluation result as 0 or a positive number based on the determination that the number of dominant items in each session slot of a set of session slots 312A is 1. When determining a set of session slots 312A, the determined set of session slots 312A can be provided to the evaluator model. The evaluator model can analyze each session slot in the determined set of session slots 312A to determine whether the number of dominant items in each session slot of a set of session slots 312A is 1. If the number of dominant items in each session slot of a set of session slots 312A is 1, it can be determined that the controller model may have followed the third rule in a set of rules in determining the set of session slots 312A. Therefore, in such a case, the controller model can be rewarded. In this paper, the evaluation result can be defined as 0 or a positive number (such as "1").

[0077] In an embodiment, circuit 202 may also be configured to determine the evaluation result as negative based on the determination that the number of dominant items in each session slot in a set of session slots is greater than 1.

[0078] If the number of dominant items in each session slot of a set of session slots 312A is greater than 1, it can be determined that the controller model may have skipped or failed to apply the third rule in a set of rules during the determination of the set of session slots 312A. Therefore, in such cases, the controller model can be penalized. In this paper, the evaluation result can be determined as a negative number, such as "-1". The controller model can then optimize the set of session slots 312A based on the determined evaluation result, such that the optimized set of session slots corresponds to the third rule in the set of rules.

[0079] At 314, the operation of applying the NN model can be performed. Circuit 202 can be configured to apply the sequential model 102B to a set of session slots 312A associated with the received user interaction information 110. The sequential model 102B can be a machine learning model that can be used for recommendation tasks. The set of session slots 312A associated with the received user interaction information 110 can be provided as input to the sequential model 102B. In embodiments, the sequential model 102B can be at least one of the following: hierarchical recurrent neural network (HRNN), recurrent neural network (RNN), short-term attention / memory prioritization (STAMP) model, neural attention recommender machine (NARM) model, or transformer model.

[0080] At point 316, the operation of determining recommended items can be performed. Circuit 202 can be configured to determine a set of recommended items 316A for user 116 in a group of users based on the application of sequence model 102B. Sequence model 102B can analyze a set of session slots 312A to determine a set of recommended items 316A for user 116. The set of recommended items 316A for user 116 can correspond to a set of features associated with user 116, such as user 116's preferences, user 116's mood, etc. In the example, based on a set of session slots 312A, sequence model 102B can infer that user 116 may like items in the comedy genre starring actor "A". Subsequently, items "X", "Y", and "Z" can be determined, which may be in the comedy genre and may be starring actor "A".

[0081] At point 316, the operation of presenting recommended items can be performed. Circuit 202 can be configured to present a determined set of recommended items 316A. In an embodiment, the determined set of recommended items 316A can be presented on display device 210. For example, the determined set of recommended items 316A may include items 112A, 112G, and 112L from a set of items 112. A user interface (UI) associated with an over-the-air (OTA) application or an Internet Direct Access (OTT) platform can be displayed on display device 210. The UI may include a first UI element associated with item 112A, a second UI element associated with item 112G, and a third UI element associated with item 112L. User input, including a user selection of item 112G from the determined set of recommended items 316A, can be received based on user interaction with the second UI element. Subsequently, based on the received user input, item 112G can be presented on display device 210.

[0082] Therefore, the disclosed electronic device 102 can employ an ML model to create homogeneous sessions. The electronic device 192 can employ a reinforcement learning model 102A to determine a set of session slots 312A associated with the received interaction information 110 of a group of users. The length of each session slot in the determined set of session slots 312A can be customized based on the interaction pattern of the user (e.g., user 116) associated with the corresponding session slot. That is, the sizes of two or more session slots in a set of session slots can be different. In the example, the group of users can include teenage users and middle-aged users. Teenage users may have unstable interaction behavior, while middle-aged users may have stable interaction behavior. That is, teenage users may switch from one type of item to another faster than middle-aged users. Therefore, the size of the determined set of session slots 312A associated with teenage users can be smaller than the size of the determined set of session slots 312A associated with middle-aged users. Furthermore, the size of the determined set of session slots 312A can vary depending on the mood of the user associated with the set of session slots 312A. The determined set of session slots 312A can be used as input to the sequential model 102B to determine a set of recommended items 316A for user 116 in a set of users. For example, a set of recommended items 316A, such as movies, can be presented on an Internet Direct (OTT) platform. Furthermore, in some cases, a set of recommended items 316A can be presented on an e-commerce platform. The set of recommended items 316A can then be analyzed, and user 116 can purchase one or more items from the set of recommended items 316A. Furthermore, in some embodiments, the maximum number of dominant categories in each session slot of the determined set of session slots 312A can be "1". This constraint allows the sequential model 102B to recommend a set of recommended items 316A that is likely optimal and corresponds to user 116's preferences. Therefore, since each session slot can correspond to a single type of item (e.g., action movies or comedy movies), recommendations based on such homogeneous slots can have higher accuracy and relevance for a specific user. In other words, cohesive session slots with items of a single dominant category can help recommend items that are likely to be more popular with the user.

[0083] Figure 4 This is a diagram illustrating an exemplary scenario for determining a set of session slots according to embodiments of the present disclosure. Combined with information from... Figure 1 , Figure 2 and Figure 3 To describe the elements Figure 4 . refer to Figure 4 An exemplary scenario 400 is shown. Scenario 400 may include interaction information 110, a set of labels 402, a controller model 404, and an evaluator model 406. Figure 4It also includes operations 408, 410, 412, and 414, which can be executed by any computing system, for example, by... Figure 1 The electronic device 102 performs the operation, or is performed by... Figure 2 Circuit 202 is executed. A set of operations associated with scenario 400 is described in this document.

[0084] refer to Figure 4 A set of labels 402 and interaction information 110 can be provided as input to the controller model 404. In an embodiment, based on a set of labels 308A and the received interaction information 110, modified interaction information associated with each user in a group of users (e.g., user 116) can be determined. The modified interaction information may include a sequentially arranged set of items. Each item in the sequentially arranged set of items can be labeled according to a set of labels 308A. The modified interaction information can be provided as input to the controller model 404 for policy creation.

[0085] At 408, the session slot index determination operation can be performed. Circuit 202 can be configured to determine the session slot index based on the application of controller model 404. The session slot index can be an index of a session slot in a set of session slots 312A. In the example, the determined session slot index can be "5".

[0086] At 408, a set of session slot determination operations can be performed. Circuit 202 can be configured to also determine a set of session slots 312A based on the determined session index. In the example, the session slot index could be "5". In this document, a fifth session slot can be determined for user 116. Similarly, each other session slot in the set of session slots 312A for user 116 can be determined. Each session slot can be determined based on a set of rules.

[0087] At 410, the operation of determining the session slot stack can be performed. Circuit 202 can be configured to stack a determined set of session slots 312A. In the example, the number of session slots in the determined set of session slots 312A can be "10". In this document, "10" session slots can be stacked together to determine a stacked set of session slots. The stacked set of session slots can be provided as input to evaluator model 406. Evaluator model 406 can use a set of rules to analyze each session slot in the determined set of session slots 312A to determine an evaluation result. If each session slot in the determined set of session slots 312A corresponds to a set of rules, then evaluator model 406 can reward controller model 404. If at least one session slot in the determined set of session slots 312A violates at least one rule in the set of rules, then evaluator model 406 can penalize controller model 404. The evaluation result can be fed back to controller model 404. Controller model 404 can update the policy associated with controller model 404 based on the feedback. Subsequently, controller model 404 can optimize a set of session slots 312A based on the determined evaluation results.

[0088] It should be noted that Figure 4 Scenario 400 is for illustrative purposes and should not be construed as limiting the scope of this disclosure.

[0089] Figure 5 This is a diagram illustrating an exemplary scenario for creating homogeneous sessions using a machine learning (ML) model, according to embodiments of this disclosure. Combined with... Figure 1 , Figure 2 , Figure 3 and Figure 4 To describe the elements Figure 5 . refer to Figure 5 An exemplary scenario 500 is illustrated. Scenario 500 may include a set of fixed-duration sessions 502, a user interaction matrix 504, a set of session slots 506, a hierarchical recurrent neural network (HRNN) 508, a set of recommendation items 510, an Internet Direct Access (OTT) application 512, and a reinforcement learning model 102A. The set of fixed-duration sessions 502 may include fixed-duration sessions 502A…fixed-duration sessions 502N.

[0090] Figure 5 The N fixed-duration sessions shown are merely examples. Without departing from the scope of this disclosure, database 106 may include information about only two fixed-duration sessions or more than N fixed-duration sessions. For the sake of brevity, Figure 5Only N fixed-duration sessions are shown. However, in some embodiments, more than N fixed-duration sessions may exist without limiting the scope of this disclosure. Figure 5 The document also shows a user 116 who can associate with or operate electronic device 102. A set of operations related to scenario 500 is described herein.

[0091] refer to Figure 5 A set of fixed-duration sessions 502 can be associated with user 116. Each fixed-duration session in the set of fixed-duration sessions 502 associated with user 116 can be the same size. For example, the duration of each fixed-duration session in the set of fixed-duration sessions 502 can be "15" minutes. Furthermore, each fixed-duration session can include multiple timestamps at which user 116 can switch from one project to another. For example, fixed-duration session 502A can include a timestamp t. 11 timestamp t 12 timestamp t 13 and timestamp t 14 At timestamp t 11 At this point, user 116 can switch from the first comedy project in a set of projects 112 to the second comedy project. At timestamp t 12 At this point, user 116 can switch from the second comedy item in a set of items 112 to the first action item. At timestamp t 13 At this point, user 116 can switch from the first action item in a set of items 112 to the third comedy item. At timestamp t 14 At this point, user 116 can switch from the third comedy project in a set of projects 112 to the first drama project.

[0092] Based on a set of fixed-duration sessions 502, a user interaction matrix 504 can be constructed. A set of items 112 can be arranged along the horizontal axis of the user interaction matrix 504, and a group of users can be arranged along the vertical axis of the user interaction matrix 504. Furthermore, each cell of the user interaction matrix 504 can correspond to an interaction between a user and the item associated with that cell. For example, cell 504A can indicate the interaction between the first user in a group (such as user 116) and item 112A in the set of items 112. The entry in cell 504A can be "1". Therefore, user 116 may interact with item 112A. If the entry in the cell is "0", then there may be no interaction between the user and the item associated with the corresponding cell.

[0093] The user interaction matrix 504 can be fed as input to the reinforcement learning model 102A. Based on the application of the reinforcement learning model 102A, a set of session slots 506 associated with the user interaction matrix 504 can be determined. The set of session slots 506 can be customized based on the interaction behavior of each user in the set of users. The set of session slots 506 associated with the user interaction matrix 504 of the set of users can be fed as input to the HRNN 508. The HRNN 508 can analyze the set of session slots 506 and provide a set of recommended items 510 for user 116 as output. The set of recommended items 510 can be presented on the Internet Direct Application 512. User 116 can consume the selected item by choosing the corresponding item from the set of recommended items 510.

[0094] It should be noted that Figure 5 Scenario 500 is for illustrative purposes and should not be construed as limiting the scope of this disclosure.

[0095] Figure 6 This is a diagram illustrating an exemplary scenario for recommending items based on a hierarchical recurrent neural network (HRNN) according to embodiments of the present disclosure. Combined with... Figure 1 , Figure 2 , Figure 3 , Figure 4 and Figure 5 To describe the elements Figure 6 . refer to Figure 6 An exemplary scenario 600 is illustrated. Scenario 600 may include OTT data 602, a preprocessor 604, a bidirectional encoder representation model 606 from a transformer (BERT), a description embedding 608, cinematic lens data 610, a cinematic identifier (ID) embedding 612, an HRNN 508, a set of session slots 614, and a set of recommended items 616. A set of operations related to scenario 600 is described herein.

[0096] OTT data 602 can be received from an OTT application (e.g., OTT application 512). In the example, OTT data 602 may include a text description of media content (i.e., items) hosted on an OTT platform. A preprocessor 604 can preprocess the OTT data 602. For example, the preprocessor 604 may apply various natural language processing techniques required to preprocess the OTT data 602 into text that can be input into the BERT model 606. In the example, the preprocessor 604 may perform steps such as word segmentation, case conversion, stop word removal, stemming, and lemmatization. The preprocessed OTT data 602 can be provided as input to the BERT model 606. Based on applying the BERT model 606 to the preprocessed OTT data 602, a description embedding 608 can be determined. Furthermore, circuit 202 can receive movie lens data 610, which may correspond to a dataset of movie ratings or tags provided by a large number (e.g., thousands) of users. Based on the received movie lens data 610, a movie identifier (ID) embedding 612 can be determined. A set of session slots 614 can be associated with user 116, and for example, can be as follows: Figure 3 As described in (at 312), the movie identifier (ID) can be determined. The determined movie identifier (ID) embedding 612 and a set of session slots 614 can be provided as input to HRNN 508. HRNN 508 can output a set of recommended items 616 for a user (e.g., user 116).

[0097] It should be noted that Figure 6 Scenario 600 is for illustrative purposes and should not be construed as limiting the scope of this disclosure.

[0098] Figure 7 This is a flowchart illustrating operations of an exemplary method for creating homogeneous sessions using a machine learning (ML) model, according to embodiments of this disclosure. Combined with... Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 To describe the elements Figure 7 . refer to Figure 7 The flowchart 700 is shown. Flowchart 700 may include operations from 702 to 720, and can be... Figure 1 This can be achieved through electronic device 102, or by means of... Figure 2 The circuit 202 is used to implement this. Flowchart 700 can start at 702 and proceed to 704.

[0099] At 704, a set of user interaction information 110 for a set of items 112 can be received. Circuit 202 can be configured to receive a set of user interaction information 110 for a set of items 112. For example, in Figure 3 Further details regarding the reception of interactive information 110 are provided (at 302).

[0100] At 706, a text description (e.g., text description 114) for each item in a set of items 112 (e.g., item 112A) can be received. Circuit 202 can be configured to receive a text description (e.g., text description 114) for each item in a set of items 112 (e.g., item 112A). For example, in Figure 3 Further details regarding the reception of text description 114 are provided (at 304).

[0101] At 708, an embedding vector 306A associated with each item in a set of items 112 (e.g., item 112A) can be determined based on the received text description (e.g., text description 114). Circuit 202 can be configured to determine the embedding vector 306A associated with each item in a set of items 112 (e.g., item 112A) based on the received text description (e.g., text description 114). For example, in Figure 3 Further details regarding the determination of the embedding vector 306A (at 306) are provided.

[0102] At 710, the identified embedding vectors 306A associated with each item in the set of items 112 can be clustered into a set of labels 308A. Circuit 202 can be configured to cluster the identified embedding vectors 306A associated with each item in the set of items 112 into a set of labels 308A. For example, in Figure 3 Further details regarding the clustering of embedding vector 306A are provided (at 308).

[0103] At 712, a reinforcement learning model 102A can be applied to a set of labels 308A based on the clustering of the determined embedding vectors 306A. Circuit 202 can be configured to apply the reinforcement learning model 102A to a set of labels 308A based on the clustering of the determined embedding vectors 306A. For example, in Figure 3 Further details regarding the application of reinforcement learning model 102A are provided (at 310).

[0104] At 714, a set of session slots 312A associated with the received user interaction information 110 can be determined based on the application of reinforcement learning model 102A. Circuit 202 can be configured to determine a set of session slots 312A associated with the received user interaction information 110 based on the application of reinforcement learning model 102A. For example, in Figure 3 Further details regarding the determination of a set of session slots 312A are provided (at 312).

[0105] At 716, a sequence model 102B can be applied to a set of session slots 312 associated with the received user interaction information 110. Circuit 202 can be configured to apply sequence model 102B to a set of session slots 312 associated with the received user interaction information 110. For example, in Figure 3 Further details regarding the application of the sequential model 102B are provided (at 314).

[0106] At 718, a set of recommended items 316A for user 116 in a group of users can be determined based on the application of the sequential model 102B. Circuit 202 can be configured to determine a set of recommended items 316A for user 116 in a group of users based on the application of the sequential model 102B. For example, in Figure 3 Further details regarding the determination of a set of recommended items 316A are provided (at 316).

[0107] At 720, a defined set of recommended items 316A can be presented. Circuit 202 can be configured to present the defined set of recommended items 316A. For example, in Figure 3 Further details regarding the presentation of a set of recommended items 316A are provided (at 318). Controls can be transferred to the end.

[0108] Although flowchart 700 is shown as discrete operations, such as 704, 706, 708, 710, 712, 714, 716, 718, and 720, this disclosure is not limited thereto. Therefore, in some embodiments, depending on the implementation, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated without departing from the spirit of the disclosed embodiments.

[0109] Various embodiments of this disclosure may provide a non-transitory computer-readable medium and / or storage medium having computer-executable instructions stored thereon, which can be executed by a machine and / or a computer to operate electronic devices (e.g., Figure 1Electronic device 102). Such instructions can cause electronic device 102 to perform operations, which may include receiving a set of user interaction information for a set of items (e.g., a set of items 112). Figure 1 The operation may also include receiving a text description (e.g., text description 114A) for each item in a set of items 112 (e.g., item 112A). The operation may also include determining an embedding vector (e.g., based on the received text description (e.g., text description 114A) associated with each item in the set of items 112 (e.g., item 112A). Figure 3 The operation may also include clustering the identified embedding vectors 306A associated with each item in a set of items 112 (e.g., item 112A) into a set of labels (e.g., embedding vectors 306A). Figure 3 The operation may also include applying a reinforcement learning model to a set of labels 308A based on clustering of the determined embedding vectors 306A (e.g., ...). Figure 1 The operation may also include applying the reinforcement learning model 102A to determine a set of session slots associated with the received user interaction information 110 (e.g., reinforcement learning model 102A). Figure 3 A set of session slots 312A). This operation may also include applying a sequence model (e.g.,) to a set of session slots 312A associated with the received interaction information 110 from a set of users. Figure 1 The sequence model 102B). This operation may also include applying the sequence model 102B to determine, for example, a user within a set of users (e.g., ...). Figure 1 A set of recommended items for user 116 (e.g., Figure 3 (A set of recommended items 316A). This operation may also include presenting the identified set of recommended items 316A.

[0110] Exemplary aspects of this disclosure may provide an electronic device (such as, Figure 1An electronic device 102 includes circuitry (such as circuitry 202). Circuitry 202 can be configured to receive a set of user interaction information 110 for a set of items 112. Circuitry 202 can also be configured to receive a text description (e.g., text description 114A) for each item in the set of items 112 (e.g., item 112A). Circuitry 202 can also be configured to determine an embedding vector 306A associated with each item in the set of items 112 (e.g., item 112A) based on the received text description (e.g., text description 114A). Circuitry 202 can also be configured to cluster the determined embedding vectors 306A associated with each item in the set of items 112 (e.g., item 112A) into a set of labels 308A. Circuitry 202 can also be configured to apply a reinforcement learning model 102A to the set of labels 308A based on the clustering of the determined embedding vectors 306A. Circuit 202 can also be configured to determine a set of session slots 312A associated with the received interaction information 110 of a group of users based on the application of reinforcement learning model 102A. Circuit 202 can also be configured to apply sequence model 102B to the set of session slots 312A associated with the received interaction information 110 of a group of users. Circuit 202 can also be configured to determine, based on the application of sequence model 102B, for each user in the group (e.g., ... Figure 1 The circuit 202 can also be configured to present the determined set of recommended items 316A to user 116.

[0111] In an embodiment, the received interaction information 110 may correspond to a user interaction matrix between a group of users and a group of items 112 (e.g., Figure 4 The user interaction matrix 504, and the entries in the cells of the user interaction matrix 504, can correspond to the interaction between a user (e.g., user 116) and an item (e.g., item 112A) associated with the corresponding cell of the user interaction matrix 504.

[0112] In an embodiment, the text description (e.g., text description 114A) of each item (e.g., item 112A) may include at least one of the following: movie plot, genre of item 112A, one or more actors of item 112A, one or more directors of item 112A, synopsis of item 112A, or script of item 112A.

[0113] In this embodiment, the clustering of the determined embedding vector 306A can be based on a K-means clustering model (e.g., Figure 1 K-means clustering model 102C).

[0114] In an embodiment, circuit 202 may also be configured to determine a set of clusters associated with the set of items 112 based on the determined embedding vector 306A associated with each item in the set of items 112 (e.g., item 112A). Circuit 202 may also be configured to determine the centroid associated with each cluster in the determined set of clusters. Circuit 202 may also be configured to determine the distance of each item (e.g., item 112A) from the determined centroid associated with each cluster in the determined set of clusters. Circuit 202 may also be configured to determine the nearest cluster to each item (e.g., item 112A) based on the determined distance. Circuit 202 may also be configured to assign a label to each item in the set of items 112 based on the determined nearest cluster, wherein clustering the determined embedding vector 306A into a set of labels 308A may also be based on assigning labels to each item.

[0115] In an embodiment, reinforcement learning model 102A may include a controller model (e.g., Figure 4 The controller model 404 is configured to determine a set of session slots associated with each user in a set of users (e.g., user 116) based on a set of rules; and the evaluator model (e.g., Figure 4 The evaluator model 406 is configured to reward or punish the controller model 404 based on a set of rules.

[0116] In an embodiment, a set of rules includes at least one of the following: a first rule, which corresponds to a session slot being a set of user item interaction timestamps; a second rule, which corresponds to the dominance of one item in a set of items 112 in a session slot; a third rule, which corresponds to the difference in the dominance of items between two consecutive session slots; a fourth rule, which corresponds to the minimum number of items in a session slot being equal to 2; or a fifth rule, which corresponds to the flexibility in the size of the session slot.

[0117] In an embodiment, circuit 202 can also be configured to target a set of tags (e.g. Figure 4 A set of tags 402) is applied to a controller model 404. Circuit 202 can also be configured to determine a session slot index based on the application of controller model 404, wherein determining a set of session slots 312A can also be based on the determined session index. Circuit 202 can also be configured to apply an evaluator model 406 to each session slot in the determined set of session slots 312A. Circuit 202 can also be configured to determine an evaluation result based on the application of evaluator model 406, wherein controller model 404 can be configured to optimize a set of session slots 312A based on the determined evaluation result.

[0118] In an embodiment, circuit 202 may also be configured to determine the number of dominant items in each session slot of a set of session slots 312A. Circuit 202 may also be configured to determine whether the number of dominant items in each session slot of a set of session slots 312A is 1. Circuit 202 may also be configured to determine the evaluation result as 0 or a positive number based on the determination that the number of dominant items in each session slot of a set of session slots 312A can be 1.

[0119] In an embodiment, circuit 202 may also be configured to determine the evaluation result as negative based on the determination that the number of dominant items in each session slot in a set of session slots 312A can be greater than 1.

[0120] In an embodiment, the sequential model 102B can be at least one of the following: a hierarchical recurrent neural network (HRNN) (e.g., Figure 5 HRNN 508), Recurrent Neural Network (RNN), Short-Term Attention / Memory Prioritization (STAMP) model, Neural Attention Recommender (NARM) model, or Transformer model.

[0121] This disclosure may also reside in a computer program product that includes all features enabling the implementation of the methods described herein and, when loaded into a computer system, the execution of those methods. In this context, a computer program means any expression of a set of instructions in any language, code, or notation, intended to cause a system capable of information processing to perform a particular function, whether directly or after being: a) translated into another language, code, or notation; or b) reproduced in a different material form.

[0122] While this disclosure has been described with reference to certain embodiments, those skilled in the art will understand that various changes can be made and equivalents can be substituted without departing from the scope of this disclosure. Furthermore, many modifications can be made to adapt a particular situation or material to the teachings of this disclosure without departing from its scope. Therefore, this disclosure is not intended to be limited to the disclosed embodiments, but rather is intended to include all embodiments falling within the scope of the appended claims.

Claims

1. An electronic device, comprising: The circuit is configured as follows: Receive interaction information from a group of users regarding a set of items; Receive a text description for each item in the set of projects; The embedding vector associated with each item in the set of projects is determined based on the received text description; Cluster the identified embedding vectors associated with each item in the set of projects into a set of labels; A reinforcement learning model is applied to the set of labels based on the clustering of the determined embedding vectors; The reinforcement learning model is used to determine a set of session slots associated with the received interaction information of the group of users; Apply a sequential model to the set of session slots associated with the received interaction information of the set of users; The sequential model is used to determine a set of recommended items for users within the group of users. as well as Present the identified set of recommended items.

2. The electronic device according to claim 1, wherein, The received interaction information corresponds to the user interaction matrix between the group of users and the group of items, and The entries in the cells of the user interaction matrix correspond to the interactions between the user and the items associated with the corresponding cells in the user interaction matrix.

3. The electronic device according to claim 1, wherein, The text description of each project includes at least one of the following: film plot, genre of the project, one or more actors in the project, one or more directors of the project, project synopsis, or project script.

4. The electronic device according to claim 1, wherein, The clustering of the determined embedding vectors is based on the K-means clustering model.

5. The electronic device according to claim 1, wherein, The circuit is also configured to: A set of clusters associated with the set of projects is determined based on the identified embedding vectors associated with each project in the set of projects; Determine the centroid associated with each cluster in the determined set of clusters; Determine the distance of each item from the determined centroid associated with each cluster in the determined set of clusters; The nearest cluster to each item is determined based on the established distance; as well as A label is assigned to each item in the set of items based on the determined nearest cluster, wherein, Clustering the determined embedding vectors into the set of labels is also based on assigning labels to each item.

6. The electronic device according to claim 1, wherein, The reinforcement learning model includes: A controller model configured to determine the set of session slots associated with each user in the set of users based on a set of rules, and An evaluator model is configured to reward or punish the controller model based on the set of rules.

7. The electronic device according to claim 6, wherein, The set of rules includes at least one of the following: The first rule, corresponding to a session slot, is a set of user item interaction timestamps. The second rule corresponds to the dominance of one item in the set of items within a session slot. The third rule corresponds to the difference in dominant items between two consecutive session slots. The fourth rule corresponds to a minimum number of items in a session slot equal to 2, or The fifth rule corresponds to the flexibility in the size of the session slot.

8. The electronic device according to claim 6, wherein, The circuit is also configured to: The controller model is applied to the set of labels; The session slot index is determined based on the application of the controller model, wherein... The determination of the set of session slots is also based on the determined session index; The evaluator model is applied to each of the determined set of session slots; and The evaluation result is determined based on the application of the aforementioned evaluator model, wherein... The controller model is configured to optimize the set of session slots based on the determined evaluation results.

9. The electronic device according to claim 8, wherein, The circuit is also configured to: Determine the number of dominant items in each session slot within the set of session slots; Determine whether the number of dominant items in each session slot of the set of session slots is 1; as well as The evaluation result is determined to be 0 or a positive number based on the determination that the number of dominant items in each session slot of the set of session slots is 1.

10. The electronic device according to claim 9, wherein, The circuit is also configured to determine the evaluation result as negative based on the determination that the number of dominant items in each session slot of the set of session slots is greater than 1.

11. The electronic device according to claim 1, wherein, The sequential model is at least one of the following: hierarchical recurrent neural network (HRNN), recurrent neural network (RNN), short-term attention / memory priority (STAMP) model, neural attention recommender (NARM) model, or transformer model.

12. A method comprising: In electronic devices: Receive interaction information from a group of users regarding a set of items; Receive a text description for each item in the set of projects; The embedding vector associated with each item in the set of projects is determined based on the received text description; Cluster the identified embedding vectors associated with each item in the set of projects into a set of labels; A reinforcement learning model is applied to the set of labels based on the clustering of the determined embedding vectors; The reinforcement learning model is used to determine a set of session slots associated with the received interaction information of the group of users; Apply a sequential model to the set of session slots associated with the received interaction information of the set of users; The sequential model is used to determine a set of recommended items for users within the group of users. as well as Present the identified set of recommended items.

13. The method according to claim 12, wherein, The received interaction information corresponds to the user interaction matrix between the group of users and the group of items, and The entries in the cells of the user interaction matrix correspond to the interactions between the user and the items associated with the corresponding cells in the user interaction matrix.

14. The method according to claim 12, wherein, The text description of each project includes at least one of the following: film plot, genre of the project, one or more actors in the project, one or more directors of the project, project synopsis, or project script.

15. The method according to claim 12, wherein, The clustering of the determined embedding vectors is based on the K-means clustering model.

16. The method of claim 12, further comprising: A set of clusters associated with the set of projects is determined based on the identified embedding vectors associated with each project in the set of projects; Determine the centroid associated with each cluster in the determined set of clusters; Determine the distance of each item from the determined centroid associated with each cluster in the determined set of clusters; The nearest cluster to each item is determined based on the established distance; as well as A label is assigned to each item in the set of items based on the determined nearest cluster, wherein, Clustering the determined embedding vectors into the set of labels is also based on assigning labels to each item.

17. The method according to claim 12, wherein, The reinforcement learning model includes: A controller model configured to determine the set of session slots associated with each user in the set of users based on a set of rules, and An evaluator model is configured to reward or punish the controller model based on the set of rules.

18. The method according to claim 17, wherein, The set of rules includes at least one of the following: The first rule, corresponding to a session slot, is a set of user item interaction timestamps. The second rule corresponds to the dominance of one item in the set of items within a session slot. The third rule corresponds to the difference in dominant items between two consecutive session slots. The fourth rule corresponds to a minimum number of items in a session slot equal to 2, or The fifth rule corresponds to the flexibility in the size of the session slot.

19. The method of claim 17, further comprising: The controller model is applied to the set of labels; The session slot index is determined based on the application of the controller model, wherein... The determination of the set of session slots is also based on the determined session index; The evaluator model is applied to each of the determined set of session slots; and The evaluation result is determined based on the application of the aforementioned evaluator model, wherein... The controller model is configured to optimize the set of session slots based on the determined evaluation results.

20. A non-transitory computer-readable medium storing computer-executable instructions thereon, the computer-executable instructions causing the electronic device to perform operations when executed by the electronic device, the operations including: Receive interaction information from a group of users regarding a set of items; Receive a text description for each item in the set of projects; The embedding vector associated with each item in the set of projects is determined based on the received text description; Cluster the identified embedding vectors associated with each item in the set of projects into a set of labels; A reinforcement learning model is applied to the set of labels based on the clustering of the determined embedding vectors; The reinforcement learning model is used to determine a set of session slots associated with the received interaction information of the group of users; Apply a sequential model to the set of session slots associated with the received interaction information of the set of users; The sequential model is used to determine a set of recommended items for users within the group of users. as well as Present the identified set of recommended items.

Images