Method and electronic device for interest level prediction based on behavior transition

Abstract

A method of interest level prediction includes providing a Fibonacci confidence interval level regarding user program rating data to differentiate user interest levels for different linear entertainment programs. The method further includes creating a behavioral transition feature space based on the behavioral sequence, the first derivative of the behavioral sequence, and the second derivative of the behavioral sequence to obtain information about user behavior over time. The method further includes mapping user data from the behavioral transition feature space into a prediction of a Fibonacci confidence interval level using a converter structure and attention based on the trained machine learning model.

Description

Technical Field

[0001] The present disclosure generally relates to machine learning systems. More specifically, the present disclosure relates to predicting interest levels based on behavior transitions. Background Art

[0002] A large amount of consumer behavior data is collected by consumer electronic devices. Here, there is a problem of using this data to deliver a meaningful experience that reaches the right audience through television and mobile and desktop devices. Summary of the Invention

[0003] Technical Solution

[0004] The present disclosure relates to predicting interest levels based on behavior transitions.

[0005] In an embodiment, a method includes providing a Fibonacci confidence interval level for user program rating data to distinguish user interest levels for different linear entertainment programs. The method further includes creating a behavior transition feature space based on a behavior sequence, a first derivative of the behavior sequence, and a second derivative of the behavior sequence to obtain information about user behavior over time. The method further includes mapping user data from the behavior transition feature space to a prediction of a Fibonacci confidence interval level based on a trained machine learning model using a transformer structure and attention.

[0006] In an embodiment, an electronic device includes at least one processor configured to provide a Fibonacci confidence interval level for user program rating data to distinguish user interest levels for different linear entertainment programs. The at least one processor is further configured to create a behavior transition feature space based on a behavior sequence, a first derivative of the behavior sequence, and a second derivative of the behavior sequence to obtain information about user behavior over time. The at least one processor is further configured to map user data from the behavior transition feature space to a prediction of a Fibonacci confidence interval level based on a trained machine learning model using a transformer structure and attention.

[0007] In an embodiment, a computer-readable medium contains instructions that, when executed, cause at least one processor to provide a Fibonacci confidence interval level for user program rating data to distinguish user interest levels for different linear entertainment programs. The instructions, when executed, further cause at least one processor to create a behavior transition feature space based on a behavior sequence, a first derivative of the behavior sequence, and a second derivative of the behavior sequence to obtain information about user behavior over time. The instructions, when executed, further cause at least one processor to map user data from the behavior transition feature space to a prediction of a Fibonacci confidence interval level based on a trained machine learning model using a transformer structure and attention.

[0008] In an embodiment, a method includes receiving an advertising campaign request related to a target linear entertainment program, season, or series. The method further includes generating inference data based on a machine learning model trained using: (i) Fibonacci confidence interval levels for user program viewership data to distinguish user interest levels in different linear entertainment programs, and (ii) a behavioral transition feature space based on a behavioral sequence, a first derivative of the behavioral sequence, and a second derivative of the behavioral sequence to obtain information about user behavior over time. The method further includes determining, from the inference data, an inference about the user interest level in the target linear entertainment program, season, or series for one or more users.

[0009] Other technical features may be apparent to those skilled in the art in light of the following figures, description, and claims.

[0010] It may be advantageous to set forth definitions of certain words and phrases used throughout this patent document. The terms "send," "receive," and "communicate," and derivatives thereof, cover both direct and indirect communication. The terms "include" and "comprise," and derivatives thereof, mean including but not limited to. The term "or" is inclusive, meaning and / or. The phrase "associated with," and derivatives thereof, means including, being included within, interconnecting with, containing, being contained within, connected to or with, coupled to or with, capable of communicating with, cooperating with, interlacing, juxtaposing, proximate to, bound to or with, having, having the property of, having a relationship to or with, and the like.

[0011] In addition, the various functions described below can be implemented or supported by one or more computer programs, where each computer program is formed from computer-readable program code and embodied in a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, instruction sets, procedures, functions, objects, classes, instances, related data, or portions thereof that are adapted to be implemented in appropriate computer-readable program code. The phrase "computer-readable program code" includes any type of computer code, including source code, object code, and executable code. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), hard disk drive, compact disc (CD), digital video disc (DVD), or any other type of memory. A "non-transitory" computer-readable medium excludes wired, wireless, optical, or other communication links that transmit transitory electrical or other signals. The computer-readable medium includes media in which data can be stored permanently and media in which data can be stored and later rewritten, such as a rewritable optical disc or an erasable memory device.

[0012] As used herein, terms and phrases such as "having", "may have", "include", or "may include" a feature (such as a number, a function, an operation, or a component such as a part) indicate the presence of the feature and do not exclude the presence of other features. In addition, as used herein, the phrases "A or B", "at least one of A and / or B", or "one or more of A and / or B" may include all possible combinations of A and B. For example, "A or B", "at least one of A and B", and "at least one of A or B" may indicate any of the following: (1) including at least one A, (2) including at least one B, or (3) including at least one A and at least one B. In addition, as used herein, the terms "first" and "second" may modify various components regardless of importance and do not limit the components. These terms are only used to distinguish one component from another. For example, a first user device and a second user device may indicate user devices that are different from each other, regardless of the order or importance of the devices. Without departing from the scope of the present disclosure, the first component may be represented as the second component, and vice versa.

[0013] It will be understood that when an element (such as a first element) is referred to as being "coupled" / "coupled to" another element (such as a second element) (operatively or communicatively) or "connected" / "connected to" another element (such as a second element), it can be coupled or connected to the other element directly or via a third element. In contrast, it will be understood that when an element (such as a first element) is referred to as being "directly coupled" / "directly coupled to" another element (such as a second element) or "directly connected" / "directly connected to" another element (such as a second element), no other element (such as a third element) intervenes between the element and the other element.

[0014] As used herein, the phrase "configured (or set) to" may be used interchangeably with the phrases "suitable for", "capable of", "designed to", "adapted to", "manufactured to", or "able to" depending on the context. The phrase "configured (or set) to" does not essentially mean "specially designed in hardware for". Instead, the phrase "configured to" may indicate that a device is capable of performing an operation together with another device or component. For example, the phrase "a processor configured (or set) to perform A, B, and C" may indicate a general-purpose processor (such as a CPU or an application processor) that can perform the operation by executing one or more software programs stored in a memory device or a dedicated processor (such as an embedded processor) for performing the operation.

[0015] The terms and phrases used herein are for describing embodiments of the present disclosure only and do not limit the scope of other embodiments of the present disclosure. It will be understood that, unless the context clearly dictates otherwise, the singular forms "a", "an", and "the" include plural references. All terms and phrases used herein, including technical and scientific terms and phrases, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of the present disclosure belong. It will be further understood that terms and phrases (such as those defined in a common dictionary) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein. In some instances, the terms and phrases defined herein may be interpreted as excluding embodiments of the present disclosure.

[0016] Examples of an "electronic device" according to an embodiment of the present disclosure may include at least one of the following items: a smart phone, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop computer, a netbook computer, a workstation, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, or a wearable device (such as smart glasses, a head-mounted device (HMD), electronic clothing, an electronic bracelet, an electronic necklace, an electronic accessory, an electronic tattoo, a smart mirror, or a smart watch). Other examples of the electronic device include smart home appliances. Examples of the smart home appliances may include at least one of a television, a digital video disc (DVD) player, an audio player, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, a dryer, an air purifier, a set-top box, a home automation control panel, a security control panel, a TV box (such as SAMSUNG HOMESYNC, APPLE TV, or GOOGLE TV), a smart speaker, or a speaker with an integrated digital assistant (such as SAMSUNG GALAXY HOME, APPLE HOMEPOD, or AMAZON ECHO), a game console (such as XBOX, PLAYSTATION, or NINTENDO), an electronic dictionary, an electronic key, a portable camera, or an electronic photo frame. Still other examples of the electronic device include various medical devices (such as various portable medical measurement devices (e.g., a blood glucose measurement device, a heartbeat measurement device, or a body temperature measurement device), a magnetic resource angiography (MRA) device, a magnetic resource imaging (MRI) device, a computed tomography (CT) device, an imaging device, or an ultrasonic device), a navigation device, a global positioning system (GPS) receiver, an event data recorder (EDR), a flight data recorder (FDR), an in-vehicle infotainment device, marine electronic devices (such as marine navigation devices or gyrocompasses), avionics, security devices, an in-vehicle head unit, industrial or domestic robots, an automated teller machine (ATM), a point of sale (POS) device, or an Internet of Things (IoT) device (such as a light bulb, various sensors, a water meter, a gas meter, a sprinkler, a fire alarm, a thermostat, a street lamp, an oven, a fitness device, a hot water tank, a heater, or a boiler). Other examples of the electronic device include at least a part of a piece of furniture or a building / structure, an electronic board, an electronic signature receiving device, a projector, or various measurement devices (such as devices for measuring water, electricity, gas, or electromagnetic waves). Note that, according to various embodiments of the present disclosure, the electronic device may be one or a combination of the devices listed above. According to an embodiment of the present disclosure, the electronic device may be a flexible electronic device. The electronic device disclosed herein is not limited to the devices listed above and may include new electronic devices depending on the development of technology.

[0017] In the following description, according to various embodiments of the present disclosure, an electronic device is described with reference to the accompanying drawings. As used herein, the term "user" may refer to a person using the electronic device or another device (such as an artificial intelligence electronic device).

[0018] Certain definitions of other words and phrases may be provided throughout this patent document. Those of ordinary skill in the art should understand that in many, if not most, instances, such definitions apply to both the prior and future use of such defined words and phrases.

[0019] None of the descriptions in this application should be construed as implying that any particular element, step, or function is an essential element that must be included within the scope of the claims. The scope of the patent subject matter is defined only by the claims. The applicant understands that the use of any other terms within the claims, including but not limited to "mechanism", "module", "device", "unit", "component", "element", "member", "apparatus", "machine", "system", "processor", or "controller", indicates a structure known to those skilled in the relevant art. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more fully understand the present disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which like reference numerals represent like parts:

[0021] Figure 1 Illustrates an example network configuration including an electronic device according to the present disclosure;

[0022] Figure 2 Illustrates an example system for predicting an interest level based on behavior transitions according to the present disclosure;

[0023] Figure 3A and Figure 3B Illustrates an alternative example of a behavior-to-interest level semantic transfer model for predicting an interest level based on behavior transitions according to the present disclosure;

[0024] Figure 4 Illustrates another example system for predicting an interest level based on behavior transitions according to the present disclosure;

[0025] Figure 5 Illustrates for use with Figure 4 an example model for predicting an interest level based on behavior transitions of a system;

[0026] Figure 6 Illustrates yet another example system for predicting an interest level based on behavior transitions according to the present disclosure;

[0027] Figure 7 Illustrates for use with Figure 6An example model for behavior-transition-based interest level prediction for use with a system;

[0028] Figure 8 Illustrates an example method for training a machine learning model according to the present disclosure; and

[0029] Figure 9 Illustrates an example method for using a trained machine learning model according to the present disclosure.

[0030] Figure 10 Illustrates an example method for training a machine learning model and using the trained machine learning model according to the present disclosure. Detailed Description

[0031] The following discussion is described with reference to the accompanying drawings Figures 1 to 9 and various embodiments of the present disclosure. However, it should be understood that the present disclosure is not limited to these embodiments, and all changes and / or equivalents or substitutions thereof also fall within the scope of the present disclosure. Throughout the specification and the drawings, the same or similar reference numerals may be used to indicate the same or similar elements.

[0032] In the present disclosure, a state-of-the-art behavior-transition-based interest level prediction system is introduced to analyze data from smart devices. The system analyzes user behavior and finds accurate target users for advertisers running campaigns on the device. For various reasons, the system has better performance than other systems. For example, the system can provide target users and user interest levels based on the system's novel viewership-based Fibonacci confidence interval interest separation algorithm. The system can also run in-depth feature analysis using a novel machine learning model on the behavior-transition feature space. Compared with previous systems, the method in the present disclosure can improve the performance of advertising campaigns and may increase significant profits. Utilizing a large dataset supported by automatic content recognition (ACR) and combining third-party data from trusted partners, the system can provide advertisers with real-time entertainment viewing insights from many individuals - such as over 50 million users in the United States and potentially over 200 million devices globally.

[0033] Current rule-based audience generation systems typically have poor performance, especially for cold start problems. Cold start problems can involve the following issues: the system cannot draw accurate inferences for users or items for which insufficient information has been aggregated. Using a machine learning-based system, the present disclosure solves various problems. For example, a common problem involves determining how to perform feature engineering for each campaign, as campaigns on linear entertainment content typically have cold start problems. Different from recommendation systems that usually try to recommend existing items (such as content or items to purchase) to users, the tuning prediction system recommends future programs to linear entertainment consumers. Therefore, the tuning campaign creation can be regarded as a cold start problem. In this case, using an embedding layer (as in most recommendation systems) is not a viable solution because (i) the embedding space will keep growing and (ii) the embedding can only represent existing items, not new items.

[0034] Another common problem involves determining how to predict the user interest level for a campaign program. For advertising, knowing the user interest level for a campaign program may be important or crucial for running a successful campaign. Linear entertainment programs are different from other items because usually users may have certain specific behaviors that can affect the campaign results. For example, some TV shows have loyal fans who will tune in to a new season regardless of whether an advertisement is running on their device. Therefore, from the perspective of results, these users have been converted, and spending the campaign budget on them is not efficient. Other common problems involve determining how to better capture changes in user behavior over time, determining how to build an end-to-end system that can solve these types of problems, and determining how to accelerate the throughput of the entire system to support as many campaigns as possible simultaneously.

[0035] The present disclosure solves the various problems mentioned above by using a machine learning-based system and other features of the described tuning prediction system. For example, an offline behavioral data feature engineering pipeline can be designed to periodically (such as daily) aggregate the behavioral data of users and store the features of user behavior in a behavioral feature database. This process transforms the original behavioral data into vectorized features, such as by using one or more training algorithms with daily sessionization. This process can map all programs on linear entertainment into a predefined feature space and solve the cold start problem. Since the behavioral features can represent daily sessionization features, the behavioral features can be flexibly used as needed or desired. The behavioral features support feature engineering in different post-processing time windows (such as weekly, monthly, quarterly, etc.). General features can be generated daily or at other times and can be reused for different campaigns, which improves the running speed of the pipeline and reduces time and costs.

[0036] The Fibonacci confidence interval level of program rating data about users can be introduced to distinguish the user interest levels in different linear entertainment programs. The rating data can be the ratio of program viewing time. The confidence interval level is flexible and can be scaled up according to the customer requirements of the system, such as advertisers. A behavioral transformation feature space can be designed, which uses traditional behavioral sequences (used in sequential machine learning architectures to represent behavioral changes in many embedding layer-based recommendation systems), and also introduces the first and second derivatives of the behavioral sequences. Using the first and second derivatives of the behavioral sequences and the behavioral sequences themselves helps to create a behavioral transformation feature space for the entire system to better understand user behavior over time. The deep learning model architecture can use a transformer architecture and attention to map user data from the behavioral transformation feature space into the Fibonacci confidence interval level prediction. The pipeline can be designed to move as many components as possible to offline processing to improve the throughput of the system and support as many activities as possible simultaneously.

[0037] Figure 1 FIG. 100 shows an example network configuration including an electronic device according to the present disclosure. Figure 1 The embodiments of the network configuration 100 shown in FIG. 100 are for illustration only. Other embodiments of the network configuration 100 can be used without departing from the scope of the present disclosure.

[0038] According to an embodiment of the present disclosure, the electronic device 101 is included in the network configuration 100. The electronic device 101 can include at least one of a bus 110, a processor 120, a memory 130, an input / output (I / O) interface 150, a display 160, a communication interface 170, or a sensor 180. In an embodiment, the electronic device 101 can exclude at least one of these components or can add at least one other component. The bus 110 includes circuitry for connecting the components 120-180 to each other and for transmitting communications (such as control messages and / or data) between the components.

[0039] The processor 120 includes one or more processing devices, such as one or more microprocessors, microcontrollers, digital signal processors (DSPs), application specific integrated circuits (ASICs), or field programmable gate arrays (FPGAs). In an embodiment, the processor 120 includes one or more of a central processing unit (CPU), an application processor (AP), a communication processor (CP), or a graphics processing unit (GPU). The processor 120 can perform control on at least one of the other components of the electronic device 101 and / or perform operations or data processing related to communication or other functions. As described below, the processor 120 can be used to execute one or more functions related to interest level prediction based on behavioral transformation.

[0040] Memory 130 can include volatile and / or non-volatile memory. For example, memory 130 can store commands or data related to at least one other component of electronic device 101. According to an embodiment of the present disclosure, memory 130 can store software and / or program 140. Program 140 includes, for example, kernel 141, middleware 143, application programming interface (API) 145, and / or application program (or “app”) 147. At least a portion of kernel 141, middleware 143, or API 145 can be represented as an operating system (OS).

[0041] Kernel 141 can control or manage system resources (such as bus 110, processor 120, or memory 130) for performing operations or functions implemented in other programs (such as middleware 143, API 145, or app 147). Kernel 141 provides an interface that allows middleware 143, API 145, or app 147 to access various components of electronic device 101 to control or manage system resources. App 147 includes one or more apps for performing one or more functions related to interest level prediction based on behavior change. These functions can be performed by a single app or by multiple apps, each of the multiple apps performing one or more of these functions. For example, middleware 143 can act as a relay to allow API 145 or app 147 to communicate data with kernel 141. Multiple apps 147 can be provided. Middleware 143 can control work requests received from app 147, such as by assigning priorities for using system resources of electronic device 101 (such as bus 110, processor 120, or memory 130) to at least one of the multiple apps 147. API 145 is an interface that allows app 147 to control functions provided by kernel 141 or middleware 143. For example, API 145 includes at least one interface or function (such as a command) for archive control, window control, image processing, or text control.

[0042] I / O interface 150 serves as an interface that can, for example, transfer commands or data input from a user or other external device to other component(s) of electronic device 101. I / O interface 150 can also output commands or data received from other component(s) of electronic device 101 to the user or other external device.

[0043] The display 160 includes, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a quantum dot light emitting diode (QLED) display, a microelectromechanical systems (MEMS) display, or an electronic paper display. The display 160 can also be a depth sensing display, such as a multi-focus display. The display 160 can display various contents (such as text, images, videos, icons, or symbols) to a user. The display 160 can include a touch screen and can receive, for example, touch, gesture, proximity, or hover inputs using an electronic pen or a user's body part.

[0044] For example, the communication interface 170 can establish communication between the electronic device 101 and an external electronic device (such as the first electronic device 102, the second electronic device 104, or the server 106). For example, the communication interface 170 can be connected to the network 162 or 164 through wireless or wired communication to communicate with an external electronic device. The communication interface 170 can be a wired or wireless transceiver or any other component for transmitting and receiving signals (such as images).

[0045] The electronic device 101 also includes one or more sensors 180, which can measure a physical quantity or detect an activation state of the electronic device 101 and convert the measured or detected information into an electrical signal. For example, one or more of the sensors 180 can include one or more cameras or other imaging sensors, which can be used to capture an image of a scene. The sensor(s) 180 can also include one or more buttons for touch input, one or more microphones, a gesture sensor, a gyroscope or gyro sensor, a barometric pressure sensor, a magnetic sensor or magnetometer, an acceleration sensor or accelerometer, a grip sensor, a proximity sensor, a color sensor (such as an RGB sensor), a biophysical sensor, a temperature sensor, a humidity sensor, an illuminance sensor, an ultraviolet (UV) sensor, an electromyogram (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, an infrared (IR) sensor, an ultrasonic sensor, an iris sensor, or a fingerprint sensor. The sensor(s) 180 can also include an inertial measurement unit, which can include one or more accelerometers, gyroscopes, and other components. Additionally, the sensor(s) 180 can include a control circuit for controlling at least one of the sensors included herein. Any one of these sensors (s) 180 can be located within the electronic device 101.

[0046] The first external electronic device 102 or the second external electronic device 104 can be a wearable device or a wearable device with an electronic device installed therein (such as an HMD). When the electronic device 101 is installed in the electronic device 102 (such as an HMD), the electronic device 101 can communicate with the electronic device 102 through the communication interface 170. The electronic device 101 can be directly connected to the electronic device 102 to communicate with the electronic device 102 without involving a separate network. The electronic device 101 can also be an augmented reality wearable device including one or more cameras, such as glasses.

[0047] Wireless communication can use, for example, at least one of Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), 5th Generation (5G) wireless systems, millimeter wave or 60 GHz wireless communication, Wireless USB, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Universal Mobile Telecommunications System (UMTS), Wireless Broadband (WiBro), or Global System for Mobile Communications (GSM) as a cellular communication protocol. The wired connection can include, for example, at least one of Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), Recommended Standard 232 (RS-232), or Plain Old Telephone Service (POTS). The network 162 includes at least one communication network, such as a computer network (such as a Local Area Network (LAN) or a Wide Area Network (WAN)), the Internet, or a telephone network.

[0048] The first external electronic device 102, the second external electronic device 104, and the server 106 can all be devices of the same or different types as the electronic device 101. According to some embodiments of the present disclosure, the server 106 includes a group of one or more servers. In addition, according to some embodiments of the present disclosure, all or some of the operations performed on the electronic device 101 can be performed on another or more other electronic devices (such as the electronic devices 102 and 104 or the server 106). In addition, according to some embodiments of the present disclosure, when the electronic device 101 should automatically or upon request perform a certain function or service, the electronic device 101 can request another device (such as the electronic devices 102 and 104 or the server 106) to perform at least some functions associated therewith, rather than performing the function or service alone or additionally performing the function or service. Another electronic device (such as the electronic devices 102 and 104 or the server 106) can perform the requested function or additional functions and transmit the result of the execution to the electronic device 101. The electronic device 101 can provide the requested function or service by processing the received result as it is or additionally. For this purpose, for example, cloud computing, distributed computing, or client-server computing technologies can be used. Although Figure 1The illustrated electronic device 101 includes a communication interface 170 that communicates with an external electronic device 104 or a server 106 via a network 162. However, according to embodiments of the present disclosure, the electronic device 101 can operate independently without a separate communication function.

[0049] The server 106 can include components (or a suitable subset thereof) that are the same as or similar to those of the electronic device 101. The server 106 can support driving the electronic device 101 by performing at least one of the operations (or functions) implemented on the electronic device 101. For example, the server 106 can include a processing module or a processor that can support the processor 120 implemented in the electronic device 101. As described below, the server 106 can be used to perform one or more functions related to interest level prediction based on behavior change.

[0050] Although Figure 1 illustrates an example of a network configuration 100 including the electronic device 101, various changes can be made to Figure 1 it. For example, the network configuration 100 can include any number of each component in any suitable arrangement. Generally, computing and communication systems have a wide variety of configurations, and Figure 1 the scope of the present disclosure is not limited to any specific configuration. Additionally, although Figure 1 illustrates an operating environment in which various features disclosed in this patent document can be used, these features can be used in any other suitable system.

[0051] Figure 2 Illustrates an example system 200 for interest level prediction based on behavior change according to the present disclosure. For ease of explanation, Figure 2 the system 200 shown in Figure 1 is described as being implemented on or supported by one or more components (such as the server 106) in the Figure 2 network configuration 100. However,

[0052] As Figure 2 shown, the system 200 includes an offline analysis subsystem 201 and an online activity service subsystem 202. The terms "offline" and "online" are merely intended to refer to different levels of data interaction with one or more user devices (such as one or more electronic devices 101, 102, 104) on which a linear entertainment program is consumed. In one example, the electronic device 101 can be a television, and the electronic device 102 can be a smart phone, both of which can be used by the same user to consume a linear entertainment program.

[0053] The offline analysis subsystem 201 includes a database 203 that contains, for example, automatic content recognition (ACR) data reflecting the consumption of linear entertainment programs by one or more users. Information from the database 203 is provided to a feature engineering function 204 that determines user behaviors of interest related to the consumed linear entertainment programs, such as the time of day the program is watched, the title and / or genre of the program watched, the duration of the length of the program watched by the user, and the like. The output from the feature engineering function 204 is provided to a periodic data aggregation function 205 that aggregates user behaviors based on a period (such as daily). A daily behavior data aggregation pipeline can be formed by the feature engineering function 204 and the periodic data aggregation function 205. The operations of the feature engineering function 204 and the periodic data aggregation function 205 are described further below. Aggregate data regarding the consumption of linear entertainment programs by users can be forwarded from the periodic data aggregation function 205 to a data extraction, transformation, and loading (ETL) function 206 that operates to provide aggregate user behavior information populating a behavior feature database 207. The behavior feature database 207 supplies user behavior data related to linear entertainment programs to a training data generation function 208 as necessary to at least determine the Fibonacci confidence interval level of each user's interest in watching a program and the behavior transition features of each user related to watching a program. The behavior feature database 207 provides user behavior inputs including the Fibonacci confidence interval level and behavior transition features to both a model training function 209 and a training data registration function 210. The model training function 209 trains a machine learning model and provides model data to a behavior model registration function 211.

[0054] Within the online activity service subsystem 202, a target feature engineering function 212 receives an input from the database 203 and an advertising campaign request 213 and derives features on which the trained machine learning model will operate to generate inferences. An inference data generation function 214 generates an inference 215 based on inputs regarding those features from the target feature engineering function 212, the machine learning model defined by the behavior model registration function 211, and user behavior information from the training data registration function 210. The inference data generation function 214 utilizes the Fibonacci confidence interval level and user behavior transition features in the process of predicting user interest in an event program. The inference 215 representing the predicted user interest in the event program accumulates in an event result 216 and is used to deploy the requested advertising campaign.

[0055] Generally speaking, the feature engineering function 204, the periodic data aggregation function 205, the training data generation function 208, the model training function 209, the inference data generation function 214, and the inference(s) 215 implement the novel behavior-transition-based interest level prediction of the present disclosure. The operations of these components are discussed in further detail below.

[0056] In some cases, there are two tuning prediction problems on the feature engineering side that can be solved using the present disclosure. First, the system of the present disclosure addresses the cold start problem. The campaign target can represent a new program to be aired in the future, and the user behavior pattern changes naturally with the new program. For example, unlike a typical recommendation system for e-commerce that builds user behavior (purchases) on existing products, for ACR data of linear entertainment programs, ad campaigns are typically built for new programs (such as newly aired programs) for which there is no prior user behavior. Thus, campaigns related to newly aired programs create a cold start problem in the feature engineering step and make it difficult to convert behavior data into vectors using dynamic embedding algorithms such as an embedding layer. Additionally, as more and more new programs are aired, the range of programs in the database grows at a very high rate.

[0057] Second, the ACR data for tuning prediction has clear time patterns. For example, looking at the TV schedule of linear TV programs, there are clear patterns because some programs are only aired at specific times of the day. For example, sports programs are typically aired in the afternoon or evening. This pattern means that traditional event-based behavior sequence methods are not the best solution for tuning prediction because the differences in the event sequences of daytime and nighttime TV viewers do not necessarily reflect differences in user interests due to selection limitations.

[0058] To address these problems, in Figure 2Two functions are introduced as solutions in the [description], namely the feature engineering function 204 and the daily data aggregation function 205. Here, the user behavior represented by user-program interactions exported from the database 203 can be mapped to a range of metadata that is not changed by the feature engineering function 204. For example, the selected metadata extracted from the data received from the database 203 can include features such as program type, program season number, program episode number, program genre, etc. to represent an entertainment program, so as to represent the user's behavior using the programs the user has watched. During the process implemented by the feature engineering function 204, these metadata are converted into vectors using one or more trained algorithms. The pipeline between the database 203 and the behavioral feature database 207 can support various machine learning algorithms, such as one-hot encoding, natural language processing (such as the word2vec algorithm), etc. For categorical data such as program type, a trained one-hot encoder can be used. For semantic data such as program title, a trained word2vec algorithm can be selected. For some numerical data such as program season number, the data can be segmented into categorical data, and then one-hot encoding can be applied.

[0059] To streamline the time pattern of ACR data related to user-program interaction behavior so that the behaviors are more standardized and comparable to each other, the user behavior data can be aggregated based on a period (such as daily) through the periodic data aggregation function 205 and used to generate an average daily feature representation for each user. An example aggregation performed by the periodic data aggregation function 205 can be executed in the following manner.

[0060]

[0061] Here, f pn represents the nth program watched by the user during a day or other time period. For example, if the user only watched two programs at 7:00 pm and 8:00 pm on a specific date, the time one-hot features for that day can be aggregated as follows.

[0062] 7 PM feature: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0]

[0063] +

[0064] 8 PM feature: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0]

[0065] / 2

[0066] =

[0067] [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0,0,0]

[0068] This aggregation behavior is flexible to support other data aggregations in different time windows. For example, the weekly aggregation can be represented as follows.

[0069]

[0070] Here, N dn represents how many shows the user watched on the nth day, and f dn represents the nth show the user watched on that day. In this way, the daily aggregation behavior characteristics can support feature engineering in different post - processing time windows. The data ETL function 206 verifies the output of the sequence formed by the feature engineering function 204 and the periodic data aggregation function 205 for storage in the behavior feature database 207.

[0071] Combined with model training, the training data generation function 208 can be used to generate Fibonacci confidence levels regarding viewer interests. For tuning predictions, both (i) predicting whether a user has any interest in an active program and (ii) predicting the user interest level in the target program of the activity may be required or desired. Accurately predicting the user interest level in an active program allows an activity to be constructed in a more cost - effective way to reach the desired users. Linear entertainment program recommendations are unlike other recommendation systems because entertainment program consumers may have some specific behaviors that can affect the outcome of an activity. As mentioned above, for example, some shows have loyal fans who will tune in to a new season or episode regardless of whether an advertisement is running on their device. From the perspective of results, these users have already been converted, and spending the activity budget on those users is not efficient. Therefore, for a tuning system in linear entertainment, the conversion rate is not the only evaluation parameter important to advertisers. In some cases, analyzing the lift evaluation of conversion rate improvement based on A / B testing between different activity segments may be more important. In some cases, the lift evaluation can be represented as follows.

[0072] Lift =

[0073] Here, C A represents the conversion rate of the user test group participating in the activity (receiving the advertisement), C BRepresents the conversion rate of the control group that has not been exposed to the advertising campaign. For tuning predictions, advertisers can request or require the system to provide, for example, four different user segments differentiated by interest level, such that the advertising campaign can be designed in a way that has optimal lift performance. In some cases, the four different user segments can be referred to as "enthusiasts", "fans", "viewers", and "not interested". However, typically in the campaign request, there is no definition of how to determine the interest level of a user, which poses a challenge to the system because no labels are provided to train the machine learning model.

[0074] To help solve this problem, a ratings-based Fibonacci confidence interval interest separation can be used for the training data generation function 208. That is, to generate the interest level for machine learning model training and prediction purposes, a ratings-based Fibonacci confidence interval interest separation algorithm can be employed. This algorithm classifies user behavior based on the user's ratings (such as the ratio of user viewing time to program time) to create corresponding Fibonacci interest levels (such as one or more of the interest levels requested by the advertiser or those described above). In some cases, four Fibonacci confidence intervals can be used in the system for classification to meet the advertiser's need for four interest levels, although the algorithm here can be easily scaled up or down. In an embodiment, the calculation of the Fibonacci confidence interval can be based on the following Fibonacci sequence F(0) to F(n).

[0075] F(0) = 0, F(1) = 1, and for all n ≥ 2, F(n) = F(n - 1) + F(n - 2)

[0076] The first Fibonacci confidence interval can be represented using the following formula:

[0077]

[0078] where the first Fibonacci confidence interval is defined as [1, B1). For a specific program, the ratio of the user's program viewing time to the program length ((viewing time) / (program length)) can fall within this interval, and for this user, this interval is labeled as "program level 1". This user is also referred to as an "enthusiast" of the program.

[0079] The fourth Fibonacci confidence interval can be represented using B4 = 0. For "program level 4", the fourth Fibonacci confidence interval is defined as [B3, B4). This user is also referred to as "not interested" in the program or a passive user.

[0080] To support the use of the behavior transformation feature space, the training data generation function 208 and the inference data generation function 214 can be used. It is popular for sequential model structure recommendation systems to use behavior sequences (such as a series of items with which a user interacts) to track changes in user behavior. However, the dynamic embeddings (embedding layers) commonly used in these models limit these methods, which can only construct sequences in such a superficial way because (i) the embeddings do not have specific physical meanings and (ii) the embeddings are always changing. In contrast, since the embeddings in the system of the present disclosure are provided from the behavior feature database 207, where there is a static range for each bit, a behavior transformation feature space can be designed for user behavior representation. In this space, the user's traditional behavior sequence is constructed, and the first derivative (such as the speed of change) and the second derivative (such as the acceleration of change) of the behavior sequence are also introduced to better present the behavior transformation attributes. In some cases, this can be represented as follows.

[0081]

[0082]

[0083] Here, n represents the position count in the behavior sequence, and t n represents the time difference on the aggregated time scale (such as daily, weekly, monthly, etc.). The behavior transformation features of the user are determined by the training data generation function 208 for the training corpus and provided to the model training function 209 together with the Fibonacci confidence interval of the user interest level. The model training function 209 trains a machine learning model for the behavior model registration function 211 based on this.

[0084] During use, the target feature engineering function 212 identifies the user behavior of interest based on the activity request 213. The inference data generation function 214 adopts those behaviors of interest (received from the target feature engineering function 212) and derives inference data on the predicted interest level of a specific user in the target program of the advertising campaign based on the machine learning model in the behavior model registration function 211 and the training data from the training data registration function 210, thereby considering the behavior transformation features of each user. The resulting inference 215 related to each user can be used to selectively deploy advertising campaigns effectively and cost-effectively.

[0085] Although Figure 2 shows an example of the system 200 for predicting the interest level based on behavior transformation, various changes can be made to Figure 2 it. For example, various components or functions in Figure 2 can be combined, further subdivided, replicated, rearranged, or omitted according to specific needs. In addition, if necessary or desired, one or more additional components or functions can be included.

[0086] Figure 3A and Figure 3B shows an alternative example of a behavior-to-interest level semantic transfer model 300 for behavior-based interest level prediction according to the present disclosure. For ease of explanation, Figure 3A the model 300 shown in is described as being implemented to be used with Figure 2 the system 200. However, Figure 3A the model 300 shown in is capable of being used with any other suitable system(s).

[0087] As Figure 3A shown, the model 300 includes a Fibonacci interest level prediction layer 301 that classifies consumers of linear entertainment programs using the four interest levels (enthusiast, fan, viewer, and uninterested) described above. A Fibonacci confidence (FC) layer 302 indicates the interest level of a particular consumer based on the output from an attention layer 303. The attention layer 303 receives input from a Fibonacci confidence interval semantic space 304, and the Fibonacci confidence interval semantic space 304 receives information related to the Fibonacci confidence interval from a convolutional function 305. For each feature, the information received by the Fibonacci confidence interval semantic space 304 may include information about user behavior features (left-slanted cross-hatching), the first derivative of user behavior feature information (right-slanted cross-hatching), and the second derivative of user behavior information (horizontal and vertical cross-hatching). The information related to the Fibonacci confidence interval received by the Fibonacci confidence interval semantic space 304 from the convolutional function 305 is flattened by a function 306, and the flattened data represents the input to the attention layer 303. In this example, the convolutional function 305 has a size of 3x3xN, where N is the number of Fibonacci confidence interval levels (such as four in the example described above).

[0088] The convolutional function 305 receives input from a behavior transition semantic space 307. The behavior transition semantic space 307 receives information about user behavior transitions from a stacking layer 308, where the information includes information about user behavior features, the first derivative of user behavior feature information, and the second derivative of user behavior feature information. The stacking layer 308 receives input from transformer layers 309a - 309c, and each of the transformer layers 309a - 309c operates on inputs respectively related to information about user behavior features, the first derivative of user behavior feature information, and the second derivative of user behavior feature information. The transformer layers 309a - 309c operate on data sourced from a behavior transition feature space 310. A fourth transformer layer 309d operates on user profile features (left-slanted and right-slanted cross-hatching) sourced from a user profile feature space 311. The data on which the transformer layers 309a - 309d operate is affected by a positional embedding 312. InFigure 3A Among them, the active target feature is indicated by a thick outline and includes one or more features from the user profile feature space 311 and information about the user behavior feature(s), the first derivative of the user behavior feature(s) information, and the second derivative of the user behavior feature information from the behavior transition feature space 310.

[0089] Figure 3B The structure of the transformer layers 309a - 309d is shown in more detail. Each transformer layer 309x includes multi - head attention 351, the output of which is received by an add and normalize layer 352 (which also receives the input to the multi - head attention 351). The output of layer 352 is received by an FC layer 353. Both the output and the input of the FC layer 353 are received by an add and normalize layer 354, and the output of the add and normalize layer 354 is the output of the transformer layer 309x.

[0090] In some cases, the feature data used in this disclosure includes the following information.

[0091] [Table 1]

[0092] User behavior characteristics / Activity target characteristics User profile characteristics 1. Season number 1. Time of day 2. Episode number 2. Program viewing length 3. Production year 3. Program viewing percentage 4. Program length 4. Geography 5. Program title 5. Day of the week 6. Program genre 7. Program type

[0093] In the behavioral transformation feature space, all three feature sequences (user behavior sequence, first derivative of user behavior sequence, and second derivative of user behavior sequence) can have the same length. In an example model, eight features are selected, and all features can be from the behavioral feature database 207. In some cases, post-processing can be applied as follows. The first seven features in each sequence can use only positive programs as defined by the advertiser (program viewership time for the user is greater than 6 minutes). In the training phase, the eighth feature will be the one where the program falls within a specific Fibonacci confidence interval for that user. The training label for each input data can be the same as the Fibonacci confidence level of the eighth feature. In the inference phase, the first seven features can remain unchanged, and the eighth feature can be replaced by an active target feature generated from the target feature engineering function 212. The eight features in each feature sequence can all be aggregated features and support flexible aggregation (such as daily, weekly, monthly, etc.). Regarding the user profile feature sequence, the user profile feature sequence can represent seven feature sequences corresponding to the first seven features in the behavioral transformation feature space. In some cases, the tasks to be done by the machine learning model can include the following. The machine learning model can obtain features from the behavioral transformation feature space and add positional embeddings for each behavior sequence. One or more transformer layers can help learn deeper representations of each behavior sequence. Multiple transformer outputs (such as three outputs) from the behavioral transformation feature space can be stacked to create a learned two-dimensional behavioral transformation semantic space. A convolutional layer, such as a convolutional layer with a kernel size of 3x3x4, can be used to transform the two-dimensional behavioral transformation semantic space into a three-dimensional Fibonacci confidence interval level semantic space. The three-dimensional Fibonacci confidence interval level semantic features are flattened and passed through an attention layer with deeper user profile features learned from the user profile feature sequence data by the transformer. Two or more fully connected layers can map the output of the attention layer to the final classification layer, which predicts the Fibonacci interest level as defined above.

[0094] In some cases, the tuning prediction process can be designed in a way that utilizes more offline work to boost its speed and throughput. For example, the machine learning model can represent a deep learning model or other very general models, which means the model does not require training activity information. This general model enables moving all heavy-lifting components in the system (such as most feature engineering and model training) to offline processing and allows for automatic scheduling. This saves a significant amount of time and helps the advertising team obtain the necessary segments as soon as possible. With the described system design, the advertising team can support more activities per day. The online part of the tuning prediction process may only need to run lightweight feature engineering for the proxy shows for the activity requests from advertisers. To obtain the final segments, the machine learning model can simply replace the Fibonacci confidence level behavior features in the behavior transition feature space with the corresponding proxy-shown features to perform inference. Inference can be executed on different cluster nodes, leveraging parallel computing to achieve faster speeds.

[0095] Although Figure 3A and Figure 3B shows an alternative example of the behavior-to-interest level semantic transfer model 300 for interest level prediction based on behavior transitions, various changes can be made to Figure 3A and Figure 3B . For example, the various components or functions in Figure 3A and Figure 3B can be combined, further subdivided, replicated, rearranged, or omitted according to specific needs. Additionally, one or more additional components or functions can be included if needed or desired.

[0096] Figure 4 shows another example system 400 for interest level prediction based on behavior transitions according to the present disclosure, and Figure 5 shows an example model 500 for interest level prediction based on behavior transitions for use with the system 400 of Figure 4 . In some specific activity requests, the advertiser can provide information about the user's interest level in the activity target based on previous user behavior. This occurs more frequently when the advertiser runs a "return show" activity. A return show activity typically means the activity target is a new season of an existing show. For example, the activity target can be a new season of the series show "Young Sheldon" which has been broadcast for several seasons previously. In this case, the alternative system 400 as shown in Figure 4 and Figure 5 can be utilized.

[0097] Similar to Figure 2In system 200, system 400 includes an offline analysis subsystem (including behavior feature engineering function 401) and an online analysis subsystem (including activity service function 402). The behavior feature engineering function 401 includes a daily engineering function 404 (similar to feature engineering function 204), which receives information from database 403 and provides output to an aggregation function 405 (similar to periodic data aggregation function 205). The output of the aggregation function 405 is stored in the feature database 407. In Figure 4 the embodiment of, still as Figure 2 in the embodiment of, offline behavior feature engineering is provided. However, in this embodiment, model training is moved online because the label (interest level) is activity-specific.

[0098] The activity service function 402 receives an activity request 413 at each of a target feature engineering function 412 and a label generation function 420, where the label generation function 420 also receives the output of the target feature engineering function 412. The outputs from the target feature engineering function 412 and the label generation function 420 are also provided, together with information from the feature database 407, to a model training function 409. The model training function 409 generates an inference 415, which becomes part of the activity result 416.

[0099] To improve the speed of the online analysis subsystem, a light-attention-based model 500 is adopted. In model 500, the time window for behavioral sequence data is six weeks. The first four weeks are aggregated together and used as long-term behavior 501, while the last two weeks are used as short-term behavior 502. Past viewing behavior 503 is also utilized. In the example shown, the feature sets 504 - 506 from each of the long-term behavior 501, short-term behavior 502, and past viewing behavior 503 are respectively reduced to smaller sets 507 - 509 and provided to attention layers 510 - 512. The outputs of the attention layers 510 - 512 are combined with the feature sets 507 - 509 to generate feature sets 513 - 515. Those feature sets 513 - 515 are provided to an FC layer 516, and the FC layer 516 outputs a label 517.

[0100] Although Figure 4 shows another example of system 400 for interest level prediction based on behavior transitions, and Figure 5 shows an example of model 500 for interest level prediction based on behavior transitions used with Figure 4 the system 400 of, various changes can be made to Figure 4 and Figure 5 . For example, can be combined, further subdivided, replicated, rearranged, or omitted according to specific needs Figure 4 and Figure 5various components or functions therein. Additionally, one or more additional components or functions may be included if desired or needed.

[0101] Figure 6 illustrates yet another example system 600 for interest level prediction based on behavior transitions according to the present disclosure, and Figure 7 illustrates an example model 700 for interest level prediction based on behavior transitions for use with Figure 6 system 600. Although system 600 is similar to system 200, it provides a faster pipeline. In the faster pipeline, a user selection model 700 is added to reduce the data size for inference. In the online activity service subsystem 602, the inference data generation function 214 is replaced by an inference feature aggregation function 614 and a user selection function 620, which together produce an inference 615. The inference feature aggregation function 614 aggregates restricted user behavior characteristics for tuning prediction of an activity program. The user selection function 620 uses the restricted user behavior characteristics and Fibonacci confidence interval levels to generate a binary (such as "0" or "1") score to predict the tuning probability of the user for the activity program.

[0102] The user selection model 700 uses a smaller Transformer-based model as the ranking model structure, as Figure 7 shown. In model 700, the user behavior feature space 710 does not include first and second derivatives as in model 300. Although specific user behavior characteristics (such as Figure 7 shown by the thick contour lines in

[0103] Although Figure 6 illustrates yet another example of system 600 for interest level prediction based on behavior transitions, and Figure 7An example of a model 700 for predicting interest levels based on behavior change for use with system 600 is shown, but various changes may be made to Figure 6 and Figure 7 . For example, various components or functions in Figure 6 and Figure 7 may be combined, further subdivided, replicated, rearranged, or omitted according to specific needs. Additionally, one or more additional components or functions may be included if desired or needed.

[0104] It should be noted that the functions shown in Figures 2 to 7 or described with respect to Figures 2 to 7 can be implemented in any suitable manner in the electronic device 101, the server 106, or other device(s). For example, in an embodiment, one or more software applications or other software instructions executed by the processor 120 of the electronic device 101, the server 106, or other device(s) can be used to implement or support at least some of the functions shown in Figures 2 to 7 or described with respect to Figures 2 to 7 . In other embodiments, dedicated hardware components can be used to implement or support at least some of the functions shown in Figures 2 to 7 or described with respect to Figures 2 to 7 . Generally, any suitable hardware or any suitable combination of hardware and software / firmware instructions can be used to perform the functions shown in Figures 2 to 7 or described with respect to Figures 2 to 7 .

[0105] Figure 8 An example method 800 for training a machine learning model according to the present disclosure is shown. For ease of explanation, method 800 is described with reference to system 200 and model 300. However, method 800 can be used with any suitable system and model and can be easily modified to accommodate changes in the underlying system and / or model.

[0106] As Figure 8 shown, an offline behavior data feature pipeline (such as the feature engineering function 204, the periodic data aggregation function 205, and the data ETL function 206 from Figure 2 ) is provided, which aggregates user behavior data (such as daily) and persists the features of the user behavior data into a behavior feature database (such as the behavior feature database 207 in Figure 2 ) (step 802). The offline behavior data feature pipeline can map linear entertainment programs into a predefined feature space and can transform the raw behavior data into vectorized features using a trained algorithm and leveraging periodic sessionization. A Fibonacci confidence interval level for the aggregated user program rating data is provided (such as through Figure 2The training data generation function 208) to distinguish the user interest levels for different linear entertainment programs (step 804). The Fibonacci confidence interval levels can be improved based on the conversion rates in the user program rating data of the linear entertainment programs. A behavior transition feature space is created based on the behavior sequence, the first derivative of the behavior sequence, and the second derivative of the behavior sequence (such as also through the training data generation function 208) to obtain information about the user behavior over time (step 806). A machine learning model (such as Figure 2 the machine learning model of the behavior model registration function 211 in

[0107] Although Figure 8 illustrates an example of method 800 for training a machine learning model, various changes can be made to Figure 8 For example, although shown as a series of steps, Figure 8 the various steps in

[0108] Figure 9 illustrates an example method 900 for using a trained machine learning model. For ease of explanation, method 900 is described with reference to system 200 and model 300. However, method 900 can be used with any suitable system and model and can be easily modified to accommodate changes in the underlying system and / or model.

[0109] As Figure 9 shown, an advertising campaign request for a target linear entertainment program, season, or episode (such as Figure 2 the campaign request 213 in Figure 2 is received (step 902). Inference data (such as through Figure 2 the inference data generation function 214 in

[0110] AlthoughFigure 9 An example of method 900 for using a trained machine learning model is shown, but various changes may be made Figure 9 to it. For example, although shown as a series of steps, Figure 9 the various steps in it may overlap, occur in parallel, occur in a different order, or occur any number of times.

[0111] Figure 10 An example method 1000 for training a machine learning model and using the trained machine learning model in accordance with the present disclosure is shown. For ease of explanation, method 1000 is described with reference to system 200 and model 300. However, method 1000 may be used with any suitable system and model and may be readily modified to accommodate changes in the underlying system and / or model.

[0112] As Figure 10 shown, a Fibonacci confidence interval level for user program rating data (such as through the training data generation function 208 in Figure 2 ) is provided to distinguish the levels of user interest in different linear entertainment programs (step 1002). A behavioral transition feature space is created based on the behavioral sequence, the first derivative of the behavioral sequence, and the second derivative of the behavioral sequence (such as also through the training data generation function 208) to obtain information about user behavior over time (step 1004). Based on the trained machine learning model, the user data from the behavioral transition feature space is mapped to a prediction of the Fibonacci confidence interval level using a transformer structure and attention (step 1006).

[0113] Although Figure 10 an example of method 1000 for training a machine learning model and using the trained machine learning model is shown, but various changes may be made Figure 10 to it. For example, although shown as a series of steps, Figure 10 the various steps in it may overlap, occur in parallel, occur in a different order, or occur any number of times.

[0114] Generally speaking, the systems and methods of the present disclosure can be used to generate audience segments in production-level activities. In embodiments, this can be accomplished in the following areas: tuning prediction, brand interest level analysis, social interest shift analysis, user clustering, and cross-device behavior analysis. Tuning prediction generally involves helping television or other advertisers find the level of user interest in a program to be promoted, and brand interest level analysis generally involves helping brand owners identify the level of user interest in branded products so that advertisements can be run on those users' devices to promote the products. Social interest shift analysis generally involves analyzing the behavioral data of a large number of users (such as television or smartphone users) to identify trend shifts in current society, which can provide very helpful data for business analysis, such as by helping fashion enterprises decide on new trends. User clustering generally involves clustering similar users together. In some cases, for example, advertisers have seed users who are considered to be very interested in a particular product or content. By providing user clusters for comparison with the seed users, the seed segment can be optimized or expanded to increase the user scope. Cross-device behavior analysis generally involves analyzing user cross-device behavior. For example, the analysis of user behavior involving television sets, smart phones, computers, and Internet of Things (IoT) devices can jointly provide flexibility to accommodate the features described above and create more accurate analysis by obtaining and processing more data.

[0115] Although the present disclosure has been described with reference to various exemplary embodiments, various changes and modifications can be suggested to those skilled in the art. The present disclosure is intended to cover such changes and modifications that fall within the scope of the appended claims.

Claims

1. A method 1000, comprising: Provide Fibonacci confidence interval levels for user program rating data to distinguish user interest levels for different linear entertainment programs (1002); Create a behavior transition feature space based on a behavior sequence, a first derivative of the behavior sequence, and a second derivative of the behavior sequence to obtain information about user behavior over time (1006); and Based on a trained machine learning model, use a transformer structure and attention to map user data from the behavior transition feature space to a prediction of Fibonacci confidence interval levels (1008).

2. The method according to claim 1, further comprising: Provide an offline behavior data feature pipeline that aggregates user behavior data and stores the features of the user behavior data in a behavior feature database.

3. The method according to claim 2, wherein The offline behavior data feature pipeline uses one or more trained algorithms and utilizes periodic sessionization to transform raw behavior data into vectorized features.

4. The method according to claim 3, wherein The offline behavior data feature pipeline maps linear entertainment programs into a predefined feature space.

5. The method according to any one of claims 1 to 4, wherein The Fibonacci confidence interval levels are improved based on the conversion rate of user program rating data for at least one of the linear entertainment programs.

6. The method according to any one of claims 1 to 5, wherein The machine learning model is trained offline.

7. The method according to any one of claims 1 to 6, wherein The machine learning model is a user selection model.

8. An electronic device 101 for predicting an interest level based on behavior change, comprising: At least one processor 120, configured to:[[]] Provide Fibonacci confidence interval levels for user program rating data to distinguish user interest levels for different linear entertainment programs; Create a behavior transition feature space based on a behavior sequence, a first derivative of the behavior sequence, and a second derivative of the behavior sequence to obtain information about user behavior over time; And Based on a trained machine learning model, use a transformer structure and attention to map user data from the behavior transition feature space to a prediction of Fibonacci confidence interval levels.

9. The electronic device 101 according to claim 8, wherein The processor is further configured to provide an offline behavior data feature pipeline that aggregates user behavior data and stores the features of the user behavior data in a behavior feature database.

10. The electronic device 101 according to claim 9, wherein The offline behavior data feature pipeline is configured to use one or more trained algorithms and utilize periodic sessionization to transform raw behavior data into vectorized features.

11. The electronic device 101 according to claim 10, wherein The offline behavior data feature pipeline is configured to map linear entertainment programs into a predefined feature space.

12. The electronic device 101 according to any one of claims 8 to 11, wherein The Fibonacci confidence interval levels are improved based on the conversion rate of user program rating data for at least one of the linear entertainment programs.

13. The electronic device 101 according to any one of claims 8 to 12, wherein The machine learning model is trained offline.

14. The electronic device 101 according to any one of claims 8 to 13, wherein The machine learning model is a user selection model.

15. A computer-readable medium containing instructions, wherein The instructions, when executed, cause at least one processor of the electronic device 101 to implement the method according to any one of claims 1 to 7.

Images