Methods, systems, and media of determining a watch time loss region

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining watch time loss regions in media content items. In one aspect, features of a video are input into a trained model trained to output watch time loss regions. The trained model is trained using labels corresponding to known watch time loss regions in training videos and features of the training videos corresponding to the known watch time loss regions. A watch time loss region defines a window of time of a video during which a user is more likely than a threshold likelihood to stop playback of the video. In response to inputting features of a first video into the trained model, data regarding watch time loss regions of the video is obtained from the model and provided to an entity participating in providing the video to users.

Description

Technical Field

[0001] This specification generally covers data processing and identifying areas of view time loss in media content projects. Background Technology

[0002] The video streamed to a user may include one or more digital components that typically overlay the original video stream. The overlay content may be provided to the user within a rectangular area that covers a portion of the original video screen. Digital components may also include in-stream content that plays before, during, or after the original video stream.

[0003] As used throughout this document, the phrase "digital component" refers to a discrete unit of digital content or digital information (e.g., a video clip, audio clip, multimedia clip, image, text, or other unit of content). A digital component may be stored electronically as a single file or a collection of files on a physical storage device, and it may take the form of a video file, audio file, multimedia file, image file, or text file. For example, a digital component may be content designed to complement the content of a video or other resource. More specifically, a digital component may include digital content related to the content of a resource (e.g., a digital component may relate to a topic that is the same as or otherwise related to the subject / content on a video). Therefore, the provision of digital components can complement and often enhance the content of a web page or application. Summary of the Invention

[0004] In general, an innovative aspect of the subject matter described in this specification can be embodied in a method comprising the following operations: receiving a first video; determining a set of features of the first video; inputting the set of features of the first video into a trained model, wherein (1) the trained model is trained to output a viewing time loss region in a specific video, (2) the trained model is trained using labels corresponding to known viewing time loss regions in a plurality of training videos and features of the plurality of training videos corresponding to known viewing time loss regions, and (3) the viewing time loss region of the specific video defines a time window of the specific video during which the probability of a user stopping playback of the specific video is greater than a threshold probability; in response to inputting the set of features of the first video into the trained model, obtaining data about the viewing time loss region of the first video from the trained model; and providing the data about the viewing time loss region of the first video to an entity involved in providing the first video to the user. Other embodiments of this aspect include corresponding methods, apparatus, and computer programs configured to perform the actions of the methods and encoded on a computer storage device. These and other embodiments may each optionally include one or more of the following features.

[0005] Specific embodiments of the subject matter described in this specification can be implemented to achieve one or more of the following advantages. The techniques described in this specification enable the identification of viewing time loss information in a video, which can be used to determine the placement of digital components within a video stream, making the digital components more likely to be viewed / played by the user during video playback. Without considering the viewing time loss region, digital components may be placed in video portions that appear after the viewing time loss region, increasing the likelihood that the digital components will not be played. This, in turn, leads to wasted computational resources in providing and receiving digital components that are ultimately not consumed by the user. Conversely, by considering the viewing time loss region of the video when placing digital components to be displayed / played during video playback, the likelihood that the digital components will be played during video playback increases, thus preventing the waste of computational resources required to provide and receive the digital components. More specifically, the media content system achieves resource savings by selecting a digital component playback point before the determined viewing time loss region. By selecting a digital component playback placement point before the viewing time loss region, the likelihood of providing unconsumed digital components is reduced (or, conversely, the likelihood of consuming digital components is increased).

[0006] Viewing time loss zone information can be used for other purposes. For example, viewing time loss information can be provided to media content project owners. Media content project owners may be interested in knowing during which parts(s) of a content project users are more likely to abandon playback. By presenting viewing time loss information to content project owners, they can identify the viewing time loss zones. Therefore, content project owners can decide to rework or adjust media content projects to reduce or change the viewing time loss zones. Over time, adjusted media content projects with fewer or shorter viewing time loss zones will be consumed by users at a greater rate compared to those with more or longer viewing time loss zones. Therefore, providing media content projects with fewer and / or shorter viewing time loss zones is more resource-efficient because resource waste (e.g., providing unused resources) will decrease over time.

[0007] Furthermore, when digital components are placed and served within a video, considering the viewing time loss area of ​​the video can lead to increased viewership for both the video and its associated digital components (compared to viewership derived from placing and serving digital components without considering the viewing time loss area). Since the digital components are actually consumed / viewed by users, this can in turn lead to more revenue from the digital components (as opposed to users ignoring or missing them because they were placed in the video without considering the viewing time loss area).

[0008] Viewing time loss zone information can also be used to optimize the service or other processing of media content and the digital components to be presented with it. For example, the first set of digital components scheduled to play with the video before a defined viewing time loss zone can be sent to the client device by the content provider based on priority (e.g., before the first set of digital components is scheduled to play, or even with the video). Other sets of digital components scheduled to play after the first defined viewing time loss zone can be provided by the content provider on a pay-as-you-go basis (e.g., exactly when the digital components are scheduled to play). For example, since some users tend to stop watching the video during each viewing time loss zone, the content provider can provide some high-priority digital components before the first viewing time loss zone and lower-priority digital components before each subsequent viewing time loss zone. It will be appreciated that this leads to more efficient use of resources, as the content provider can only provide digital components scheduled to play after the viewing time loss zone if the video is played back, or if the user does not stop playing back before needing a specific digital component.

[0009] The machine learning methods described in this specification offer technical advantages that would otherwise be impossible. Media content systems can provide, for example, billions of content items. Manually determining viewing time loss zones and the placement of digital components for billions of content items is impractical. The machine learning system described in this specification can be used to analyze media content items and automatically determine viewing time loss zones within them, rather than using manual methods. Furthermore, the machine learning system can automatically predict viewing time loss zones for media content items with little or no user interaction data. Without using a machine learning system, significant resources must be allocated to collect vast amounts of user interaction data in order to directly calculate viewing time loss information. Using a machine learning system makes it possible to predict viewing time loss information, even for newly created videos, without waiting for or expending resources on user interaction data collection and real-world data processing.

[0010] Details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of this subject matter will become apparent from the description, drawings, and claims. Attached Figure Description

[0011] Figure 1 It is a block diagram of a sample environment for distributing and providing digital content for display on client devices.

[0012] Figure 2 An example system is shown for determining areas of lost viewing time in media content projects.

[0013] Figure 3 Example viewing time loss graph for video.

[0014] Figure 4 Example pseudocode for determining viewing time loss areas is shown.

[0015] Figure 5 An example architecture for a viewing time loss model is shown.

[0016] Figure 6 This is a flowchart of an example process for determining areas of viewtime loss in media content projects.

[0017] Figure 7 This is a block diagram of an example computer system that can be used to perform the operations described. Detailed Implementation

[0018] This specification generally relates to determining the viewing time loss region within a media content item. A media content system may provide media content (e.g., video content, audio content) to a viewer. Generally, a user may not consume a media content item from beginning to end. For various reasons, a user may leave or abandon playback of a media content item at a point in time before it is completed. The timeframe during which a user may abandon playback of a media content item can be referred to as the viewing time loss region. As another definition, the viewing time loss region for a specific video can describe the time window within which the probability of a user stopping playback of that specific video is greater than a threshold probability.

[0019] As summarized below and described in this specification, the techniques described in this application enable the automatic determination of predicted viewing time loss regions for input videos using a trained viewing time loss model. A model trainer can train a viewing time loss model using a set of training videos. For each training video, a viewing time loss curve generator can generate a viewing time loss curve for the training video based on user interaction data (e.g., video session start / stop times). Viewing time loss regions can be determined by a viewing time loss region identifier within the viewing time loss curve generated for the training video. The viewing time loss region identifier can determine the average slope of the viewing time loss curve and identify viewing time loss regions by identifying regions of the viewing time loss curve with slopes greater than the average slope. The viewing time loss model can be trained based on the viewing time loss regions determined for the training videos. Specifically, the viewing time loss model can be trained based on the viewing time loss regions determined for the training videos and features of the training videos appearing within a predetermined time window of the viewing time loss regions (e.g., determined by a feature extractor).

[0020] The trained watchtime loss model can be used to predict watchtime loss regions for an input video. For example, the input video could be a video with little or no user interaction data. A feature extractor identifies a set of features for the input video and feeds these features into the trained watchtime loss model. The trained watchtime loss model can output data on the predicted watchtime loss regions for the input features of the input video. This data may include the start and stop times of the predicted watchtime loss regions, the count of watchtime loss regions in the video, the total duration of the watchtime loss regions in the video, or a percentage of the video length corresponding to the identified watchtime loss regions.

[0021] Data on the predicted viewing time loss region of an input video can be used for various purposes. For example, this data can be provided to a content provider, who uses it to determine the timing of digital component playback within the input video. For instance, the content provider can select a digital component playback point in the input video prior to the determined viewing time loss region, as described above, which can achieve various types of resource savings. As another example, the data on the predicted viewing time loss region can be provided to the owner of the input video. The owner can choose to modify the input video to reduce the count or duration of the viewing time loss region. Therefore, an updated input video may be more likely to be consumed at completion or a later point in time, which can lead to resource efficiency regarding the resources spent on delivering content that is actually consumed. See below for reference. Figures 1-7 These features and additional features and benefits will be described in more detail.

[0022] In addition to the descriptions throughout this document, users may be provided with controls that allow them to select whether and when the systems, programs, or features described herein may collect user information (e.g., information about the user's social networks, social actions or activities, occupation, user preferences, or the user's current location) and whether content or communications may be sent to the user from the server. Furthermore, some data may be processed in one or more ways before being stored or used, thereby removing personally identifiable information. For example, a user's identity may be processed to make it impossible to determine personally identifiable information for the user, or, if location information is available, the user's geographic location may be generalized (e.g., to a city, zip code, or state level) to make it impossible to determine the user's specific location. Therefore, users have control over what information about themselves is collected, how that information is used, and what information is provided to them.

[0023] Figure 1This is a block diagram of an example environment 100 for distributing and providing digital content for display on client devices. Example environment 100 includes a network 104. Network 104 may include a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof. Network 104 may also include any type of wired and / or wireless network, satellite network, cable network, Wi-Fi network, mobile communication network (e.g., 3G, 4G, etc.), or any combination thereof. Network 104 may utilize communication protocols, including packet-based and / or datagram-based protocols such as Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), or other types of protocols. Network 104 may also include multiple devices that facilitate network communication and / or form the network hardware infrastructure, such as switches, routers, gateways, access points, firewalls, base stations, repeaters, or combinations thereof.

[0024] Network 104 connects client device 102, content platform 106, content provider 108, and video processing system 110. Example environment 100 may include many different content platforms 106, video processing systems 110, client devices 102, and content providers 108.

[0025] Content platform 106 is a computing platform capable of distributing content (such as, for example, a web server or reference). Figure 7 (Another data processing device described). Example content platform 106 includes search engines, social media platforms, video sharing platforms, news platforms, data aggregator platforms, or other content sharing platforms. Each content platform 106 can be operated by a content platform service provider.

[0026] Content platform 106 can publish and make its own content available. For example, content platform 106 could be a news platform that publishes its own news articles. Content platform 106 can also display content provided by one or more content providers 108 that are not part of content platform 106. In the example above, the news platform could also display third-party content provided by one or more content providers 108. As another example, content platform 106 could be a data aggregator platform that does not publish its own content but aggregates and displays third-party content provided by different content providers 108.

[0027] In some implementations, content platform 106 may store information about client devices (e.g., device preference information, content consumption information, etc.). This user information can be used by the content platform, for example, to customize content provided to client device 102 or to make it easy to access specific content frequently accessed by client device 102. In some implementations, content platform 106 may not store such device information on the platform itself; however, content platform 106 may still provide such information for storage on a specific server (separate from the content platform). Therefore, content platform 106 (also referred to herein as content platform / server 106, or simply server) refers to either the content platform storing such device information or the server storing such device information (separate from the content platform).

[0028] In some implementations, content platform 106 is a video service through which users can watch streamed video content. The video streamed to the user may include one or more digital components (e.g., provided by content provider 108) overlaid on the initial video stream. For example, for a given video streaming bandwidth, it is generally desirable to provide overlay content on the underlying video stream to provide viewers of the video stream with (multiple) digital components and increase the amount of content delivered within the viewing area. In addition to or instead of video streaming scenarios, content platform 106 may include a video processor that processes video files to modify them to include overlay content, the processed video file with overlay content being provided to client device 102 for display on client device 102.

[0029] Video processing system 110 can determine the predicted viewing time loss region for a given video and provide viewing time loss region information to different entities or systems (as outlined above and in reference). Figures 2-6 (Further description). For example, video processing system 110 can provide content provider 108 with information / data about viewing time loss areas, so that the content provider can schedule digital components before the predicted viewing time loss areas. As another example, viewing time loss information can be provided to the video owner's client device 102 as feedback on the video. For example, the owner can adjust the video based on the viewing time loss information.

[0030] In some implementations, the video processing system 110 uses the trained model to generate viewing time loss information (as outlined here and in references). Figure 2 , Figure 5 and Figure 6(Further description). That is, the trained model can be trained to output the viewing time loss regions in a specific input video. The trained model can be trained using known viewing time loss regions from multiple training videos and features from the multiple training videos corresponding to the known viewing time loss regions. Once the model is trained, the video processing system 110 can use the trained model to process the input video. A set of features (e.g., video features, audio features, semantic features) can be determined for the input video, and these features can be provided to the trained model. The trained model can identify the viewing time loss regions in the input video. (Reference) Figures 2 to 7 The additional structural and operational aspects of these components of example environment 100 are described below.

[0031] Figure 2 An example system 200 is illustrated for determining viewing time loss regions in a media content project. System 200 depicts components of, for example, the video processing system 110 described below. Each component of system 200 is a software component, which includes instructions executed by a processing entity such as a processor.

[0032] Video repository 202 (e.g., database or other storage device) includes videos, for example, managed by content platform 106. A set of training videos 204 can be selected from video repository 202 (e.g., by an administrator or training video selector 205) to train the viewing time loss model 206. In some implementations, training videos 204 can be selected from video repository 202 based on the number of views (e.g., by an administrator or training video selector 205). For example, the more views a particular video receives, the more interaction data (described below) exists for that video (e.g., stored in user activity log 208). Therefore, for example, training videos 204 can include the top X most viewed videos, or can include videos that have been viewed at least Y times. By selecting the most viewed videos, the viewing time loss curve generator 210 can generate a viewing time loss curve based on the largest amount of interaction data (see reference below). Figures 3-4 (Further description).

[0033] The interaction data included in user activity log 208 may include start and stop time information, indicating the start time when a user begins watching the video and the end time when the user ends their video viewing session for each video viewing session. The end time may correspond to the end of the video or may be a point in time before the video ends. In addition to start and stop time information, user activity logs may also include other types of data, such as user interactions with digital components in the video. The information included in user activity log 208 may be anonymized, thus not storing a PIN (Personally Identifiable Information).

[0034] The viewing time loss curve generator 210 can create a viewing time loss curve for each corresponding training video in the training video 204 based on the aggregated information of the training videos included in the user activity log 208. See below for reference. Figure 3 Describe and depict an example viewing time loss curve generated by the viewing time loss curve generator 210.

[0035] Figure 3 Example viewing time loss graph 300 for a video is shown. This video could be one of the examples mentioned above. Figure 2 One of the training videos described is 204.

[0036] like Figure 3 As shown, the X-axis 302 corresponds to the video length ratio. For example, a value of 0 on the X-axis (labeled 304) corresponds to the beginning of the video, while a value of 1 on the X-axis (labeled 306) corresponds to the end of the video. The Y-axis 308 corresponds to the percentage of viewers in a video session who are still watching the video at a given point in time. For example, for point 310 on the watchtime loss curve 312, a value of 1 on the Y-axis and a value of 0 on the X-axis indicate that 100% of viewers were watching the video at the beginning of their video viewing session. For point 314, a value of 0.5 on the Y-axis and a value of 0.48 on the X-axis indicate that when 48% of the video time has ended / passed, 50% of viewers remain and continue watching the video. In other words, this point on the watchtime loss curve also indicates that when 48% of the video time has ended / passed, 50% of viewers have stopped watching the video. For point 316, a value of 0.08 on the Y-axis and a value of 1 on the X-axis indicate that 8% of viewers watched the video until its end.

[0037] Brief reference again Figure 2 The viewing time loss region identifier 212 can identify the viewing time loss region of the training video 204 based on the viewing time loss curve created by the viewing time loss curve generator 210. The identified viewing time loss region information can be stored by the viewing time loss region identifier 212 in the viewing time loss region repository 213. For a given viewing time curve of the video, the viewing time loss region identifier 212 can identify the viewing time loss region as the part of the curve whose slope is greater than the average slope of the entire curve.

[0038] For example, and refer again Figure 3The viewing time loss region identifier 212 has determined the average slope for the viewing time loss curve 312. The average slope is represented by line 318 in Figure 300. The identified viewing time loss regions 320, 322, and 324 correspond to areas of the viewing time loss curve 312 where the slope is greater than the average slope. For example, a slope greater than the average and the identified viewing time loss regions 320, 322, or 324 may correspond to a significant drop in viewership at those time windows of the video. The viewing time loss region identifier 212 can store information such as the identified viewing time loss regions 320, 322, and 324 in the viewing time loss region repository 213.

[0039] The following is for reference. Figure 4 Additional details are discussed regarding the determination of the viewing time loss region by the viewing time loss region identifier 212.

[0040] Figure 4 Example pseudocode 400 is shown for determining viewing time loss regions. For example, the code corresponding to pseudocode 400 can be executed by a processor and can be referenced above. Figure 2 The described viewing time loss region identifier 212 is provided. Section 402 includes pseudocode for finding viewing time loss regions in a video using an average slope method.

[0041] Line 404 includes code for determining average viewing time. Viewing time refers to the y-axis value of a viewing time loss curve (such as viewing time loss curve 312). Viewing time corresponds to the percentage of remaining viewers (e.g., the number of people still watching the video at a given point in time). The percentage of remaining viewers (e.g., the count of remaining viewers divided by the total number of viewers) has an initial value of 1 at the beginning of the video and a value of 0 at the end of the curve. Therefore, the slope of the line connecting the start and end points of the curve can be calculated as 1 (e.g., the change in y) divided by the video length (e.g., the change in x).

[0042] Line 406 begins the iterative construction, where each time point of the video (e.g., per second) is processed and iterated through instructions encapsulated by lines 408 and 410 (described below).

[0043] In line 408, it is determined whether the difference between the viewing time at the current time point and the viewing time at the immediately preceding time point is greater than the average viewing time. Line 408 is equivalent to determining whether the slope of the viewing time loss curve between the current time point and the immediately preceding time point is greater than the average slope. Line 408 also corresponds to determining whether the local derivative of the viewing time loss curve at the current time point is less than the average slope. In line 410, if the difference between the viewing time at the current time point and the viewing time at the immediately preceding time point is greater than the average viewing time, then that time point is added to the viewing time loss region. For example, when adding the first time point to the viewing time loss region, the new viewing time loss region can be "opened" as the current viewing time loss region, and that time point can be added to the current viewing time loss region. Subsequent consecutive time points can be added to the current viewing time loss region as long as the determination in line 408 returns true for those time points(s). Once the determination in line 408 returns false for a time point, the current viewing time loss region can be closed. The viewing time loss region identifier 212 can store information about closed viewing time loss regions in the viewing time loss region repository 213, corresponding to the identified viewing time loss regions. If the determination in line 408 returns true for a subsequent time point, another viewing time loss region can be opened as the current viewing time loss region.

[0044] Refer again Figure 3 For general video content (e.g., for most, all, or almost all videos), the slope of the watchtime loss curve may drop relatively sharply at the points corresponding to the start and end of the video, compared to the average slope. For example, if a video does not include expected or anticipated content, some users may abandon the video near the beginning. As another example, some users may watch a video to the end or near the end, but typically abandon it when it is almost finished. Figure 3As shown, the curve portions 326 and 328 at the beginning and end of the viewing time loss curve 312 have relatively steep slopes compared to the average slope. Due to the larger slopes at the beginning and end of the viewing time loss curve 312, the slope of the flatter middle portion 330 of the viewing time loss curve 312 is generally lower than the average slope of the entire viewing time loss curve 312. Therefore, if the slope increases in regions of the flatter middle portion 330 (e.g., as in viewing time loss regions 320, 322, and 324) and becomes higher than the average slope, the viewing time loss region identifier 212 can identify these regions as viewing time loss regions. Users who abandon the video in viewing time loss regions of the flatter middle portion 330 might do so due to the characteristics of the video. These characteristics will be described in more detail below.

[0045] Because most (if not all, or almost all) viewing time loss curves of all videos exhibit a sharp drop in slope at the beginning and end compared to the average slope, the beginning and end video portions can be considered uninteresting for viewing time loss region identification, as most of all viewing time loss curves exhibit this behavior. Therefore, for the purpose of training the viewing time loss model 206, the viewing time loss region identifier 212 can be excluded from identifying the viewing time loss regions at the beginning and end of the video. See below for reference. Figure 4 Further details regarding excluding areas of viewing time loss are discussed.

[0046] Refer again Figure 4 Section 412 corresponds to excluding (e.g., removing) the continuous viewing time loss region from the beginning of the video, while section 422 corresponds to removing the continuous viewing time loss region from the end of the video. The viewing time loss curve typically has a relatively sharp drop at the beginning and end of the video. In some implementations, the first viewing time loss region appearing at the beginning of the video and the last viewing time loss region appearing at the end of the video, as identified viewing time loss regions, are removed by viewing time loss region identifier 212. In other implementations, time-point analysis can be used to remove the initial and final viewing time loss regions, as described below for lines 412-422.

[0047] For example, the iterative construction begins at line 414, and its analysis starts from the time point at the beginning of the video (i.e., t). i =t1). In line 416, for the current time point (t... i ), determine the next time point (t) i+1Is the next time point in the first viewing time loss region (e.g., the region identified as a viewing time loss region by the viewing time loss region identifier 212 using operations in lines 406-410)? In line 418, if the next time point is in the viewing time loss region, the current time point is removed from the viewing time loss region. In line 420, once the next time point is no longer in the viewing time loss region, the iterative construction exits. Removing the starting time point from the initial viewing time loss region effectively delays the time point at which the viewing time loss region is considered until the viewing time loss curve begins to flatten after an initial sharp drop. Line 422 begins the iterative construction, which includes similar logic to the iterative construction that begins at line 414, but uses time points to process the video in reverse from the last time point.

[0048] In some implementations and / or for some videos, different methods can be used to reduce the number of identified time-loss regions. For example, a time-loss region reduction method can be used when a video identifies regions exceeding a predetermined threshold. In some implementations, time points can be filtered from the time-loss regions using the high percentile value of the derivative to create a more refined list of candidate time-loss regions that includes only regions with the steepest slope. For example, a predetermined number of regions with the steepest slope can be identified as time-loss regions, provided that each of those regions has a steeper slope than the average slope. As another example, the second derivative can be calculated (e.g., at each time point) and used to create a more refined set of identified time-loss regions. For example, regions of time-loss curves with negative second derivatives can be selected as time-loss regions, while regions of time-loss curves with non-negative second derivatives can remain unselected.

[0049] When model trainer 215 trains viewing time loss model 206, the viewing time loss region identified by viewing time loss region identifier 212 can be used by model trainer 215 as label 214. To define label 214, the following equation can be used, where i represents a given region of the viewing time loss curve generated by viewing time loss curve generator 210:

[0050] Target label: If it is the viewing time loss region, then y (i) =1; otherwise 0

[0051] The viewing time loss model 206 can also be trained by the model trainer 215 using features 216 retrieved from the video feature database 218 and / or determined by the feature extractor 219 of the training video 204. Features 216 used to train the viewing time loss model 206 can include features appearing in the training video 204 within a predetermined time window of the time points included in the identified viewing time loss regions. For example, the predetermined time window can include configurable time points X seconds before and after the time points in the viewing time loss regions (e.g., X can be 15 seconds, 6 seconds, 9 seconds, or some other value). The model trainer 215 can train the viewing time loss model 206 based on the assumption that features within the predetermined time window of the viewing time loss regions can predict the viewing time loss regions. Generally, different machine learning methods can be used in the viewing time loss model 206 to generate predicted viewing time loss regions. For example, the viewing time loss model 206 can be a supervised offline model. In such a model, a supervised machine learning model (see below) is trained using the determined features of the video and corresponding labels identifying the viewing time loss regions in the video. Figure 5 and Figure 6 (Describe additional details regarding model training). This allows the use of one or more machine learning methods that lead to the most accurate predictions. See below for reference. Figure 5 Describe the example architecture of the viewing time loss model 206.

[0052] The features 216 used by the model trainer 215 to train the viewing time loss model 206 can include various types of features, such as audio features and video features. The feature extractor 219 can identify features per second and / or per frame of the video based on different feature extraction models 220. For example, the feature extraction model 220 can include image models and audio models. Audio features can correspond to various types of sounds and can be determined by the feature extractor 219 using one or more audio models. Audio features can include embeddings and annotations related to audio information in the video. Video data can be processed using, for example, color space methods, object detection, and / or other methods. Video features can correspond to color space properties, the presence or absence of objects, etc. Features 216 can include annotation features that identify semantic information about specific frames(s), such as objects or semantic entities already identified in the audio and / or video data. For example, video features identified by the feature extractor 219 can include frame annotations generated using the semantic models included in the feature extraction model 220. For example, for a video frame of a person riding a bicycle, the semantic model can generate annotations for "cycling" and "athlete". Other features can include derived features, such as the most common annotations in the video. Some features can be raw embedding features. Raw embedding features are features that describe the appearance of a video, generated by an image understanding model. Raw embedding features can capture visual / semantic concepts and similarities in frames or images.

[0053] In some implementations, the training video 204 is fed as input to the viewing time loss model 206, and the viewing time loss model 206 uses machine learning to analyze each frame of the training video 204 and its corresponding audio to learn which features correspond to the viewing time loss model, rather than having the feature extractor 219 use a separate feature extraction model 220 to extract features and then feed the extracted features to the viewing time loss model 206. However, using such an approach with the viewing time loss model 206 can lead to unacceptable use of resources for training and analysis. A complete evaluation of all pixels in all frames and their corresponding audio can add unacceptable complexity, size, and processing to the viewing time loss model 206. Having the feature extractor 219 use a separate feature extraction model 220 and feed the extracted features to the viewing time loss model 206 can improve the efficiency of the viewing time loss model 206 and enable it to make predictions with fewer training and processing resources.

[0054] Once trained by model trainer 215 as described above, the viewing time loss model 206 can be applied to the input video 221 to generate a predicted viewing time loss region 222 for the input video 221. The input video 221 can be, for example, a recently uploaded video that has little or no interaction data stored in the user activity log 208. See below for reference. Figure 5 In more detail, features 224 of the input video 221 can be extracted by feature extractor 219 and provided as input to the viewing time loss model 206. Features 224 of the input video 221 can be features of a similar type to features 216 determined for training video 204. The features 224 determined for the input video 221 can then be input into the trained viewing time loss model 206, which uses features 224 to generate predicted viewing time loss regions 222 for the input video 221. The viewing time loss model 206 can generate a confidence score for each predicted viewing time loss region 222, which indicates the probability that the predicted viewing time loss region 222 is a viewing time loss region. For example, a predicted viewing time loss region 222 can represent the portion of the input video 221 where the confidence score is greater than a predetermined threshold.

[0055] The predicted viewing time loss region 222 can be used for various purposes and / or provided to various entities. For example, the predicted viewing time loss region 222 can be used to select specific time points in the video to be presented as digital components. For example, time points outside the predicted viewing time loss region 222 (i.e., earlier time points) can be selected as time points for presenting digital components in association with the playback of the input video 221. Therefore, the likelihood of a user abandoning the playback of the digital components (and the input video 221) is reduced compared to the playback of digital components completed after the predicted viewing time loss region. In this way, resource consumption can be more efficient because the possibility of spending resources on unviewed content is reduced. Other advantages of presenting digital components before the predicted viewing time loss region 222 can be realized, such as the potential for additional revenue from a large number of actual views of the digital components that may occur by placing the digital components before the predicted viewing time loss region 222.

[0056] As another example, video processing system 110 may provide the predicted viewing time loss region 222 (and possibly the corresponding confidence value) to the video creator or owner (e.g., in a video studio, such as after the video creator uploads the input video 221 to content platform 106). The presentation of viewing time loss information may also include derived, aggregated, or presented information generated by viewing time loss information generator 226. For example, viewing time loss information generator 226 may generate derived viewing time loss information 228 for the input video 221, which may include a count of the predicted viewing time loss region 222, a timeline view or other presented information describing where the predicted viewing time loss region 222 exists in the input video 221, and / or various other statistics or information about the predicted viewing time loss region 222. For example, viewing time loss information generator 226 may calculate the percentage of runtime of the input video 221 included in the predicted viewing time loss region 222 (e.g., time / total length of video included in the predicted viewing time loss region). As yet another example, the video quality score of input video 221 can be calculated by the viewing time loss information generator 226 based at least on the predicted viewing time loss region 222 (such as the total count and / or total duration based on the predicted viewing time loss region 222). Other examples may include information that the viewing time loss information generator 226 determines how the predicted viewing time loss region 222 of input video 221 compares to viewing time loss predictions or calculations of other videos (e.g., compared to other videos similar in category, duration, or other video characteristics relative to the total count and / or duration of the viewing time loss region). The derived viewing time loss information 228 can be presented along with the predicted viewing time loss region 222 to, for example, a video creator or content provider.

[0057] The video creator can use the received information about the predicted viewing time loss region 222 to adjust the input video 221 to improve its quality. For example, the video creator can adjust or replace the video content in the predicted viewing time loss region 222 and re-upload the input video 221 for processing again using the viewing time loss model 206. The viewing time loss model 206 can be used to generate new predicted viewing time loss regions 222 for the input video 221. The video creator can repeatedly adjust, upload, and review the predicted viewing time loss region 222 information until the video creator is satisfied with the reduction in the count and / or duration of the predicted viewing time loss region 222 for the input video 221.

[0058] By reducing the count and / or duration of the predicted viewing time loss regions 222 in the input video 221, video creators can create videos with improved resource utilization. For example, a video with fewer viewing time loss regions is more likely to be viewed for longer periods compared to a video with a large number of viewing time loss regions. Therefore, resources used to serve the video to users are less likely to be wasted for videos with fewer viewing time loss regions. Furthermore, resources used to serve digital components (which are provided for playback associated with video playback) are less likely to be wasted for videos with fewer viewing time loss regions. With fewer viewing time loss regions in the video, the likelihood of playing digital components at the time points corresponding to the viewing time loss regions in the video decreases.

[0059] The viewing time loss model 206 can be improved over time using a feedback loop. For example, when processing input video 221 using the viewing time loss model 206, as described above, the predicted viewing time loss region 222 of input video 221 can be determined. Over time, input video 221 will likely be viewed by more and more users, and user activity log 208 can include an increasing amount of interaction data for input video 221. As the interaction data for input video 221 grows, the viewing time loss region identifier 212 can identify the actual viewing time loss region of input video 221 (e.g., once a sufficient amount of interaction data has been collected for input video 221) (in the same manner as described above). For example, model trainer 215 can compare the actual viewing time loss region with the predicted viewing time loss region 222, and model trainer 215 can feed the comparison result into viewing time loss model 206 to update viewing time loss model 206. For example, when the actual viewing time loss region matches the predicted viewing time loss region 222, the confidence value or weights used by the viewing time loss model 206 can be increased (e.g., by the model trainer 215). As another example, when the actual viewing time loss region does not match the predicted viewing time loss region 222, the model trainer 215 can decrease the confidence value or weights used by the viewing time loss model 206, or the model trainer 215 can otherwise adjust the viewing time loss model 206, and / or the model trainer 215 can retrain the viewing time loss model 206 using at least the input video 221.

[0060] As another example, in some implementations, the features 216 used to train the viewing time loss model 206 can be refined over time (i.e., as training progresses) by the feature refiner 230. For example, a feature importance calculator 231 can calculate feature importance for each type of feature that may be included in feature 216 (e.g., various types of video, audio, or semantic features that may be included in feature 216). Feature importance can be determined by the feature importance calculator 231 based on which features of the training video 204 correspond to viewing time loss regions identified by the viewing time loss region identifier 212, with a higher number of viewing time loss regions corresponding to features that lead to higher feature importance, and a lower number of viewing time loss regions corresponding to features that lead to lower feature importance. The feature refiner 230 can sort the features according to feature importance, and the feature refiner 230 can remove features with feature importance below a threshold from the features 216 used to train the viewing time loss model 206.

[0061] As another example, the feature ablation process can be performed by feature ablation unit 232. For example, in each training video 204, each feature 216 can be ablated iteratively and separately by feature ablation unit 232 (e.g., with noise). Feature extractor 219 can extract features from each ablated training video (e.g., including features that were ignored during ablation) and provide the features extracted from each ablated training video to viewing time loss model 206. Feature refiner 230 can compare the predicted viewing time loss region 222 generated by viewing time loss model 206 based on the features extracted from the ablated training videos with the viewing time loss region previously generated by viewing time loss region identifier 212 for the corresponding unablated training video 204. If the predicted viewing time loss region 222 generated by the viewing time loss model 206 based on features extracted from the ablated training video is smaller (e.g., in terms of counts and / or duration) than the predicted viewing time loss region of the corresponding unablated training video 204, then the feature refiner 230 can determine that the ablated feature is important for identifying the viewing time loss region. That is, fewer viewing time loss regions are identified in the absence of that feature. If the predicted viewing time loss region 222 generated by the viewing time loss model 206 based on features extracted from the ablated training video is not smaller (e.g., in terms of counts and / or duration) than the predicted viewing time loss region of the corresponding unablated training video 204, then the feature refiner 230 can determine that the ablated feature is not important for identifying the viewing time loss region. That is, the viewing time loss region of the training video is still identified even though the feature is ablated. The feature refiner 230 can remove features that were determined to be unimportant by the feature refiner 230 after ablation from the features 216 used to train the viewing time loss model 206.

[0062] Figure 5 An example architecture for the viewing time loss model 500 is shown below. For instance, the viewing time loss model 500 could be the one referenced above. Figure 2The viewing time loss model 206 is described. The viewing time loss model 500 includes a bidirectional RNN (Recurrent Neural Network) 516. The bidirectional RNN 516 may include a set of gated recurrent units (GRUs), including GRU 518 and GRU 520. For each time T in the input video, cascaded features 502 and 504 may be provided (e.g., by feature extractor 219) as input to the bidirectional RNN 516, specifically as input to the individual GRUs of the bidirectional RNN 516. The cascaded feature 502, which may include the original embedding feature 506, visual feature 508, and audio feature 510, includes features from, for example, 15 seconds before time T to time T. Similarly, the cascaded feature 504, which may include the original embedding feature 512, visual feature 514, and audio feature 515, includes features from, for example, time T to 15 seconds after time T. While a 15-second time window is described, other time window lengths (e.g., 6 seconds, 9 seconds, 20 seconds) may also be used. The results of this study indicate that choosing time windows before and after time T can help the watchtime loss model 500 by providing more context about the video. Choosing the length of the time window may involve trade-offs. Longer time windows can provide more context about the video to the watchtime loss model, but may require more data for training and / or more model complexity (e.g., more GRUs).

[0063] A bidirectional RNN 516 can include a series of steps, each with features at a one-second time marker in the video. For example, regarding the 15-second time window example above, a bidirectional RNN 516 could include 31 steps (e.g., corresponding to 15 seconds before time T, time T itself, and 15 seconds after time T). As another example, in some implementations, a bidirectional RNN 516 can use the LSTM (Long Short-Term Memory) method.

[0064] A fully connected deep neural network 522 can be used as an additional layer on top of the bidirectional RNN 516 as the viewing time loss model 500. For example, the output of the GRU of the bidirectional RNN 516 (e.g., a vector of 31 scaler values) can be fed into a fully connected ReLU (Corrected Linear Unit) layer of the fully connected deep neural network 522. For example, the fully connected deep neural network 522 can use ReLU as the activation function. Various numbers of layers can be used in the fully connected deep neural network 522.

[0065] The output layer of the viewing time loss model 500 can use a sigmoid function 524, which transforms the output from the fully connected deep neural network 522 into a real-valued prediction 526 between 0 and 1, representing the probability that time T is a viewing time loss region in the input video. The larger the prediction 526, the more likely T is to be a viewing time loss region in the input video. More specifically, and as another example, the sigmoid function 524 can generate a score (e.g., a binary score of 0 or 1, or a score appearing in a range such as 0 to 1) for each region of the input video. The score for each region represents the probability that the region is a viewing time loss region. If the score of a particular region meets (e.g., meets or exceeds) a certain threshold (e.g., 1 or greater than 0.5), the sigmoid function 524 can determine that the particular region is a viewing time loss region. On the other hand, if the score of a particular region does not meet (e.g., is less than) a certain threshold (e.g., 1 or 0.5), the sigmoid function can determine that the particular region is not a viewing time loss region. For example, an administrator could set such a threshold. As another example, a certain threshold can be determined automatically.

[0066] Other types of architectures can be used for the viewing time loss model 500. As mentioned above, an LSTM model can be used instead of a GRU. As another example, in some implementations, both an LSTM model and a GRU can be used.

[0067] Figure 6 This is a flowchart of an example process 600 for determining watchtime loss zones in a media content project. The operation of process 600 is described below by... Figures 1 to 5 The operation of process 600 is performed by components of the system described and depicted herein. The operation of process 600 described below is for illustrative purposes only. The operation of process 600 can be performed by any suitable device or system (e.g., any suitable data processing apparatus). The operation of process 600 can also be implemented as instructions stored on a computer-readable medium that may be non-transitory. Execution of the instructions causes one or more data processing apparatuses to perform the operation of process 600.

[0068] The video processing system 110 receives the first video (at 602). For example, as referenced above. Figure 2 The process 600 can receive input video 221, such as user input in response to an administrator selecting input video 221 or user input from a batch of processes that trigger processing of input video 221 (and other videos). Although describing video content, the process 600 can also be used for other media content, such as audio content.

[0069] The video processing system 110 determines a set of features of the first video (at 604). For example, as referenced above. Figure 2 The feature extractor 219 can determine features 224 of the input video 221. This set of features may include video features, audio features, or semantic features.

[0070] The video processing system 110 inputs this set of features from the first video into the trained model (in 606). For example, as referenced above. Figure 2 As described above, feature 224 can be provided as input to a trained model such as viewing time loss model 206. As discussed above with respect to viewing time loss model 500, the trained model can be or includes bidirectional recurrent neural networks, fully connected deep neural networks, LSTM networks, and / or other machine learning engines or systems (as referenced above). Figure 5 The trained model is trained to output the viewing time loss region in a specific video, as described above. Figures 2-5 As stated above and summarized below, the viewing time loss region for a specific video defines a time window within which the probability of a user stopping playback of that specific video is greater than a threshold probability. The trained model is constructed using known viewing time loss regions from multiple training videos and features from the training videos corresponding to those known viewing time loss regions. For example, as referenced above... Figure 2 The viewing time loss model 206 is trained using labels 214 and features 216 of the training video 204, where labels 214 are based on the viewing time loss regions of the training video 204 identified by the viewing time loss region identifier 212.

[0071] Training a watchtime loss model may include: identifying a set of training videos; generating a watchtime loss curve for each training video; identifying a watchtime loss region within each watchtime loss curve; and using the watchtime loss region within the watchtime loss curve to train the trained model. For example, as referenced above... Figure 2 The viewing time loss curve generator 210 can generate a viewing time loss curve for each training video 204. The viewing time loss model 206 can be trained by the model trainer 215 using labels 214, which are based on viewing time loss regions identified in the viewing time loss curve by the viewing time loss region identifier 212. The viewing time loss curve for a specific training video can be generated by the viewing time loss curve generator 210 based on interaction data from user activity logs, including, for example, the start and stop times of a user's viewing session for that specific training video. For example, as referenced above... Figure 2 The viewing time loss curve generator 210 can generate a viewing time loss curve based on the interaction data in the user activity log 208.

[0072] In some implementations, determining the viewing time loss region within a viewing time loss curve may include determining the average slope of the viewing time loss curve. For example, as referenced above... Figure 3 As described above, line 318 depicts the average slope of the viewing time loss curve 312. The average slope of the viewing time loss curve can correspond to a threshold probability used to determine a viewing time loss region. For example, a region of the viewing time loss curve with a slope greater than the average slope can be identified as a viewing time loss region in the viewing time loss curve. For example, as referenced above... Figure 3 The viewing time loss regions 324, 322, and 320 have been identified as viewing time loss regions based on having a slope that is greater than the average slope of the viewing time loss curve 312.

[0073] In some implementations, the first viewing time loss region at the beginning of the viewing time loss curve and the last viewing time loss region at the end of the viewing time loss curve are not used to train the trained model. For example, as referenced above. Figure 2 and Figure 3 The viewing time loss regions corresponding to curve portions 328 and 326 are not used to train the viewing time loss model 206, even though the slope of those portions of the viewing time loss curve is greater than the average slope.

[0074] Training a trained model using viewing time loss regions in a viewing time loss curve can include: for each viewing time loss region determined for a specific training video, determining features of the specific training video appearing within a predetermined time window of the viewing time loss region; and training the trained model using the features of the specific training video appearing within the predetermined time window of the viewing time loss region determined for the specific training video. For example, as referenced above... Figure 2 and Figure 3 The feature extractor 219 can generate features of the training video 204 within a predetermined time window of the viewing time loss regions 324, 322, and 320 as features 216.

[0075] The video processing system 110 obtains data (in 608) about the viewing time loss region of the first video from the trained model. For example, as referenced above. Figure 2 The viewing time loss model 206 can output the predicted viewing time loss region 222 of the input video 221.

[0076] The video processing system 110 provides data (at 610) regarding the identified viewing time loss region to the entities involved in providing the first video to the user. For example, as referenced above. Figure 1The data regarding the identified viewing time loss region can be provided to the client device 102 of the owner of the first video. As another example, the data regarding the viewing time loss region can be provided to the content provider 108 to determine the time points in the first video where digital components are displayed. For example, as referenced above... Figure 1 Content provider 108 can determine when to schedule the presentation of digital components before a defined viewing time loss zone. As another example, content provider 108 can use data about the identified viewing time loss zones to determine when to serve digital components to client device 102. For example, a first set of digital components scheduled to play before the defined viewing time loss zone can be sent to client device 102 based on priority (e.g., before scheduling the first set of digital components, possibly even along with the video), and a second set of digital components scheduled to play after the first defined viewing time loss zone (e.g., by content provider 108) can be configured to be served on a pay-as-you-go basis (e.g., exactly when the digital components are scheduled to play). That is, since some users may tend to stop watching the video during the first defined viewing time loss zone, content provider 108 can serve some high-priority digital components before the first viewing time loss zone and lower-priority digital components before each subsequent viewing time loss zone.

[0077] Data about the viewing time loss regions provided to the client device 102 of the owner of the first video can be presented as quality information for the first video. The quality information may indicate the count and / or duration of the identified viewing time loss regions of the first video. If the quality information indicates low or otherwise unacceptable quality (e.g., the count or duration of the viewing time loss regions exceeds a predetermined threshold), the owner can use the viewing time loss region information to adjust those portions of the first video and re-upload the first video (e.g., to content platform 106) as an updated version. The video processing system 110 can use a trained model to process the updated version of the first video to generate updated viewing time loss information for the updated version of the first video and provide the updated viewing time loss information for the updated version of the first video to the owner of the first video. The owner can repeat the adjustment and uploading process until an acceptable quality level is achieved.

[0078] Figure 7This is a block diagram of an example computer system 700 that can be used to perform the operations described above. System 700 includes a processor 710, memory 720, storage device 730, and input / output device 740. Each of components 710, 720, 730, and 740 can be interconnected, for example, using a system bus 750. Processor 710 is capable of processing instructions that execute within system 700. In some implementations, processor 710 is a single-threaded processor. In another implementation, processor 710 is a multi-threaded processor. Processor 710 is capable of processing instructions stored in memory 720 or on storage device 730.

[0079] Memory 720 stores information within system 700. In one implementation, memory 720 is a computer-readable medium. In some implementations, memory 720 is a volatile memory cell. In another implementation, memory 720 is a non-volatile memory cell.

[0080] Storage device 730 provides massive storage for system 700. In some implementations, storage device 730 is a computer-readable medium. In various implementations, storage device 730 may include, for example, a hard disk drive, an optical disk drive, a storage device shared by multiple computing devices over a network (e.g., a cloud storage device), or some other high-capacity storage device.

[0081] Input / output device 740 provides input / output operations for system 700. In some implementations, input / output device 740 may include one or more of a network interface device (e.g., an Ethernet card), a serial communication device (e.g., an RS-232 port), and / or a wireless interface device (e.g., an 802.11 card). In another implementation, input / output device may include a driver device configured to receive input data and send output data to peripheral device 760 (e.g., a keyboard, printer, and display device). However, other implementations, such as mobile computing devices, mobile communication devices, set-top box television client devices, etc., may also be used.

[0082] Although already Figure 7 An example processing system is described herein, but the subject matter and functional operations described herein can be implemented in other types of digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed herein and their equivalents, or in a combination of one or more of them.

[0083] The embodiments of the subject matter and operation described in this specification can be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in a combination of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a computer storage medium (or media) for execution by or control of the operation of a data processing device. Alternatively or additionally, the program instructions can be encoded on artificially generated propagated signals, such as machine-generated electrical, optical, or electromagnetic signals, generated to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium can be or is included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of these. Furthermore, although the computer storage medium is not a propagated signal, it can be a source or destination of computer program instructions encoded in artificially generated propagated signals. The computer storage medium can also be or be included in one or more separate physical components or media (e.g., multiple CDs, discs, or other storage devices).

[0084] The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0085] The term "data processing apparatus" encompasses all kinds of devices, apparatuses, and machines used for processing data, including, for example, programmable processors, computers, systems-on-a-chip, or a combination thereof. The apparatus may include special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, cross-platform runtime environments, virtual machines, or combinations thereof. The apparatus and execution environment can implement a variety of different computing model infrastructures, such as network services, distributed computing, and grid computing infrastructures.

[0086] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, objects, or other units suitable for use in a computing environment. A computer program may, but does not necessarily, correspond to a file in a file system. A program may be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), as a single file dedicated to the program in question, or as multiple collaborating files (e.g., a file storing one or more modules, subroutines, or code sections). A computer program can be deployed to execute on a single computer, located in one place, or distributed across multiple computers interconnected by a communication network.

[0087] The processes and logic flows described in this specification can be executed by one or more programmable processors that execute one or more computer programs to perform actions by manipulating input data and generating outputs. The processes and logic flows can also be executed by special-purpose logic circuits, and the devices can be implemented as special-purpose logic circuits, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).

[0088] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for performing actions according to instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, to receive data from or transfer data to, or both. However, a computer does not need to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive). Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as: semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM discs. The processor and memory can be supplemented or incorporated by dedicated logic circuitry.

[0089] To provide interaction with the user, embodiments of the subject matter described in this specification can be implemented on a computer having: a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor; and a keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser on the user's client device in response to a request received from a web browser.

[0090] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes backend components, such as a data server; or middleware components, such as an application server; or frontend components, such as a client computer with a graphical user interface or web browser through which a user can interact with an implementation of the subject matter described in this specification; or any combination of one or more such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), the Internet (e.g., the Internet) and peer-to-peer networks (e.g., self-organizing peer-to-peer networks).

[0091] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network. The client-server relationship arises from computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data (e.g., HTML pages) to a client device (e.g., for the purpose of displaying data to a user interacting with the client device and receiving user input from them). Data generated at the client device (e.g., the result of user interaction) can be received from the client device at the server.

[0092] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features specific to particular embodiments of a particular invention. Some features described in this specification in the context of independent embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in some combinations, and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and the claimed combination may be for sub-combinations or variations thereof.

[0093] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order or sequence shown, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0094] Therefore, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific order or sequence shown to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

Claims

1. A computer-implemented method for determining regions of viewing time loss, comprising: Receive the first video; Identify a set of features in the first video; The features of the first video are input into the trained model, wherein the trained model is trained to output the viewing time loss region in a specific video. The trained model is trained using labels corresponding to known viewing time loss regions in multiple training videos and features of multiple training videos corresponding to known viewing time loss regions. The viewing time loss region of a specific video defines the time window of that specific video, during which the probability that the user stops playing the specific video is greater than a threshold probability. In response to inputting this set of features from the first video into the trained model, data on the viewing time loss region of the first video is obtained from the trained model; Provide data on the viewing time loss region of the first video to the entities involved in providing the first video to users; and The trained model is trained using a set of training videos, wherein training the trained model includes: For each training video in this set: Generate the viewing time loss curve for the training video; Identify the viewing time loss region in the viewing time loss curve; and The model was trained using the viewing time loss region from the viewing time loss curves of the training videos. The viewing time loss region in the viewing time loss curve includes: Determine the average slope of the viewing time loss curve, where the average slope corresponds to the threshold probability; and The region of the viewing time loss curve with a slope greater than the average slope is defined as the viewing time loss region in the viewing time loss curve. Specifically, data regarding the viewing time loss region of the first video is provided to the content provider. The content provider uses this data to determine time points within the first video at which digital components containing content different from that in the first video should be played. In this process, content providers use data on the viewing time loss region of the first video to determine when to serve digital portions to client devices during playback of the first video. The first set of digital components, which is scheduled to be played before the first determined viewing time loss zone, is sent to the client device by the content provider along with the video, and the second set of digital components, which is scheduled to be played after the first determined viewing time loss zone, is sent to the client device by the content provider based on on-demand after the first determined viewing time loss zone.

2. The computer-implemented method of claim 1, wherein, The method further includes providing data on the viewing time loss region of the first video to the owner of the first video, and the method also includes: Receive an updated version of the first video that has been modified by the owner of the first video; The trained model is used to process the updated version of the first video to generate updated data on the viewing time loss region of the updated version of the first video; and Provide the owner of the first video with updated data on the viewing time loss area of ​​the updated version of the first video.

3. The computer-implemented method of claim 1, wherein, The viewing time loss curve involves the video length ratio and the ratio of viewers in a video session who are still watching the video at a given point in time.

4. The computer-implemented method of claim 1, wherein, Data regarding the viewing time loss regions of the first video includes the predicted start and stop times of the viewing time loss regions, the count of viewing time loss regions in the first video, the total duration of the viewing time loss regions in the first video, and / or the percentage of the length of the first video corresponding to the identified viewing time loss regions.

5. The computer-implemented method of claim 1, wherein, Using the viewing time loss region in the viewing time loss curve of the training video set to train the trained model includes: using viewing time loss regions that are different from the first viewing time loss region at the beginning of the viewing time loss curve and the last viewing time loss region at the end of the viewing time loss curve to train the trained model.

6. The computer-implemented method of claim 1, wherein, Using the viewing time loss region in the viewing time loss curve to train the trained model includes: For each viewing time loss region determined for a specific training video, identify the characteristics of the specific training video appearing within a predetermined time window of the viewing time loss region; and The trained model is trained using features identified from specific training videos.

7. The computer-implemented method of claim 1, wherein, The viewing time loss curve for a specific training video is generated from interaction data obtained from user activity logs, which include the start and stop times of user viewing sessions for that specific training video.

8. The computer-implemented method of claim 1, wherein, The trained models include bidirectional recurrent neural networks and fully connected deep neural networks.

9. The computer-implemented method of claim 1, wherein, The features of the first video and the features of multiple training videos corresponding to the known viewing time loss region include video features, audio features, or semantic features.

10. A system for determining regions of viewing time loss, comprising: One or more memory devices that store instructions; as well as One or more data processing means are configured to interact with the one or more memory devices and, when executing instructions, perform the method according to any one of claims 1 to 9.

11. A computer-readable medium storing instructions that, when executed by one or more data processing devices, cause the one or more data processing devices to perform the method according to any one of claims 1 to 9.

Images