Receiving terminal, content distribution system

Abstract

To improve a level of satisfaction obtained after viewing contents when the contents is played back in a shortened manner.SOLUTION: The present invention is a receiving terminal 10 playing back contents distributed from contents distribution server 50, and includes: contents information acquisition portion which acquires importance information of a segment in which the plurality of contents from the content distribution server are divided into a plurality of pieces of the contents; a viewer situation estimation portion which determines a viewable time during which a viewer can view the contents and a first playing back speed at the viewing; a segment selection portion which selects on the basis of the importance information, the segment to be played back in such a manner that the content can be played back in the viewable time at the first playing back speed; and a playing back portion which plays back at the first playing back speed, segment selected by the segment selection portion.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to video streaming distribution technology.

Background Art

[0002] In the streaming video distribution of video on the Internet (hereinafter referred to as content), there is a shift from a streaming distribution method using a dedicated server and a dedicated protocol to a method of streaming distribution using the HTTP protocol by a general-purpose Web server. This method is the mainstream in distributions with a large number of receiving terminals. In the streaming distribution method using the HTTP protocol, a technology called adaptive streaming is known, which reduces the image quality somewhat but tries to avoid stopping the playback as much as possible. Adaptive streaming is a method of preparing a plurality of videos with different bitrates and switching to an appropriate video according to the communication bandwidth and playing it back. As adaptive streaming, in addition to the spread of the proprietary technology by IT vendors, MPEG-DASH (ISO / IEC 23009-1), an international standard for unifying the streaming method, has been established. The basic concept of any of these technologies is the same. The Web server prepares content obtained by encoding video content in one or more qualities (screen size and bitrate) into files of about several seconds to several tens of seconds each (segments), and a manifest file describing the attributes and URLs of these video contents. The receiving terminal selects the appropriate quality in a timely manner considering the screen size of the receiving terminal and the state of the network bandwidth of the transmission path from the manifest file, receives the segments one after another, and connects them into one video content for playback (see, for example, Non-Patent Document 1).

[0003] There are no standardized rules for selecting quality, and methods that control it using network conditions and the remaining buffer capacity of the viewing terminal are widely used (see, for example, Non-Patent Document 2). Furthermore, a known method involves determining the current transmission speed each time the receiving terminal receives a segment, obtaining statistical information related to the expected future transmission speed as a function of the current transmission speed, deriving a target transmission speed from the statistical information, and receiving subsequent segments according to the target transmission speed (see, for example, Patent Document 1). Additionally, there is a growing demand for the ability to watch videos exactly as one wishes (see, for example, Non-Patent Document 3). [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Special Publication No. 2016-506644 [Non-patent literature]

[0005] [Non-Patent Document 1] "Next-Generation Video Streaming Technology 'MPEG-DASH': Technical Overview, Standardization, and Related Technology Trends," Journal of the Institute of Image Information and Television Engineers, Vol. 67, No. 2, 2013, pp. 109-115. [Non-Patent Document 2] "Neural Adaptive Video Streaming with Pensieve", SIGCOMM, pp197-210(Aug.2017) [Non-Patent Document 3] "People Who Watch Movies on Fast Forward: Fast English, Spoilers - The Current State of Content Consumption," by Toyoshi Inada, 1st edition, Japan, Kobunsha Co., Ltd., April 2022. [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] However, conventional technology has the problem that viewing content in a shortened format may result in lower viewer satisfaction. In recent years, with the increasing amount of video content available, viewers sometimes watch videos in a shortened format, fast-forwarding or skipping, in order to obtain as much information as possible within their available time. The demand for shortened playback is expanding, with various streaming services introducing settings for double-speed viewing. One organization conducted a survey on the genres of videos that people want to watch in shortened format, and it was reported that this includes not only informational programs but also dramas and movies. Furthermore, there is a growing demand for watching videos at one's own pace.

[0007] However, the technology to play videos in a way that viewers can understand within their available time is not yet available, and there is room for improvement in terms of satisfaction when watching important scenes within a limited time.

[0008] In view of the above problems, the present invention aims to improve viewer satisfaction when content is played back in a shortened format. [Means for solving the problem]

[0009] In view of the above issues, the present invention provides a receiving terminal for playing content distributed from a content distribution server, comprising: a content information acquisition unit that acquires importance information of segments into which the content is divided from the content distribution server; a viewer status estimation unit that determines the viewing time during which a viewer can view the content and a first playback speed during viewing; a segment selection unit that selects segments to be played back based on the importance information so that the content can be played back at the first playback speed during the viewing time; and a playback unit that plays back the segments selected by the segment selection unit at the first playback speed. [Effects of the Invention]

[0010] This can improve viewer satisfaction when content is played in a shortened format. [Brief explanation of the drawing]

[0011] [Figure 1] This is a diagram for explaining adaptive streaming, which is one of the streaming distributions by the HTTP protocol. [Figure 2] This is a diagram for explaining an outline of the video playback method of this embodiment. [Figure 3] This is a diagram showing a system configuration of an example of a content distribution system. [Figure 4] This is a diagram showing a hardware configuration of an example of a content distribution server and a receiving terminal. [Figure 5] The functional block diagram of an example of the content distribution system is shown. [Figure 6] This is a diagram showing an example of a manifest file. [Figure 7] This is a diagram showing an example of importance information stored in the manifest storage unit. [Figure 8] This is a diagram showing an example of a viewing purpose setting screen displayed on the receiving terminal. [Figure 9] This is an example of a sequence diagram for explaining the flow in which the receiving terminal communicates with the content distribution server and the receiving terminal plays back the content. [Figure 10] This is an example of a flowchart diagram for explaining the process in which the segment selection unit determines a segment. [Figure 11] This is a diagram showing an example of a playback policy setting screen displayed on the receiving terminal. [Figure 12] This is a diagram showing an example of the language information content of each segment. [Figure 13] This is an example of a flowchart diagram for explaining the process in which the segment selection unit determines a segment based on the language information content.

Mode for Carrying Out the Invention

[0012] Hereinafter, as an example of an embodiment for carrying out the present invention, a content distribution system and a content distribution method performed by the content distribution system will be described.

[0013] <An Example of Adaptive Streaming> First, referring to FIG. 1, adaptive streaming will be described. FIG. 1 is a diagram for explaining adaptive streaming, which is one of the streaming distributions using the HTTP protocol.

[0014] (1) On the Web server 102 side, the video is encoded by the encoder 101 in multiple qualities (e.g., high quality, medium quality, low quality). The video is segmented into segments in units of several seconds for each quality. In FIG. 1, any n (n is a natural number) of low-quality 1 to 4, medium-quality 1' to 4', and high-quality 1" to 4" is one segment 7. In FIG. 1, the size of one segment 7 schematically shows the size of the segment 7.

[0015] Also, a manifest file 103 is registered in the Web server 102. In the manifest file 103, attribute information such as the URL of the segment 7, the bitrate and image size for each quality are registered.

[0016] (2) The receiving terminal 10 receives the manifest file 103 from the Web server 102 and grasps the content bitrate and segment configuration from the manifest file 103.

[0017] (3) Also, the receiving terminal 10 selects the quality in units of segments based on the screen size of the receiving terminal 10 and the status of the network reception bandwidth 104, and receives the segment 7 by the HTTP protocol. For example, if the reception bandwidth is wide, the receiving terminal 10 selects high quality, and if the reception bandwidth is narrow, it selects low quality.

[0018] (4) The receiving terminal 10 stitches together the received segments 7 in order according to the manifest file 103 and plays them back.

[0019] In this way, the receiving terminal 10 can receive a series of videos and continue playback by repeating the processes of (3) and (4).

[0020] <Content playback method of this embodiment> However, with the increase in VOD (Video On Demand) providers, the amount of content being supplied has surged. As a result, viewers often lack the time to consume all the content. The receiving terminal 10 does not configure the video playback method based on the amount of time the viewer can dedicate to watching the content or the playback speed at which the viewer can keep up with understanding it, nor does it perform any processing to allow viewers to grasp the content of the video in a short amount of time. Therefore, viewers' satisfaction when watching important scenes within a limited viewing time is not improved.

[0021] Therefore, in this embodiment, the receiving terminal 10 selects segments of high importance based on factors such as the viewing time available to the viewer and the playback speed at which the viewer can keep up with understanding the content. In this way, the viewer can grasp the content of the video in a short amount of time.

[0022] <About Terminology> A segment is a video file created by dividing an encoded video into segments ranging from a few seconds to tens of seconds in length. Segments may also be referred to as sections, parts, or parts.

[0023] Content refers to the information provided through a medium for educational or entertainment purposes. Content can be video, sometimes also called film or video. Content can be anything, including movies, animation, television broadcasts, or self-produced videos.

[0024] A manifest file, primarily used in MPEG-DASH (an adaptive HTTP-based protocol for streaming content over the internet), contains information necessary for a receiving device to retrieve and play segments, such as the segment order, a list of segments, URLs, information about the entire content, and encryption information. Note that video distribution communication protocols are not limited to MPEG-DASH. Furthermore, a manifest file may be referred to by other names, such as a configuration file.

[0025] A video distribution protocol is a communication protocol used for streaming distribution. It can be MPEG-DASH, HLS (HTTP Live Streaming), or a proprietary communication protocol of the content distribution server. [Examples]

[0026] In Example 1, we describe a receiving terminal 10 that determines which segments to play based on the viewer's available viewing time and understandable playback speed, using the importance of each segment.

[0027] First, Figure 2 is a diagram illustrating the general outline of the video playback method of this embodiment. (1) The content creators, etc., have previously recorded the importance of each scene in the manifest file 103. The content distribution server 50 holds the manifest file 103 along with the content. (2) The receiving terminal 10 searches for information on the viewer's schedule management application or transfer application within the terminal. Alternatively, the receiving terminal 10 may search for similar information from a server on the network. (3) The receiving terminal 10 obtains the viewing time that will be available until the next event or transfer time, or the viewing time set in advance by the viewer 9. (4) Next, the receiving terminal 10 determines the maximum playback speed that does not exceed the viewer's level of understanding. The method for estimating the level of understanding will be described later. (5) The receiving terminal 10 prioritizes segments of high importance so that they fit within the available viewing time and plays the content at the determined playback speed.

[0028] Thus, the content distribution system 100 of this embodiment determines and plays segments based on the importance of scenes and other elements, at a playback speed that viewers can understand within the limited viewing time of the viewer 9, thereby increasing satisfaction after watching the video.

[0029] <Example System Configuration> Referring to Figure 3, the system configuration of the content distribution system 100 according to this embodiment will be described. Figure 3 is a diagram showing the system configuration of an example of the content distribution system 100. The content distribution system 100 includes a content distribution server 50 and a receiving terminal 10, and the content distribution server 50 and the receiving terminal 10 are connected to each other via a wide-area network N1 such as the Internet. The receiving terminal 10 is connected to network N1 via network N2. Network N2 is a LAN or Wi-Fi (registered trademark) within the home or facility where the receiving terminal 10 is located, or a mobile phone network such as 3G, 4G, 5G, or 6G.

[0030] The content distribution server 50 is implemented by one or more information processing devices. The content distribution server 50 provides a video distribution service that distributes content to the receiving terminal 10 via the network N1. A video distribution service is a service that provides movies, TV programs (anime, dramas), user-created videos, etc., to viewers 9 at the time they want to watch them, mainly using the internet. A video distribution service is also called VOD.

[0031] The content distribution server 50 may be implemented using cloud computing or by a single information processing device. Cloud computing refers to a form of computing where network resources are used without being aware of specific hardware resources. The content distribution server 50 may reside on the internet or on-premises. On-premises residency is an example of a situation where content is distributed only within the company for seminars, etc.

[0032] The receiving terminal 10 is, for example, an information processing device used by the viewer 9 to view content. The receiving terminal 10 can be, for example, a desktop PC, a notebook PC, a smartphone, a tablet device, or a PDA (Personal Digital Assistant). However, the receiving terminal 10 only needs to be able to receive and play content, and can run a web browser or a native playback application. The viewer 9 operates the receiving terminal 10 to connect to the content distribution server 50 and download the content. The viewer 9 can also upload their own videos to the content distribution server 50.

[0033] In the following, when distinguishing between multiple receiving terminals 10, we will use subscripts such as "receiving terminal 10a," "receiving device 10b," "receiving terminal 10c," etc.

[0034] <Example Hardware Configuration> Referring to Figure 4, the hardware configuration of the content distribution server 50 and the receiving terminal 10 according to this embodiment will be described.

[0035] <<Content distribution server and receiving terminal>> Figure 4 shows the hardware configuration of an example of a content distribution server 50 and a receiving terminal 10 according to this embodiment. As shown in Figure 4, the content distribution server 50 and the receiving terminal 10 are built using a computer and include a CPU 501, ROM 502, RAM 503, HD (Hard Disk) 504, HDD (Hard Disk Drive) controller 505, display 506, external device connection I / F (Interface) 508, network I / F 509, bus line 510, keyboard 511, pointing device 512, DVD-RW (Digital Versatile Disk Rewritable) drive 514, and media I / F 516.

[0036] Of these, the CPU 501 controls the operation of the entire content distribution server 50 and receiving terminal 10. The ROM 502 stores programs used to drive the CPU 501, such as IPL. The RAM 503 is used as the work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading or writing of various data to the HD 504 according to the control of the CPU 501. The display 506 displays various information such as cursors, menus, windows, characters, or images. The external device connection I / F 508 is an interface for connecting various external devices. In this case, external devices include, for example, USB (Universal Serial Bus) memory and printers. The network I / F 509 is an interface for data communication using the network N2. The bus line 510 is an address bus, data bus, etc., for electrically connecting each component such as the CPU 501 shown in Figure 4.

[0037] The keyboard 511 is a type of input means equipped with multiple keys used for inputting characters, numbers, or various instructions. The pointing device 512 is a type of input means used for selecting and executing various instructions, selecting processing targets, moving the cursor, etc. The DVD-RW drive 514 controls the reading or writing of various data to the DVD-RW 513, which is an example of a removable recording medium. Note that the DVD-RW drive 514 is not limited to DVD-RW, but may also be DVD-R, etc. The media I / F 516 controls the reading or writing (storage) of data to the recording medium 515, such as flash memory.

[0038] <About the features> Next, with reference to Figure 5, the functional configuration of the content distribution system 100 will be described. Figure 5 shows a functional block diagram of an example of the content distribution system 100.

[0039] <<Content distribution server>> The content distribution server 50 includes a communication unit 51, a segment distribution unit 52, a segment extraction unit 53, a distributed segment list storage unit 54, a request reception unit 55, a segment storage unit 56, and a manifest management unit 57. Each of these functional units is a function or means realized by the CPU 501 shown in Figure 4 executing instructions contained in one or more programs installed on the content distribution server 50. The distributed segment list storage unit 54 and the segment storage unit 56 are built on the HD 504, etc., shown in Figure 4.

[0040] The communication unit 51 communicates with the receiving terminal 10 using a communication protocol such as HTTP. The protocol stack, from the lower layer, consists of Ethernet (registered trademark) or Wi-Fi (registered trademark), IP, TCP or UDP, and HTTP. These are just examples, and a communication protocol specifically for the video distribution service may be used.

[0041] The segment storage unit 56 stores the content to be distributed in segments. A segment is a distribution unit in which content is divided into fixed time intervals. Content is distributed in segments. The length of the segments may be determined as appropriate. The length of the segments does not have to be constant. Shorter segments allow for more accurate selection of segments according to their importance, but distribution overhead is more likely to occur. In this embodiment, for example, the segment length is set to 5 seconds. Also, segments of different lengths may be prepared depending on the communication bandwidth.

[0042] The manifest management unit 57 manages manifest files 103 for each content item. An example of a manifest file 103 is shown in Figure 6.

[0043] The request receiving unit 55 receives a request for a manifest file from the receiving terminal 10. The request receiving unit 55 also receives a request for content (segments) from the receiving terminal 10. Segments are specified by URLs listed in the manifest file. The segment requested by the receiving terminal 10 is determined according to the importance of the segment, etc.

[0044] The segment extraction unit 53 obtains segment information from the manifest management unit 57 based on the request from the receiving terminal 10 obtained by the request receiving unit 55. The segment extraction unit 53 then extracts segments from the segment storage unit 56 based on the segment information obtained from the manifest management unit 57.

[0045] The segment distribution unit 52 distributes the segments extracted by the segment extraction unit 53 to the receiving terminal 10.

[0046] The distributed segment list storage unit 54 stores a list of segment numbers distributed by the content distribution server 50, corresponding to the receiving terminal 10. This is to prevent the content distribution server 50 from unnecessarily sending the same segment to the receiving terminal 10 again.

[0047] Figure 6 shows an example of a manifest file 103. The manifest file 103 contains information that associates segment numbers with bitrate, playback time, URL, and importance information. Note that the manifest file 103 may contain many other items.

[0048] The segment number is the identification information for the segment. In this embodiment, as an example, the segment numbers are sequential.

[0049] Bitrate is the amount of data per second and is one of the factors that determine the image quality of content.

[0050] The playback time is the playback time of the segment relative to the start of the content. For example, if one segment is 5 seconds long, the playback time of the 10th segment will be 45 seconds. The playback time may be used by the receiving terminal 10 to determine the playback order of the segments.

[0051] The importance information indicates how important the segment is to the viewer's understanding. A higher importance rating means greater importance. For example, an importance rating of "0" indicates a scene with only background music, an importance rating of "1" indicates a scene with dialogue (with subtitles), and an importance rating of "2" indicates a scene with dialogue that must be listened to in order to understand the story. Importance information can be pre-set by the person in charge, or it can be automatically determined and set by an AI (artificial intelligence) that has learned the relationship between segments and importance through machine learning. Details of the importance information will be described later in Figure 7.

[0052] <<Receiving terminal>> Let's return to Figure 5 for explanation. The receiving terminal 10 includes a communication unit 11, a content information acquisition unit 12, a manifest storage unit 13, a viewer status estimation unit 14, a segment selection unit 15, a segment acquisition unit 16, a display unit 18, and a playback unit 17.

[0053] The content information acquisition unit 12 obtains the manifest file of the content desired by the viewer 9 from the content distribution server 50. The content information acquisition unit 12 sends a request for the manifest file to the content distribution server 50 to a pre-known URL, and receives the manifest file from the content distribution server 50 as a response to the request.

[0054] The content information acquisition unit 12 analyzes the received manifest file and creates a video quality list associating the content's bitrate, playback time, segment URL, and importance information, and stores it in the manifest storage unit 13. The video quality list does not have to be exactly the same as the manifest file, but in this embodiment, for the sake of explanation, the video quality list is assumed to contain the same information as the manifest file.

[0055] The viewer status estimation unit 14 determines the maximum playback speed when the viewer's understanding of the content is above a threshold, based on the viewer's status measured by sensors and other means, as well as the viewer's status obtained from input by the viewer 9 themselves. The viewer status estimation unit 14 also estimates the available viewing time based on various applications running on the receiving terminal 10. The viewer status estimation unit 14 outputs the available viewing time and playback speed to the segment selection unit 15.

[0056] The segment selection unit 15 determines the segment to be acquired and its playback speed based on the appropriate playback speed and viewing time for the viewer 9 obtained from the viewer status estimation unit 14, and importance information obtained from the manifest storage unit 13, and transmits the segment to the segment acquisition unit 16 and the playback speed to the playback unit 17.

[0057] The segment acquisition unit 16 requests the content distribution server 50 to send a request containing the URL of the segment to be acquired, which was obtained from the segment selection unit 15, and receives the segment from the distribution server as a response to the request. The segment acquisition unit 16 stores each segment in a buffer.

[0058] The playback unit 17 sequentially acquires segments from the buffer according to the current playback time, decodes the acquired segments according to the media format (video, audio, text, etc.) and playback speed of the segments, and outputs them to the display unit 18.

[0059] The display unit 18 displays the decoded content input from the playback unit 17 according to the format of that media.

[0060] <Importance information> Figure 7 shows an example of importance information stored in the manifest memory unit 13. The importance information is the importance of each segment determined for each viewing purpose. The importance of each viewing purpose may be determined by voting by monitors (test viewers) or by AI, etc. Note that the maximum importance value may differ depending on the viewing purpose.

[0061] • Overall importance is the overall importance of each segment, judged based on factors such as plot summary, background music, dialogue, plot points, and their resolution.

[0062] • The plot importance rating indicates how important each segment is to understanding the plot, including whether or not it has narration or subtitles.

[0063] • Music importance indicates the extent to which music that is important to the content's image, such as background music and theme songs, is played within the segment.

[0064] • The main cast members are the actors and actresses appearing in the segment.

[0065] In response to this importance information, viewers 9 can input their viewing purpose, such as "I want to know the synopsis" or "I want to see scenes featuring my favorite actor," into the receiving terminal 10. The segment selection unit 15 selects a segment using the importance level of the specified viewing purpose.

[0066] Figure 8 shows an example of the viewing purpose setting screen 210 displayed by the receiving terminal 10. The viewing purpose setting screen 210 is a screen where the viewer 9 sets their viewing purpose (which segments are important to them). The viewing purpose setting screen 210 has radio buttons 211 to 214 corresponding to each viewing purpose, such as overall, synopsis, music, and cast. The user can set their viewing purpose by pressing radio buttons 211 to 214.

[0067] Additionally, when a performer's radio button 214 is pressed, the viewer 9 can operate the performer settings section 215. The performer settings section 215 is a pull-down menu, and performers included in the main performers of the manifest file 103 are displayed in a way that avoids duplication.

[0068] In addition, viewer 9 may also set viewing objectives such as "fast forward all scenes."

[0069] <Operating Procedure> Next, with reference to Figure 9, the overall flow of content playback by the receiving terminal 10 will be explained. Figure 9 is a sequence diagram illustrating the flow of communication between the receiving terminal 10 and the content distribution server 50, and how the receiving terminal 10 plays the content. In the explanation of Figure 9, it is assumed that the receiving terminal 10 and the content distribution server 50 have already started communicating. The viewer 9 is logged into the content distribution server 50 as needed.

[0070] S1: Upon initiating communication between the receiving terminal 10 and the content distribution server 50, the receiving terminal 10 displays a list of content, including recommended content. The viewer 9 selects the content they wish to watch from the list of content displayed by the receiving terminal 10.

[0071] S2: The content information acquisition unit 12 accepts the selection and requests the content's manifest file 103 from the content distribution server 50 via the communication unit 11.

[0072] S3: The request receiving unit 55 of the content distribution server 50 receives a request for the manifest file 103 via the communication unit 51. The request receiving unit 55 then sends the manifest file 103, obtained from the manifest management unit 57 via the segment extraction unit 53, to the receiving terminal 10 via the communication unit 51.

[0073] S4: The content information acquisition unit 12 of the receiving terminal 10 acquires the manifest file 103 via the communication unit 11, converts it into a video quality list, and saves it in the manifest storage unit 13. The viewer 9 displays the viewing purpose setting screen 210 and sets the viewing purpose. The content previously set on the viewing purpose setting screen 210 may be used, in which case the viewer 9 can play the content simply by selecting the content in step S1.

[0074] S5: The viewer status estimation unit 14 of the receiving terminal 10 obtains the viewer's available viewing time and determines the playback speed according to the viewer's level of understanding. The segment selection unit 15 determines the segments to be obtained from the content distribution server 50 using the appropriate available viewing time and playback speed for viewer 9, as well as the importance of the segments. Details of this process are explained in Figure 10. S6: The segment acquisition unit 16 requests the segment determined by the segment selection unit 15 from the content distribution server 50 via the communication unit 11. Since the content is streamed, the content information acquisition unit 12 requests the content distribution server 50 a segment of a quality determined according to the communication bandwidth, as explained in Figure 1.

[0075] S7: The request receiving unit 55 of the content distribution server 50 receives a segment request via the communication unit 51 and transmits it to the segment extraction unit 53. The segment extraction unit 53 retrieves the requested segment from the segment storage unit 56 and transmits it to the segment distribution unit 52. The segment distribution unit 52 transmits the segment to the receiving terminal 10 via the communication unit 51.

[0076] S8: The segment acquisition unit 16 of the receiving terminal 10 receives the segment via the communication unit 11 and stores it in a buffer. The playback unit 17 acquires the segment from the buffer according to the playback time and plays the segment at the playback speed instructed by the segment selection unit 15. The receiving terminal 10 repeats steps S6 to S8 until the playback of the content is finished.

[0077] Next, with reference to Figure 10, the details of how segments are determined will be explained. Figure 10 is a flowchart illustrating the process by which the segment selection unit 15 determines segments. For example, when the receiving terminal 10 accepts the settings on the viewing purpose setting screen 210, the viewer status estimation unit 14 obtains the viewing time and an appropriate playback speed (an example of the first playback speed) (S001). The segment selection unit 15 also sets the initial value "0" to the variable i which specifies importance. i is a variable used to specify importance in ascending order.

[0078] The viewing time can be set to, for example, until the next transfer point if viewer 9 is traveling using a transit guidance application. For example, if viewer 9 is traveling from Tokyo Station to Nagano Station, the viewing time will be from the present until the arrival time at Nagano Station. The viewer status estimation unit 14 obtains the arrival time at Nagano Station from the transit guidance application. It is preferable that the viewer status estimation unit 14 confirms from the location information detected by the receiving terminal 10 that the viewer is traveling along the route proposed by the transit guidance application.

[0079] Alternatively, the viewer status estimation unit 14 may use the time until the next scheduled event from the schedule management application used by the viewer 9 as the available viewing time. If the next scheduled event in the schedule management application is a meeting at 13:00, the available viewing time is from the present until 13:00. The viewer status estimation unit 14 obtains the time of the next scheduled event from the schedule management application.

[0080] Additionally, viewer 9 may input or select a viewing time, such as "30 minutes."

[0081] Regarding playback speed, the viewer status estimation unit 14 may, for example, select the playback speed used during the previous viewing. Alternatively, the viewer status estimation unit 14 may gradually increase the playback speed from the start of playback, and if the level of comprehension decreases (if the level of comprehension falls below a threshold), decrease the playback speed to set the maximum playback speed at which comprehension is possible. Methods for estimating the viewer's level of comprehension include methods based on brain waves, facial expressions, or heart rate. For brain waves, headset-type electroencephalographs are known. It is assumed that the correspondence between brain waves and comprehension is known in advance. For facial expressions, the receiving terminal 10 can acquire a facial image of the viewer 9 with a camera it has or is connected to, and estimate the level of comprehension from the degree of eye opening, etc. If the degree of eye opening is above a threshold, it can be estimated that the level of comprehension is high. For heart rate, the receiving terminal 10 can receive values ​​measured by a wristwatch-type sensor, etc., and estimate the activity level of the autonomic nervous system (sympathetic and parasympathetic nerves) by analyzing the heart rate. Heart rate analysis involves performing a Fourier transform on the interval heart rate signal (RRI) to obtain the frequency distribution, focusing on the LF component (which increases when the sympathetic or parasympathetic nervous system is activated) and the HF component (which indicates parasympathetic nervous system activation). A sympathetic nervous system dominance suggests a high level of comprehension.

[0082] In any case, such as brain waves, facial expressions, and heart rate (other measurable parameters such as blood pressure and electrocardiograms may also be used), the level of comprehension may be associated with these parameters through machine learning. In this case, using brain waves and comprehension, facial expressions (face images) and comprehension, and heart rate (heart rate waveform or Fourier transform results) and comprehension as training data, an identification model is generated that outputs comprehension from brain waves, facial expressions, or heart rate. When the brain waves, facial expressions, heart rate, etc. of the viewer 9 during viewing are input to this identification model, the identification model can output comprehension.

[0083] Machine learning is a technique for enabling computers to acquire human-like learning abilities. It involves computers autonomously generating algorithms necessary for data identification and other decision-making processes from pre-programmed training data, and then applying these algorithms to new data to make predictions. The learning method for machine learning can be supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, or deep learning, or a combination of these methods; the learning method for machine learning is not limited. Examples of machine learning techniques include perceptrons, deep learning, support vector machines, logistic regression, naive Bayes, decision trees, and random forests, and are not limited to the techniques described in this disclosure.

[0084] Furthermore, the "playback speed" here can be a relative value, such as 1.5 times the speed of the original video provided, or an absolute value, such as the speaker's speaking speed (e.g., 300 characters / minute).

[0085] Furthermore, when the viewer status estimation unit 14 determines the playback speed, the viewer may pre-set a range for the playback speed. For example, the viewer can limit the range of playback speed by setting it so that it does not go below 1.0x or so that it does not go above 2.0x.

[0086] Next, the segment selection unit 15 calculates the required playback time when segments with an importance of variable i or greater are played back at the playback speed of step S001 (S002).

[0087] The following is an example of calculating playback time using the overall importance shown in Figure 7 as an example. Note that the length of each segment is assumed to be 5 seconds. 1. Play all segments at 1x speed for 30 seconds. 2. Play all segments at x times the normal speed (30 / x seconds) 3. Play segments with an importance level of 0 or higher at 1x speed for 30 seconds. 4. Play segments with an importance level of 0 or higher at x times the normal speed (30 / x seconds). 5. Play segments with an importance level of 1 or higher at 1x speed for 20 seconds. 6. Play segments with an importance level of 1 or higher at x times the normal speed (20 / x seconds). 7. Play segments with an importance level of 2 or higher at 1x speed for 15 seconds. 8. Play segments with an importance level of 2 or higher at x times the normal speed (15 / x seconds) The calculation method is the same even if other viewing purposes are selected. However, if viewer 9 sets a performer's name on the viewing purpose setting screen 210, the following occurs. The segment selection unit 15 selects only segments in which the performer's name entered by viewer 9 is registered, based on the viewing purpose "I want to watch scenes featuring my favorite actor". For example, if "Mr. A" is entered, the segment selection unit 15 calculates the playback time as follows. • Play the segment containing person A at 1x speed for 15 seconds. Play the segment containing person A at x times the normal speed (15 / x seconds). Next, the segment selection unit 15 compares the playback time calculated in step S002 with the viewable time specified in step S001 and determines whether the playback time calculated in step S002 is longer than the viewable time specified in step S001 (S003).

[0088] If the decision in step S003 is No, there is insufficient viewing time, so the segment selection unit 15 determines whether the variable i is equal to or greater than the number of importance types n-1 (S004). This determination determines whether the variable i has exceeded the maximum importance value. When the importance is 0, 1, or 2, the number of importance types n is 3, and n-1 is 2. Therefore, if i is 2 (the importance is already at its maximum), the decision in step S004 is Yes.

[0089] If the decision in step S004 is No, the segment selection unit 15 adds 1 to the variable i (S005). After this, the process proceeds to step S002, and S002 is executed. Therefore, the segment selection unit 15 repeats steps S002 to S005 until the decision in step S003 is Yes or in S004 is Yes.

[0090] If the decision in step S004 is Yes, then even if variable i is the same as the maximum importance value, the viewable time is less than or equal to the playback time. In this case, the segment selection unit 15 displays a playback policy setting screen to the viewer 9, asking if it is acceptable to exceed the viewable time or playback speed, and accepts the viewer 9's selection (S006).

[0091] Figure 11 shows an example of the playback policy setting screen. The viewer's responses are: allow to exceed the viewing time, allow to increase the playback speed, or stop playback. If the viewer selects to stop playback, the content will not play and the process shown in Figure 10 will end.

[0092] If viewer 9 indicates that it is acceptable to exceed the viewing time limit, the process proceeds to step S008.

[0093] If viewer 9 inputs that it is acceptable to increase the playback speed, the segment selection unit 15 determines the playback speed at which the most important segment can be played within the available viewing time (an example of a second playback speed) (S007). In this case, the total playback time of the most important segments is divided by the available viewing time.

[0094] If the determination in step S003 is Yes, if the determination in step S006 is that it is acceptable to exceed the viewing time, or if, following step S007, the segment acquisition unit 16 acquires segments with an importance of variable i or greater from the content distribution server 50, and the playback unit 17 plays them at the playback speed of step S001 or S007 (S008). Further details regarding the playback speed and importance in step S008 are provided below. If the answer in step S003 is Yes, the segment selection unit 15 decides to play segments with an importance equal to or greater than the current variable i at the playback speed of S001. If the answer in step S006 is Yes, the segment selection unit 15 determines to play the segment with the highest importance at the playback speed of S001. If S007 is used, the segment selection unit 15 determines to play the segment with the highest importance at the playback speed of S007.

[0095] Next, if the playback speed change button is pressed during playback (Yes in S009), the process returns to step S001.

[0096] The process returns to step S008 and continues until the current playback time reaches the content's end time or the stop button is pressed (No in S010). If the determination in step S010 is Yes, the playback process ends.

[0097] Figure 11 shows an example of the playback policy setting screen 220 displayed by the receiving terminal 10. The playback policy setting screen 220 has radio buttons 221 to 223 corresponding to "It is okay to exceed the viewing time," "Increase the playback speed so that playback can be completed within the viewing time," and "Stop playback," respectively. The user can set the playback policy by pressing radio buttons 221 to 223.

[0098] <<When performers are selected for viewing purposes>> The following is a supplement to the processing flow when a performer is selected on the viewing purpose setting screen 210. If the viewing purpose is a performer and the answer is No in step S003, the process proceeds to step S006 without going through step S004. This is because there is no need to change the variable i. Also, in step S007, the playback speed should be determined for the segment in which the specified performer appears.

[0099] <Main effects> As described above, the content distribution system 100 of this embodiment determines and plays segments based on the importance of scenes, etc., within the limited viewing time available to the viewer 9, at a playback speed that the viewer 9 can understand, thereby increasing satisfaction after watching the video. [Examples]

[0100] In this embodiment, the importance is defined as the linguistic information content for each segment, and a receiving terminal 10 that determines segments based on the linguistic information content will be described.

[0101] In this embodiment, we will explain the process using the hardware configuration diagram shown in Figure 4 and the functional block diagram shown in Figure 5, which were described in the above embodiment.

[0102] As shown in Figure 12, in this embodiment, the manifest file 103 contains the language information content of each segment. Figure 12 is an example of the language information content of each segment. Therefore, the language information content is an example of importance information. The language information content is the number of characters in the segment's subtitles or the number of characters in the speaker's dialogue. By determining segments based on the language information content, users can play segments that contain language information as important segments.

[0103] Figure 13 is a flowchart illustrating the process by which the segment selection unit 15 determines a segment based on the linguistic information content. Note that the explanation of Figure 13 mainly focuses on the differences from Figure 10. The process in step S101 may be the same as S001 in Figure 10.

[0104] Next, in step S102, the segment selection unit 15 determines the playback time when segments with a non-zero linguistic information content are played back at the playback speed of S101 (S102). The segment selection unit 15 may exclude segments that do not have a certain minimum linguistic information content, even if they do not have zero linguistic information content, from the playback target.

[0105] The next step, S103, can be the same as S003 in Figure 10.

[0106] If the determination in step S103 is No, there is insufficient viewing time, so the segment selection unit 15 calculates the ratio of the average language information content of segments that do not have zero language information content to the language information content of each segment (S104). The ratio is "Language information content of each segment ÷ Average". This is done to shorten the playback time by increasing the playback speed of segments with high or low language information content.

[0107] Next, the segment selection unit 15 increases the playback speed of segments with a ratio of less than 1 by the reciprocal of the ratio of that segment (S105). In step S105, the playback speed of segments with a ratio of less than 1 was increased, but the playback speed of segments with a ratio of 1 or more may also be increased. In this case, the segment selection unit 15 multiplies the playback speed of segments with a ratio of 1 or more by the ratio of that segment.

[0108] Segments with a high linguistic information content contain a lot of information, but even if the continuous audio is fast, it may be easy for the listener 9 to understand if they are interpreting the meaning as they listen. On the other hand, segments with a low linguistic information content contain less information, so even if the listener 9 misses something, it may not affect their ability to grasp the overall meaning of the content. Therefore, it is acceptable to increase the playback speed for segments with high or low linguistic information content. Also, some viewers may prefer to play all segments at the same speed rather than having different playback speeds for each segment. If the receiving terminal 10 plays all segments at the same speed, the playback speed of all segments is multiplied by the above ratio.

[0109] Therefore, it would be desirable for viewer 9 to be able to set whether to play segments with a high or low amount of linguistic information faster, or to play all segments faster.

[0110] Furthermore, in step S104, the ratio to the average is calculated, but the segment selection unit 15 may multiply the playback speed of segments with less than the average language information content by the ratio of the minimum and maximum values ​​(maximum / minimum) of the language information content of all segments. This allows for faster playback of segments with low language information content. In this case, it is preferable to set a lower limit for the minimum value (considering the language information content to be below the threshold if it is below the threshold) and an upper limit for the maximum value (considering the language information content to be above the threshold if it is above the threshold). This can prevent the playback speed from becoming excessively fast. Alternatively, the playback speed of each segment may be multiplied by the ratio of the minimum and maximum values ​​of the language information content.

[0111] The segment selection unit 15 determines whether the viewable time is greater than the playback time in step S105 (S106). If the determination in step S106 is Yes, the process proceeds to step S109.

[0112] If the determination in step S106 is No, the segment selection unit 15 asks the viewer 9 whether it is acceptable to exceed the viewing time or playback speed (S107). The playback policy setting screen 220 can be the same as in Figure 11.

[0113] If viewer 9 indicates that it is acceptable to exceed the viewing time limit, the process proceeds to step S109.

[0114] If viewer 9 inputs that it is acceptable to increase the playback speed, the segment selection unit 15 determines a playback speed (an example of a third playback speed) that allows segments with non-zero linguistic information content to be played within the available viewing time (S108). In this case, the total playback time of segments with non-zero linguistic information content should be divided by the available viewing time.

[0115] The processing in steps S109 to S111 can be the same as in steps S008 to S010 in Figure 10. A supplementary explanation is provided for step S109. If the answer in step S103 is Yes, the segment selection unit 15 plays segments with a non-zero linguistic information content at the playback speed specified in S101. In this case, the playback speed of each segment is the same. If the answer in step S106 is Yes, the segment selection unit 15 plays segments with a non-zero linguistic information content at the playback speed specified in S105. In this case, segments with a ratio of less than 1 will have different playback speeds. For segments with a ratio of 1 or more, the playback speed will be the value determined by the playback speed in S101. If the viewer inputs in step S107 that it is acceptable to exceed the viewing time, the segment selection unit 15 determines to play segments with non-zero linguistic information content at the playback speed of S101. In this case, the playback speed of each segment is the same. The segment selection unit 15 can also play segments with non-zero linguistic information content at the playback speed of S105, so it may ask the viewer 9 whether to select the playback speed of step S101 or S105. In this case, the segment selection unit 15 may present the respective playback times. If step S108 is followed, the segment selection unit 15 plays segments with a non-zero linguistic information content at a playback speed that allows playback within the available viewing time. In this case, the playback speed of each segment is the same.

[0116] <Main effects> According to this embodiment, based on the linguistic information content of the segment, content can be played at an optimal playback speed for the viewer 9 within the available viewing time, thereby increasing post-viewing satisfaction.

[0117] <Other application examples> Although the best mode for carrying out the present invention has been described above using examples, the present invention is not limited in any way to these examples, and various modifications and substitutions can be made without departing from the spirit of the present invention.

[0118] For example, in this embodiment, the receiving terminal 10 determines the segment to be played, but the content distribution server 50 may also determine the segment. In this case, the receiving terminal 10 sends the viewable time and playback speed to the content distribution server 50, and the content distribution server 50 performs the processing shown in Figure 10. Alternatively, the content distribution server 50 may also determine the viewable time and playback speed. In this case, the content distribution server 50 obtains the viewable time from the viewer's schedule or transfer application on the cloud. The receiving terminal 50 sends sensor values ​​to the content distribution server 50 to estimate the viewer's level of understanding.

[0119] Furthermore, in this embodiment, the segments and playback speed are determined when content is streamed, but this can also be applied when the receiving terminal 10 plays content that has been saved as a file. In this case, the receiving terminal 10 either stores the manifest file and the content, or downloads the entire content from the network and reads the segments determined based on the importance information from the HDD, SSD, etc.

[0120] Furthermore, the configuration examples shown in Figure 5 and other figures are divided according to their main functions to facilitate understanding of the processing performed by the content distribution server and the receiving terminal. The present invention is not limited by the way the processing units are divided or the names of those units. The processing of the content distribution server and the receiving terminal can be further divided into more processing units depending on the processing content. Alternatively, a single processing unit can be divided to include even more processing.

[0121] Each of the functions of this disclosure described above can be implemented not only by software processing through program execution, but also by one or more processing circuits. Here, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and conventional circuit modules designed to execute each of the functions described above. [Explanation of Symbols]

[0122] 10 Receiving terminal 50 Content distribution servers 100 Content Distribution Systems

Claims

1. A receiving terminal that plays content delivered from a content distribution server, A content information acquisition unit that acquires importance information for segments into which the content is divided into multiple parts from the content distribution server, A viewer status estimation unit that determines the viewing time during which the viewer can view the aforementioned content, and the first playback speed during viewing, A segment selection unit selects segments to be played based on the importance information so that the content can be played at the first playback speed within the viewable time, A playback unit plays back the segment selected by the segment selection unit at the first playback speed, A receiving terminal characterized by having the following features.

2. The segment selection unit compares the playback time of segments whose importance information is greater than or equal to a variable with the viewable time. If the viewing time is greater than the playback time, select a segment having importance information greater than or equal to the variable. The receiving terminal according to claim 1, characterized in that, if the viewable time is less than or equal to the playback time, the variable is increased within a range in which the variable does not exceed the maximum value of the importance information, and the playback time of the segment for which the importance information is greater than or equal to the variable is compared again with the viewable time.

3. If the aforementioned variable exceeds the maximum value of the importance information, The receiving terminal according to claim 2, characterized in that the segment selection unit accepts a selection of whether the viewable time may be exceeded or whether playback may be faster than the first playback speed.

4. If the option to play at a speed faster than the first playback speed is accepted, The segment selection unit determines a second playback speed at which the segment with the maximum importance information can be played back within the viewing time. The receiving terminal according to claim 3, characterized in that the playback unit plays back the segment selected by the segment selection unit at the second playback speed.

5. The receiving terminal according to any one of claims 1 to 4, characterized in that the viewer status estimation unit acquires information for estimating the available viewing time from the viewer's schedule management application or transfer application.

6. The receiving terminal according to any one of claims 1 to 4, characterized in that the viewer status estimation unit estimates the viewer's level of understanding from one or more of the brain waves, heart rate, or facial images while increasing the playback speed of the content, and determines the maximum first playback speed within a range in which the level of understanding does not fall below a threshold.

7. The aforementioned importance information is set for each segment according to the viewing purpose, with multiple pieces of importance information set accordingly. The segment selection unit displays a viewing purpose setting screen that accepts the viewing purpose, and accepts the selection of the viewing purpose. The receiving terminal according to claim 1, characterized in that it selects a segment to be played back at the first playback speed within the available viewing time, based on the importance information set based on the viewing purpose for which the selection was accepted.

8. The receiving terminal according to claim 7, wherein the viewing purpose is the performers of the content, and the segment selection unit accepts the names of the performers.

9. The aforementioned importance information is the linguistic information content of the segment, The segment selection unit is, If the viewing time is less than or equal to the playback time of the segment containing non-zero linguistic information, The ratio between the average linguistic information content of the aforementioned segments and the linguistic information content of each segment is calculated, and the third playback speed is calculated by multiplying the playback speed of segments with a ratio of less than 1 by the reciprocal of the ratio. The receiving terminal according to claim 1, characterized in that the playback unit plays back segments with a ratio of 1 or more at the first playback speed and plays back segments with a ratio of less than 1 at the third playback speed.

10. A content distribution system in which a content distribution server distributes content, and a receiving terminal plays that content. The aforementioned receiving terminal is A content information acquisition unit that acquires importance information for segments into which the content is divided into multiple parts from the content distribution server, A viewer status estimation unit that determines the viewing time during which the viewer can view the aforementioned content, and the first playback speed during viewing, A segment selection unit selects segments to be played based on the importance information so that the content can be played at the first playback speed within the viewable time, A playback unit plays back the segment selected by the segment selection unit at the first playback speed, It has, The aforementioned content distribution server is A manifest management unit that associates the content segments with the importance information, A content distribution system characterized by having a segment distribution unit that transmits the segment requested by the receiving terminal to the receiving terminal.

Images