Video playback device and program

Abstract

To present a playing back section in which a viewer does not concentrate on a moving image.SOLUTION: A moving image playing back apparatus comprises: a moving image playing back section which displays a moving image on a display device; a biological information acquisition section which acquires biological information on viewers viewing the moving image; a concentration determination section which determines whether or not the viewers concentrate on the moving image on the basis of the biological information; a result recording section which records a playing back section in which the moving image is divided by a time base and a determination result by the concentration determination section in association with each other; and a result display section which displays a viewing result expressing the playing back section indicating the determination result indicating that the viewers do not concentrate on the moving image on the display device after finishing the reproduction of the moving image.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a video playback device and a program.

Background Art

[0002] Video distribution services that stream video content through communication networks such as the Internet are being used. In video distribution services, there is a shift from a method of streaming distribution using a dedicated protocol by a dedicated server to a method of streaming distribution using a general-purpose protocol by a general-purpose server.

[0003] For example, Non-Patent Document 1 discloses MPEG-DASH (Moving Picture Experts Group - Dynamic Adaptive Streaming over HTTP, ISO / IEC 23009-1), which is an international standard. MPEG-DASH is formulated with the intention of unifying the streaming distribution method. In MPEG-DASH, streaming distribution is performed using the HTTP (Hypertext Transfer Protocol) protocol by a general-purpose web server.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, conventional technologies have a challenge in that it is difficult to rewatch a video after viewing has ended, even if there was a section during playback where the viewer was not fully engaged. For example, while streaming using a general-purpose protocol makes it possible to watch videos in various environments, it also means that events that disrupt the viewer's concentration on watching the video are more likely to occur in the surrounding environment. If there is a section during playback where the viewer was not fully engaged, they may not have a sufficient understanding of the video's content.

[0006] One aspect of the present invention, in view of the above-mentioned technical problems, aims to present playback sections in which the viewer was not paying attention to the video. [Means for solving the problem]

[0007] To solve the above problems, a video playback device according to one aspect of the present invention comprises: a video playback unit that displays a video on a display device; a biometric information acquisition unit that acquires biometric information of a viewer watching a video; a concentration determination unit that determines whether or not the viewer is concentrating on the video based on the biometric information; a result recording unit that records playback sections obtained by dividing the video along the time axis in association with the determination results from the concentration determination unit; and a result display unit that, after the playback of the video is finished, displays on a display device a viewing result representing a playback section that indicates the viewer was not concentrating on the video. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to present playback sections in which the viewer was not paying attention to the video. [Brief explanation of the drawing]

[0009] [Figure 1] This is a conceptual diagram illustrating an example of a streaming distribution method. [Figure 2] This is a block diagram showing an example of the overall configuration of a video streaming system. [Figure 3] A block diagram showing an example of a computer hardware configuration. [Figure 4]This is a block diagram showing an example of the functional configuration of a video streaming system. [Figure 5] This flowchart shows an example of the processing steps for video playback. [Figure 6] This is a conceptual diagram showing an example of control information. [Figure 7] This is a conceptual diagram showing an example of a judgment result. [Figure 8] This is a conceptual diagram showing an example of viewing results. [Modes for carrying out the invention]

[0010] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0011] [Embodiment] <Overview> In recent years, there has been a shift in the streaming distribution of video content over the internet from methods using dedicated servers and proprietary protocols to methods using general-purpose web servers and the HTTP protocol. Streaming content using a general-purpose protocol makes it possible to deliver video content to a variety of devices.

[0012] The method of streaming content at the optimal quality according to the network communication status using the HTTP protocol is called adaptive streaming. While proprietary technologies from IT (Information Technology) vendors are prevalent in adaptive streaming, the international standard MPEG-DASH has been established with the aim of unifying streaming delivery methods. Both delivery technologies share a similar basic concept and stream video content as follows:

[0013] The web server prepares the segments that make up the video content and a manifest file. A segment is video data that has been encoded with one or more quality levels (e.g., screen size, bitrate, etc.) and divided into electronic files of a few seconds to tens of seconds each. The manifest file is an electronic file that describes the attribute information and location information of each segment.

[0014] The receiving terminal selects the optimal quality from the manifest file in a timely manner, taking into account the screen size of the display device and the network's receiving bandwidth, and sequentially receives segments encoded at that quality. The receiving terminal decodes the segments received from the web server, stitches them together into a single video content, and plays it on a display device or similar.

[0015] Figure 1 is a conceptual diagram illustrating an example of a streaming distribution method. As shown in Figure 1, first, the encoder encodes the video content in multiple quality levels and generates segments divided into units of several seconds to tens of seconds for each quality level. In the example in Figure 1, the encoder generates segments 1-4... encoded at low quality, segments 1'-4'... encoded at medium quality, and segments 1"-4"... encoded at high quality. The manifest file contains location information such as the URL (Uniform Resource Locator) for each segment, as well as attribute information such as the bitrate and image size for each quality level. The segments and manifest file generated by the encoder are supplied to the web server.

[0016] The receiving terminal receives a manifest file from a web server, and grasps the configuration of segments and the attribute information of each quality based on the manifest file. Next, the receiving terminal selects the quality in units of segments based on the screen size of the display device and the state of the reception bandwidth of the network, and receives the segments by means of the HTTP protocol. Subsequently, the receiving terminal connects the received segments in order according to the manifest file and plays them on the display device. In this way, the receiving terminal can receive a series of segments constituting one video content and continue playing.

[0017] The method for selecting quality in adaptive streaming is not defined in the standard. For example, a method for selecting quality based on the state of the reception bandwidth of the network, the remaining amount of the buffer provided in the receiving terminal, etc. is known (see, for example, Reference 1). Also, for example, a method is known in which every time the receiving terminal receives a segment, it determines the current transmission speed, obtains statistical information related to the expected future transmission speed as a function of the current transmission speed, and receives subsequent segments according to the target transmission speed derived from the statistical information (see, for example, Reference 2).

[0018] 〔Reference 1〕Hongzi Mao, Ravi Netravali, Mohammad Alizadeh, "Neural Adaptive Video Streaming with Pensieve", SIGCOMM, pp. 197-210, Aug. 2017. 〔Reference 2〕Japanese Patent Publication No. 2016-506644

[0019] In recent years, with the ability to view video content in a variety of environments, there are increasingly more distractions that interfere with video viewing. For example, receiving stimuli unrelated to video viewing (such as notifications from social networking services) or feeling unwell can prevent viewers from concentrating on the video, and even if they are watching the video content, they may not fully understand its content. In such cases, while the apparent viewership may increase, viewer comprehension is not enhanced, and the video cannot be said to be truly being watched.

[0020] One embodiment of the present invention aims to facilitate a substantive understanding of the video content being viewed in a video distribution system that delivers video content over a communication network. In this embodiment, playback sections in which the viewer is determined to be not concentrating on watching the video, based on biometric information such as brain waves or facial images, are recorded and presented after the video viewing is finished. This allows the viewer to identify the playback sections in which they were not concentrating on watching the video, and by rewatching those sections, they can improve their understanding of the video content.

[0021] Known biometric information that can estimate concentration levels include, for example, electroencephalograms (EEGs) and facial images. Concentration levels can be estimated, for example, from the number of blinks and facial expressions (see, for example, Reference 3). Also, for example, the object of concentration can be estimated by detecting gaze from facial images. Emotions can be estimated, for example, from EEGs and facial expressions (see, for example, Reference 4).

[0022] [Reference 3] Japanese Patent Publication No. 2021-170315 [Reference 4] Kiyohisa Natsume, "Are there any brainwaves that serve as indicators of 'boredom'?", Research Results Report of the Grant-in-Aid for Scientific Research, Project Number 22650205, 2011.

[0023] Traditionally, measuring brainwaves (EEGs) has been problematic due to the significant stress it causes on viewers during the measurement process. For example, the bulkiness of the measuring equipment, the restrictions on movement during measurement, and the long duration of the measurement process are all sources of stress. In recent years, however, advancements such as the development of earphone-type EEG devices have made it possible to measure EEGs without placing a psychological burden on viewers (see, for example, reference 5).

[0024] [Reference 5] Toppan Printing Co., Ltd., "Toppan Printing Begins Sales of Earphone-Type EEG Device "b-tone", [online], [Retrieved August 3, 2022], Internet<URL: https: / / www.toppan.co.jp / news / 2021 / 09 / newsrelease210907_1.html>

[0025] <Overall configuration of the video distribution system> The overall configuration of the video distribution system in this embodiment will be described with reference to Figure 2. Figure 2 is a block diagram showing an example of the overall configuration of the video distribution system in this embodiment.

[0026] As shown in Figure 2, the video distribution system 1 in this embodiment includes a video distribution server 10 and one or more video playback devices 20. The video distribution server 10 and the video playback devices 20 are connected via a communication network N1 such as a LAN (Local Area Network) or the Internet, enabling data communication.

[0027] The video distribution server 10 is an information processing device such as a personal computer, workstation, or server that distributes video content via the communication network N1. The video distribution server 10 transmits video content to the video playback device 20 in response to a request from the video playback device 20.

[0028] The video playback device 20 is an electronic device such as a television receiver, set-top box, personal computer, or smartphone on which viewers watch video content. The video playback device 20 requests video content from the video distribution server 10 in response to the viewer's operation and plays the video content received from the video distribution server 10 on the display device.

[0029] Note that the overall configuration of the video distribution system 1 shown in Figure 2 is just one example, and various system configurations are possible depending on the application and purpose. For example, the video distribution server 10 may be implemented using multiple computers, or it may be implemented as a cloud computing service.

[0030] <Hardware configuration of the video streaming system> The hardware configuration of the video distribution system 1 in this embodiment will be described with reference to Figure 3.

[0031] Computer Hardware Configuration In this embodiment, the video distribution server 10 and the video playback device 20 are implemented, for example, by a computer. Figure 3 is a block diagram showing an example of the computer hardware configuration in this embodiment.

[0032] As shown in Figure 3, the computer 500 in this embodiment includes a CPU (Central Processing Unit) 501, ROM (Read Only Memory) 502, RAM (Random Access Memory) 503, HDD (Hard Disk Drive) 504, input device 505, display device 506, communication interface 507, and external interface 508. The CPU 501, ROM 502, and RAM 503 form a so-called computer. Each piece of hardware in the computer 500 is interconnected via a bus line 509. The input device 505 and display device 506 may also be used by connecting them to the external interface 508.

[0033] The CPU 501 is a processing unit that reads programs and data from storage devices such as the ROM 502 or HDD 504 onto the RAM 503 and executes processing, thereby realizing the overall control and functions of the computer 500.

[0034] ROM502 is an example of non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. ROM502 functions as the main memory, storing various programs and data necessary for the CPU501 to execute the programs installed on HDD504. Specifically, ROM502 stores boot programs such as BIOS (Basic Input / Output System) and EFI (Extensible Firmware Interface) that are executed when the computer 500 starts up, as well as OS (Operating System) settings, network settings, and other data.

[0035] RAM503 is an example of volatile semiconductor memory (storage device) whose programs and data are erased when the power is turned off. RAM503 includes, for example, DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory). RAM503 provides a working area that is expanded when various programs installed on HDD504 are executed by CPU501.

[0036] HDD504 is an example of a non-volatile storage device that stores programs and data. The programs and data stored in HDD504 include the operating system (OS), which is the basic software that controls the entire computer 500, and applications that provide various functions on the OS. Note that computer 500 may use a storage device that uses flash memory as its storage medium (e.g., SSD: Solid State Drive) instead of HDD504.

[0037] The input device 505 includes a touch panel used by the user to input various signals, operation keys and buttons, a keyboard and mouse, and a microphone for inputting sound data such as voice.

[0038] The display device 506 consists of a display such as a liquid crystal or organic EL (Electro-Luminescence) that displays a screen, and a speaker that outputs sound data such as audio.

[0039] Communication I / F 507 is an interface that connects to a communication network and allows computer 500 to perform data communication.

[0040] External I / F 508 is an interface for external devices. Examples of external devices include the drive device 510.

[0041] The drive device 510 is a device for setting the recording medium 511. The recording medium 511 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The recording medium 511 may also include semiconductor memory that records information electrically, such as ROMs and flash memory. This allows the computer 500 to read and / or write to the recording medium 511 via the external I / F 508.

[0042] The various programs to be installed on the HDD 504 are installed, for example, when the distributed recording medium 511 is set in a drive device 510 connected to an external I / F 508, and the various programs recorded on the recording medium 511 are read by the drive device 510. Alternatively, the various programs to be installed on the HDD 504 may be downloaded via the communication I / F 507 from a network other than the communication network and installed that way.

[0043] <Functional Configuration of Video Streaming System> The functional configuration of the video distribution system in this embodiment will be described with reference to Figure 4. Figure 4 is a block diagram showing an example of the functional configuration of the video distribution system 1 in this embodiment.

[0044] ≪Functional Configuration of Video Streaming Server≫ As shown in Figure 4, the video distribution server 10 in this embodiment includes a video distribution unit 101, a control information distribution unit 102, a video storage unit 111, and a control information storage unit 112.

[0045] The video distribution unit 101 and the control information distribution unit 102 are implemented, for example, by a process in which a program loaded onto RAM 503 from HDD 504 shown in Figure 3 is executed by CPU 501. The video storage unit 111 and the control information storage unit 112 are implemented, for example, using HDD 504 shown in Figure 3.

[0046] The video storage unit 111 stores video content to be distributed to the video playback device 20. In this embodiment, the video content is a collection of segments obtained by encoding a single video data for a predetermined number of quality levels and dividing it into multiple playback sections along the time axis.

[0047] The control information storage unit 112 stores control information used to play video content stored in the video storage unit 111. In this embodiment, the control information is information that associates the playback time, segment position information, and expected emotion value. An example of the control information in this embodiment is a manifest file.

[0048] Segment location information is, for example, a URL indicating the location of the electronic file corresponding to the segment. Expected emotions represent the emotions that are expected to arise when viewing the segment. Expected emotions can be pre-set by the video content creator or other relevant parties.

[0049] The control information storage unit 112 also stores a distributed list containing information about the segments distributed to the video playback devices 20. A distributed list is stored for each video playback device 20.

[0050] The video distribution unit 101 transmits a segment stored in the video storage unit 111 to the video playback device 20 in response to a segment acquisition request received from the video playback device 20. The video distribution unit 101 records information about the segment transmitted to the video playback device 20 in the distributed list stored in the control information storage unit 112.

[0051] The control information distribution unit 102 transmits the control information stored in the control information storage unit 112 to the video playback device 20 in response to a control information acquisition request received from the video playback device 20.

[0052] ≪Functional Configuration of Video Playback Devices≫ As shown in Figure 4, the video playback device 20 in this embodiment includes a control information acquisition unit 201, a video acquisition unit 202, a buffer unit 203, a video playback unit 204, a biometric information acquisition unit 205, a centralized determination unit 206, a result recording unit 207, a result display unit 208, a control information storage unit 211, and a determination result storage unit 212.

[0053] The control information acquisition unit 201, video acquisition unit 202, buffer unit 203, video playback unit 204, biometric information acquisition unit 205, centralized judgment unit 206, result recording unit 207, and result display unit 208 are implemented, for example, by a process in which a program loaded from the HDD 504 shown in Figure 3 onto the RAM 503 is executed by the CPU 501. The control information storage unit 211 and the judgment result storage unit 212 are implemented, for example, using the HDD 504 shown in Figure 3.

[0054] The control information storage unit 211 stores control information received from the video distribution server 10. Each time a video content is played, control information related to that video content is stored in the control information storage unit 211.

[0055] The judgment result storage unit 212 stores a judgment result list that records the judgment results determined by the centralized judgment unit 206. The judgment result list is a list that records the association between segments and judgment results, and is generated each time video content is played.

[0056] The control information acquisition unit 201 sends a control information acquisition request to the video distribution server 10 in response to a video playback operation by the viewer. The control information acquisition unit 201 stores the control information received from the video distribution server 10 in the control information storage unit 211.

[0057] The video acquisition unit 202 sends a segment acquisition request to the video distribution server 10 according to the control information stored in the control information storage unit 211. The video acquisition unit 202 stores the segment received from the video distribution server 10 in the buffer unit 203.

[0058] The buffer unit 203 stores segments received from the video distribution server 10, associating them with the playback start time and playback duration (segment length). The buffer unit 203 reads the segments according to the playback start time and outputs them sequentially to the video playback unit 204.

[0059] The video playback unit 204 obtains the next segment to be played from the buffer unit 203. The video playback unit 204 decodes the obtained segment according to the media format (video, audio, or text, etc.) according to the control information stored in the control information storage unit 211. The video playback unit 204 plays the decoded segment on the display device 506.

[0060] The biometric information acquisition unit 205 acquires the viewer's biometric information. In this embodiment, the biometric information is the viewer's brainwaves or facial image, etc.

[0061] The concentration determination unit 206 estimates the viewer's level of concentration or emotion based on the biometric information acquired by the biometric information acquisition unit 205. Based on the estimated level of concentration or emotion, the concentration determination unit 206 determines whether or not the viewer is concentrating on the video.

[0062] The result recording unit 207 records the judgment result determined by the centralized judgment unit 206 in the judgment result list stored in the judgment result storage unit 212, associating it with the segment that was being played back at the time of judgment. The result recording unit 207 associates a segment with a judgment result, for example, by pairing identification information that identifies the segment with the judgment result.

[0063] The result display unit 208 generates viewing results based on the judgment result list stored in the judgment result storage unit 212 after the video playback unit 204 has finished playing the video content. The viewing results are information indicating the playback sections in which the viewer was not paying attention to the video. The result display unit 208 stores the generated viewing results in the judgment result storage unit 212 and displays them on the display device 506.

[0064] <Processing procedure for video distribution system> The processing procedure of the video distribution method executed by the video distribution system 1 in this embodiment will be described with reference to Figure 5. Figure 5 is a flowchart showing an example of the processing procedure of the video distribution method in this embodiment.

[0065] In step S1, the video playback unit 204 of the video playback device 20 receives a video playback operation from the viewer. The video playback operation is performed by specifying the video content to be played. For example, the video playback operation is an operation in which the video playback device 20 selects the desired video content from the playable video content displayed on the display device 506, such as a menu screen.

[0066] In step S2, the control information acquisition unit 201 of the video playback device 20 sends a control information acquisition request to the video distribution server 10. The control information acquisition request includes identification information (hereinafter also referred to as "video ID") that identifies the video content specified in the video playback operation.

[0067] In step S3, the control information distribution unit 102 of the video distribution server 10 receives a control information acquisition request from the video playback device 20. Next, the control information distribution unit 102 reads control information from the control information storage unit 112 based on the video ID included in the control information acquisition request. Subsequently, the control information distribution unit 102 transmits the read control information to the video playback device 20.

[0068] In step S4, the control information acquisition unit 201 of the video playback device 20 receives control information from the video distribution server 10. Next, the control information acquisition unit 201 stores the received control information in the control information storage unit 211. Subsequently, the video playback unit 204 starts playing the video content.

[0069] Figure 6 is a conceptual diagram showing an example of control information in this embodiment. As shown in Figure 6, the control information in this embodiment is information associated with time, segment URL, and perception ID for each quality level. Quality refers to the quality in which the video content is encoded. For example, if the video content includes segments encoded at high quality, medium quality, and low quality, control information exists for each of the high quality, medium quality, and low quality levels.

[0070] The time is the start time of playback for that segment. The start time of playback is the elapsed time from the time playback of the video content began. The segment URL is the location information of the video data corresponding to that segment.

[0071] The perceptual ID represents the expected value of the emotion. In this embodiment, the perceptual ID will be set according to Ekman's theory. Ekman's theory defines human emotions as six basic emotions. The perceptual ID may be, for example, one of the following: (1) joy, (2) sadness, (3) disgust, (4) anger, (5) fear, or (6) surprise. The perceptual ID can be set in advance by the video content creator or the like.

[0072] Let's return to Figure 5 for explanation. In step S5, the video acquisition unit 202 of the video playback device 20 determines the quality according to the screen size of the display device 506 or the state of the communication network N1. Next, the video acquisition unit 202 sends a segment acquisition request to the video distribution server 10 according to the control information stored in the control information storage unit 211. The segment acquisition request includes a segment URL corresponding to the determined quality.

[0073] In step S6, the video distribution unit 101 of the video distribution server 10 receives a segment acquisition request from the video playback device 20. Next, the video distribution unit 101 reads a segment from the video storage unit 111 based on the segment URL included in the segment acquisition request. Subsequently, the video distribution unit 101 transmits the read segment to the video playback device 20. At the same time, the video distribution unit 101 records information about the read segment in the distributed list stored in the control information storage unit 112.

[0074] In step S7, the video acquisition unit 202 of the video playback device 20 receives a segment from the video distribution server 10. Next, the video acquisition unit 202 stores the received segment in the buffer unit 203, associating it with the playback start time and playback duration.

[0075] The video playback unit 204 retrieves a segment from the buffer unit 203. The request includes the playback start time of the next segment. The playback start time of the next segment is the playback start time of the currently playing segment plus the playback duration. The buffer unit 203 reads the segment corresponding to the requested playback start time and outputs it to the video playback unit 204.

[0076] The video playback unit 204 decodes the segment output from the buffer unit 203 according to the media format, in accordance with the control information stored in the control information storage unit 211. The video playback unit 204 outputs the decoded segment from the display device 506 according to the media format.

[0077] In step S8, the biometric information acquisition unit 205 of the video playback device 20 acquires the viewer's biometric information. Next, the biometric information acquisition unit 205 sends the acquired viewer's biometric information to the centralized determination unit 206.

[0078] When acquiring brain waves as biological information, the biometric information acquisition unit 205 connects to a brainwave measuring device attached to the viewer via a wired or wireless interface and receives signals output by the brainwave measuring device. When acquiring a facial image as biological information, the biometric information acquisition unit 205 controls a camera or other shooting unit connected to the video playback device 20 to capture a facial image of the viewer.

[0079] In step S9, the concentration determination unit 206 of the video playback device 20 receives the viewer's biometric information from the biometric information acquisition unit 205. Next, the concentration determination unit 206 estimates the viewer's level of concentration or emotion based on the viewer's biometric information. Subsequently, the concentration determination unit 206 determines whether or not the viewer is concentrating on the video based on the estimated level of concentration or emotion. Then, the concentration determination unit 206 sends the determination result to the result recording unit 207.

[0080] The central judgment unit 206 determines whether a negative change in perception has occurred when it has estimated the viewer's emotion. A negative change in perception means that if the previously estimated emotion was positive or neutral, the currently estimated emotion has changed to a negative emotion. Positive emotions include, for example, (1) joy and (6) surprise. Negative emotions include, for example, (2) sadness, (3) disgust, (4) anger, and (5) fear.

[0081] If a negative change in perception occurs, the concentration determination unit 206 compares the emotion estimated this time (hereinafter also referred to as the "estimated emotion") with the expected emotion included in the control information. If the estimated emotion differs from the expected emotion, the concentration determination unit 206 determines that the viewer is not concentrating on the video.

[0082] The concentration determination unit 206 determines that the viewer is concentrating on the video in all other cases. That is, if no negative change in perception has occurred, or if the estimated value of emotion is the same as the expected value of emotion, the concentration determination unit 206 determines that the viewer is concentrating on the video.

[0083] When the concentration determination unit 206 estimates the viewer's level of concentration, it compares the estimated level of concentration (hereinafter also referred to as the "estimated level of concentration") with a predetermined threshold. If the estimated level of concentration is equal to or greater than the threshold, the concentration determination unit 206 determines that the viewer is concentrating on the video. On the other hand, if the estimated level of concentration is less than the threshold, the concentration determination unit 206 determines that the viewer is not concentrating on the video.

[0084] In step S10, the result recording unit 207 of the video playback device 20 receives the judgment result from the central judgment unit 206. The result recording unit 207 combines the received judgment result with identification information that identifies the segment being played at the time of judgment and records it in the judgment result list stored in the judgment result storage unit 212.

[0085] Figure 7 is a conceptual diagram showing an example of a judgment result list in this embodiment. As shown in Figure 7, the judgment result list in this embodiment is information associated with time, segment URL, and a decentralized flag.

[0086] The time is the start time of playback for that segment. The segment URL is the location information of the video data obtained in the segment corresponding to each time.

[0087] The non-concentration flag is a truth value that is true if the viewer is not concentrating on the video, and false if the viewer is concentrating on the video. The non-concentration flag is set based on the determination result made by the concentration determination unit 206 in that segment.

[0088] Let's return to Figure 5 for explanation. In step S11, the video playback unit 204 of the video playback device 20 determines whether or not playback of the video content has finished. The video playback unit 204 determines that playback has finished if, for example, the last segment of the video content has been played, or if the user has performed a playback termination operation. On the other hand, the video playback unit 204 determines that playback has not finished in other cases (for example, if there is a segment to be played next and the user has not performed a playback termination operation).

[0089] If the video playback unit 204 determines that playback has finished (YES), it proceeds to step S12. On the other hand, if the video playback unit 204 determines that playback has not finished (NO), it returns to step S5.

[0090] In step S12, the result display unit 208 of the video playback device 20 generates viewing results based on the judgment result list stored in the judgment result storage unit 212. Next, the result display unit 208 stores the generated viewing results in the judgment result storage unit 212. Subsequently, the result display unit 208 displays the generated viewing results on the display device 506.

[0091] Figure 8 is a conceptual diagram showing an example of viewing results in this embodiment. As shown in Figure 8, the viewing results in this embodiment are information associated with the playback start time, playback duration, and segment URL. The viewing results are the result of extracting segments from the judgment result list shown in Figure 7 in which the non-concentration flag was set to true (i.e., the viewer was not concentrating on the video).

[0092] The playback start time is the playback start time of the segment for which the "non-centralized" flag is set to true. If there are multiple consecutive segments with the "non-centralized" flag set, the playback start time of the segment with the earliest playback start time among those segments will be displayed.

[0093] The playback duration is the playback duration of the segments for which the "decentralized" flag is set to true. If there are multiple consecutive segments with the "decentralized" flag set, the sum of the playback durations of those segments is displayed.

[0094] The segment URL is the segment URL of the segment for which the decentralization flag is set to true. If there are multiple consecutive segments with the decentralization flag set, the segment URL of the segment with the earliest playback start time among those segments will be displayed.

[0095] In the example in Figure 8, when there are multiple consecutive segments with the "non-centralized" flag set, the viewing results are shown as a continuous playback section. However, the display format of the viewing results is not limited to this. For example, the viewing results may be displayed by listing information about each segment with the "non-centralized" flag set individually. Also, for example, if the video content contains multiple chapters, the viewing results may be displayed by listing information about the chapters containing segments with the "non-centralized" flag set.

[0096] The results display unit 208 may display the viewing results at any time after the playback of the video content has finished. For example, the results display unit 208 may display the viewing results immediately after the playback of the video content has finished. Alternatively, for example, the results display unit 208 may display the previous viewing results for the video content when the same video content is played again after the playback of the video content has finished once.

[0097] In step S13, the result display unit 208 of the video playback device 20 determines whether or not to replay the video content. The result display unit 208 determines to replay the content if the user performs a replay operation. A replay operation is, for example, an operation to select a desired playback section from the viewing results. On the other hand, the result display unit 208 determines not to replay the content if the user performs an operation other than a replay operation. An operation other than a replay operation is, for example, a viewing end operation. A viewing end operation is, for example, an operation to close the viewing results.

[0098] If the result display unit 208 determines that the user will rewatch the video (YES), it identifies the playback start time of the selected playback section and returns to step S5. On the other hand, if the result display unit 208 determines that the user will not rewatch the video (NO), it terminates the video playback method process.

[0099] When processing returns to step S5, the video acquisition unit 202 sends a segment acquisition request to the video distribution server 10, which includes a segment URL corresponding to the identified playback start time, according to the control information stored in the control information storage unit 211. As a result, the video playback device 20 starts playback of the playback section selected by the user.

[0100] As shown in Figure 8, when multiple consecutive segments are aggregated and displayed, the video playback device 20 plays the multiple segments included in the playback section identified by the replay operation in succession. After playback of the identified playback section is completed, the playback of the video content may be terminated, or playback of the video content may be continued.

[0101] <Effects of the Embodiment> The video playback device in this embodiment determines whether the viewer is concentrating on the video based on biometric information obtained from the viewer, and displays viewing results after the video viewing is finished, indicating the playback sections in which the viewer was not concentrating on the video. Therefore, the video playback device in this embodiment can present playback sections in which the viewer was not concentrating on the video.

[0102] The video playback device in this embodiment determines whether a viewer is concentrating on a video based on their level of concentration or emotion, which is estimated based on brainwaves or facial images acquired from the viewer. Therefore, according to the video playback device in this embodiment, it is possible to determine whether a viewer is concentrating on a video based on objective indicators.

[0103] In this embodiment, the video playback device determines that a viewer is not concentrating on the video if the emotion estimated from the viewer's biometric information differs from the expected emotion set for each playback section of the video. In other words, even if the emotion estimated from the viewer's biometric information is negative, if it is the emotion intended by the content of the video, the device determines that the viewer is concentrating on the video. Therefore, according to this embodiment, it is possible to determine with high accuracy whether or not a viewer is concentrating on the video, depending on the content of the video.

[0104] In this embodiment, the video playback device plays a selected playback section based on the viewer's viewing experience. Therefore, with this embodiment, viewers can focus on rewatching sections they were not paying attention to when they are in a position to concentrate on the video. In other words, this embodiment facilitates understanding of the video content.

[0105] [Differentiation] In the above embodiment, an example was described in which the video playback device 20 plays a video streamed from the video distribution server 10. The video playback device 20 may also play video data recorded on a recording medium. In this case, the recording medium may record control information associated with the playback time and the expected value of emotion. The control information may be obtained from an external storage device before playing the video data.

[0106] The video playback device in this modified example can play video data recorded on a recording medium, in addition to video content streamed via a communication network. Therefore, the video playback device in this modified example can present playback sections where the viewer was not fully engaged with the video, regardless of the video content distribution medium.

[0107] [supplement] Each of the embodiments described above can be implemented by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and conventional circuit modules designed to execute each of the functions described above.

[0108] Although embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications or changes are possible within the scope of the gist of the present invention as described in the claims. [Explanation of Symbols]

[0109] 1. Video distribution system 10 Video streaming servers 101 Video Streaming Department 102 Control Information Distribution Unit 111 Video Memory Unit 112 Control Information Storage Unit 20 Video playback device 201 Control Information Acquisition Unit 202 Video Acquisition Section 203 Buffer section 204 Video Playback Section 205 Biological Information Acquisition Unit 206 Centralized judgment section 207 Results Recording Section 208 Result display section 211 Control Information Storage Unit 212 Judgment result storage unit

Claims

1. A video playback unit that displays the video on a display device, A biometric information acquisition unit acquires biometric information of viewers who watch the aforementioned video, A concentration determination unit that determines whether the viewer is concentrating on the video based on the aforementioned biometric information, A result recording unit records the playback intervals obtained by dividing the aforementioned video along the time axis and the judgment results from the centralized judgment unit in association with each other. After the playback of the aforementioned video has finished, the display unit displays the viewing results on the display device, which represent the playback section indicating that the judgment result is not concentrated in the aforementioned video. Equipped with, The system further includes a storage unit that stores control information in which an expected value of emotion is set for each of the aforementioned playback intervals. The concentration determination unit determines that the viewer is not concentrating on the video when the emotion estimated from the biological information differs from the expected value set in the control information. Video playback device.

2. A video playback device according to claim 1, The biometric information acquisition unit acquires the viewer's brainwaves or the viewer's facial image as biometric information. The centralized determination unit estimates the emotion based on the electroencephalogram or the facial image. Video playback device.

3. A video playback device according to claim 1, The video playback unit displays the video corresponding to the playback section selected by the viewer from the viewing results on the display device. Video playback device.

4. A video playback device according to claim 3, The results display unit, when it finds that there are viewing results for the video, displays the viewing results before starting playback of the video. Video playback device.

5. A video playback device according to claim 1, The aforementioned video playback unit plays video content streamed via a communication network, or video data recorded on a recording medium. Video playback device.

6. A program for causing a computer to function as a video playback device according to any one of claims 1 to 5.

Images