Playback device and method, system, program, and storage medium

Abstract

To view a highlight scene in edited moving images, while easily allowing viewing of part of the same moving images related to the highlight scene.SOLUTION: A playback device has communication means that communicates with an external device editing and distributing moving images, display means, and control means. The control means performs control to receive, through the communication means, a list indicating one or more partial moving images extracted from the moving images based on predetermined characteristics, play back the partial moving images on the display means, receive selected partial images in the displayed list through the communication means, and play back the partial moving images. At the completion of playback of the selected partial moving images, the control means displays, on the display means, options for inquiring whether to view related moving images, of the moving images, which are related to the partial moving images and moving images of different parts, and when viewing the related images is selected, receives the related moving images through the communication means and plays back the related moving images on the display means.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a playback device and method, a system, a program, and a storage medium, and more particularly to playback control of playback content. [Background technology]

[0002] In recent years, there has been a system that uses AI (Artificial Intelligence) to analyze video captured by a camera and automatically extract and distribute highlight scenes from the video based on the results. For example, when this system is used with filmed sports footage such as soccer, it can identify specific play events such as shots and dribbles, or specific players, and then extract and distribute scenes that show the identified play events or players, edited, and distributed.

[0003] In the system described above, a cloud server detects specific events, people, etc. that appear in a captured video. Specifically, the server uses a pre-trained inference model stored in the server and performs editing, such as cutting out parts of the video based on the inference results. The edited scenes are distributed as highlight scenes from the captured video, and users can view them on information terminals such as tablets and smartphones.

[0004] When viewing, a user operates an application such as a web browser on an information terminal such as a tablet terminal to request a video of highlight scenes based on a communication protocol such as HTTP (Hypertext Transfer Protocol).

[0005] In addition, a method of presenting multiple thumbnail images to a user to allow the user to select a desired highlight scene video on an information terminal may be used. In this case, when the user selects a thumbnail image on the information terminal to request the desired highlight scene video from a server, streaming playback of the highlight scene video associated with the selected thumbnail image begins.

[0006] There are various prior art methods for selecting a thumbnail image and viewing the associated video. Patent Document 1 discloses a method for selecting a thumbnail image to play edited and generated content stored in advance on a cloud server from the beginning. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-32906 Summary of the Invention [Problem to be solved by the invention]

[0008] However, when playing back a video segment extracted based on the results of AI inference as described above, it is possible that the segment may be extracted in a range that is not suitable as a highlight scene, and the user may not be able to play the scene they expect. For example, if the inference accuracy of the trained model is insufficient, when extracting a soccer shooting scene, the video segment extracted based on the results of AI inference may end just before the scene indicating whether or not a goal was scored. If this happens, the decisive scene will not be viewable in the edited video segment.

[0009] In Patent Document 1, by selecting a thumbnail image, it is possible to request content stored in advance on a server and start playing the content from the beginning. However, as shown in the above-mentioned problem, when a video of an extracted highlight scene stops at an undesirable timing while viewing the video, the patent document 1 does not disclose a method for viewing the rest of the extracted highlight scene or a method for prompting or requesting viewing of the rest.

[0010] The present invention has been made in consideration of the above problems, and aims to make it possible, when viewing a highlight scene of an edited video, to easily view a part of the same video related to the highlight scene. [Means for solving the problem]

[0011] In order to achieve the above-mentioned object, the playback device of the present invention has a communication means for communicating with an external device that edits and distributes videos, a display means, and a control means for controlling the communication means and the display means, wherein the control means receives a list showing one or more partial videos extracted from a video based on predetermined characteristics from the external device via the communication means and displays it on the display means, receives a selected partial video from the displayed list from the external device via the communication means and plays it on the display means, and when playback of the selected partial video is completed, displays on the display means an option to inquire whether or not to watch a related video, which is a video of a part of the video that is related to the partial video but different from the partial video, and when viewing the related video is selected, controls the related video to be received from the external device via the communication means and played on the display means. [Effects of the Invention]

[0012] According to the present invention, when viewing a highlight scene of an edited video, it is possible to easily view a portion of the same video related to the highlight scene. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram showing the main configuration of a tablet terminal according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the main configuration of a server according to the embodiment. [Figure 3] 10 is a flowchart showing the operation of a server in an embodiment for generating a playlist of highlight scenes. [Figure 4]FIG. 10 is a diagram showing an example of table data indicating information on generated highlight scenes in the embodiment. [Figure 5] 5 is a flowchart showing the operation of the tablet terminal according to the first embodiment. [Figure 6] 6 is a flowchart showing a data distribution operation of a server in the first embodiment. [Figure 7] FIG. 4 is a diagram showing an example of a screen presenting thumbnail images according to the first embodiment. [Figure 8] FIG. 3 is a diagram showing an example of a user interface for displaying an inquiry about whether to continue viewing in the first embodiment. [Figure 9] 10 is a flowchart showing the operation of a tablet terminal according to a second embodiment. [Figure 10] 10 is a flowchart showing a data distribution operation of a server in the second embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a user interface for inquiring about whether to continue viewing and presenting other highlight scenes in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0015] First Embodiment In this embodiment, a tablet terminal is used as the playback device and a server is used as the distribution device, and the two are connected to each other. In the following explanation, the main configurations and basic operations of the tablet terminal and the server are described, followed by a description of the characteristic collaborative operation of the present invention between the tablet terminal and the server.

[0016] ■ Explanation of the main components of the tablet terminal 100 FIG. 1 is a block diagram showing the main functional configuration of a tablet terminal 100 according to this embodiment. 1, a control unit 101 controls each component of the tablet terminal 100. The control unit 101 is connected to each component of the tablet terminal 100, and issues control instructions to each component and reads and writes data from and to recording media such as RAM 103 and ROM 104.

[0017] The communication unit 102 is connected to an intranet and the Internet, and transmits and receives various information data files such as video data, image data, video files, and other command signals. The RAM 103 is used for expanding the programs stored in the ROM 104, as a frame memory for display images, and for temporarily storing other information. The ROM 104 is an internal memory that stores various data such as program data, image data, and user setting data, and is configured from a semiconductor memory or the like.

[0018] The operation unit 105 is configured with various buttons, a touch panel, etc., and receives various instructions from the user, such as a power-on instruction and an instruction to transition to a sleep state. The display unit 106 is used to display video data, various information, etc. A light-transmitting touch panel is provided on the front surface, and a user can input instructions to the tablet terminal 100 by operating the touch panel in accordance with the display on the display unit 106. The display control unit 107 drives the display unit 106 to display image data read by the control unit 101 from the ROM 104 or the like and moving images acquired via the communication unit 102.

[0019] The decoder 108 decompresses the video data read from the ROM 104 or the video data acquired via the communication unit 102 based on the compression method of the data. The speaker 109 converts the supplied audio signal into vibrations, which are then emitted through a diaphragm.

[0020] The battery 110 supplies power to the power supply unit 111. The power input unit 112 receives AC power from the outside, rectifies it to a predetermined voltage, and supplies it to the power supply unit 111. The power supply unit 111 controls the power supply to each component within the tablet terminal 100.

[0021] ■ Explanation of the main components of the server 200 FIG. 2 is a block diagram showing the main configuration of the server 200 in this embodiment. 2, a control unit 201 controls each component of the server 200. The control unit 201 is connected to each component of the server 200, and controls access to each component, such as control instructions and data exchange.

[0022] The communication unit 202 is connected to an intranet and the Internet, and transmits and receives various information data files such as video data, image data, video files, and other command signals. The RAM 203 is used for expanding programs stored in the HDD 204, as a frame memory for display images, and for temporarily storing other information. The HDD 204 is a disk-type recording device that stores various data such as program data, image data, and user setting data.

[0023] The video processing unit 205 encodes the video data acquired via the communication unit 202 into a predetermined format. The video processing unit 205 also includes an inference unit 255 that extracts features of each frame from the video data, and extracts the features included in the video data using a trained model stored in advance within the video processing unit 205.

[0024] The power supply unit 206 controls the power supply to each component in the server 200. The power input unit 207 receives AC power from the outside, rectifies it to a predetermined voltage, and supplies it to the power supply unit 206.

[0025] ■Explanation of the basic operation of the tablet terminal 100 Next, the basic operation of the tablet terminal 100 in this embodiment will be described. The operation of the tablet terminal 100 is basically realized by the control unit 101 controlling each component based on a program stored in the ROM 104. The operation of the tablet terminal 100 starts when the user issues a power-on instruction via the operation unit 105, and continues thereafter through transition to a sleep state via the operation unit 105, recovery from the sleep state, and until the power is turned off in response to a power-off instruction.

[0026] When a user issues an instruction to power on the tablet terminal 100 via the operation unit 105, the control unit 101 supplies power to each unit of the tablet terminal 100 via the power supply unit 111 using power supplied from the battery 110 and the power input unit 112. The control unit 101 then reads a program from the ROM 104 and instructs each unit to enter an operable state based on the program. Next, the control unit 101 reads image data previously stored in the ROM 104 and instructs the display control unit 107 to display the image data on the display unit 106. The image data displayed here is image data corresponding to a so-called desktop screen that is displayed by the tablet terminal 100 in a steady state. Upon receiving the instruction, the display control unit 107 controls the display unit 106 based on the image data, and the tablet terminal 100 transitions to a steady state in which an image after startup is displayed on the display unit 106.

[0027] After that, if there is no user operation for a certain period of time, an interrupt occurs to notify the control unit 101 of a timeout. In response to this, the control unit 101 instructs the display control unit 107 to stop display in order to transition to a sleep state. Upon receiving the instruction from the control unit 101, the display control unit 107 controls the display unit 106 to stop display, and transitions to the sleep state. Furthermore, in the sleep state, if an operation is performed on the operation unit 105 including a touch panel, or if a predetermined vibration or the like is detected by a gyro sensor (not shown), the device returns to the steady state described above.

[0028] Next, a basic operation will be described in which a user operates the tablet terminal 100 to start a web browser application, and the tablet terminal 100 receives, displays, and plays various information, still images, and video data.

[0029] The user can activate a web browser application stored in ROM 104 and perform operations after activation by operating the touch panel according to the display on display unit 106. For example, the user can activate the web browser application by touching an icon representing the web browser application displayed on the desktop screen. Control unit 101 receives a touch operation on the touch panel and reads out a web browser application program from ROM 104 based on the position information and operation type of the user's operation and the display content on display unit 106.

[0030] Next, the control unit 101 initializes each unit related to the operation of the application, and transmits a request for HTML (Hyper Text Markup Language) data, which is data required for display, via the communication unit 102 based on HTTP.

[0031] When the control unit 101 receives HTML data via the communication unit 102, it writes the HTML data to the RAM 103, calculates the layout of the included document data and image data, and outputs an instruction to the display control unit 107 to display the results. The display control unit 107 controls the display unit 106 based on the instruction from the control unit 101. The analysis of the document data and the layout of the image data included in the HTML data are performed based on a so-called HTML rendering program, which is stored in advance in the ROM 104.

[0032] By carrying out the processing up to this point, it is possible to present to the user the video to be played by receiving and displaying a thumbnail image showing the video to be played, for example.

[0033] Thereafter, when a thumbnail image is selected by the user by operating the touch panel, the control unit 101 transmits a request for video data corresponding to the selected thumbnail image via the communication unit 102. Then, when the requested video data is received via the communication unit 102, the control unit 101 instructs the decoder 108 to decode the received video data in order to convert it into playable data. When the decoder 108 writes the decoded data to the RAM 103, the control unit 101 instructs the display control unit 107 to display the decoded data on the display unit 106. As a result, the video is played on the display unit 106.

[0034] ■ Explanation of the operation of generating a playlist of highlight scenes in the server 200 Next, we will explain the basic operation for generating a playlist of highlight scenes by the server 200. The operation of the server 200 is basically realized by the control unit 201 controlling each component based on a program stored in the HDD 204.

[0035] The operation of the server 200 starts when the user issues a power-on command via the operation unit (not shown), and continues thereafter, transitioning to and returning from a sleep state via the operation unit, until the power is turned off by a power-off command received via the operation unit or the communication unit 202.

[0036] When a user issues an instruction to power on server 200 via the operation unit, control unit 201 supplies power to each unit of server 200 via power supply unit 206 based on the power supplied from power input unit 207. Then, control unit 201 reads a program from HDD 204 and instructs each unit to enter an operable state based on the program, thereby causing server 200 to transition to a steady state.

[0037] Next, the encoding operation for distributing video data uploaded by a user and the operation for linking features extracted from the video data by inference using a trained model to the video data in server 200 will be described with reference to the flowchart in Figure 3. Note that these operations are provided as a service that automatically encodes uploaded videos for distribution, and the series of operations are automatically executed according to a predetermined program.

[0038] When the control unit 201 of the server 200 receives video data uploaded by a user via the communication unit 202, the control unit 201 instructs the video processing unit 205 to encode the received video data in S301. In this embodiment, the encoding by the video processing unit 205 involves packaging the video data for distribution based on the HTTP Live Streaming (HLS) format in the distribution operation described below. Specifically, the video processing unit 205 encodes the video data using H.265 and the audio using Advanced Audio Coding (AAC). The data is then formatted as an MP4 (Moving Picture Experts Group phase 4) container and packaged into an MPEG2-TS (Moving Picture Experts Group phase 2-Transport Stream). At this time, the video processing unit 205 writes thumbnail images to the HDD 204, linking them to the packaged video data.

[0039] Next, in S302, the video processing unit 205 further divides the packaged content into segment files separated by predetermined time units, generates a manifest file describing the playback order of the divided files, etc., and writes it to the HDD 204.

[0040] Furthermore, in S303, the video processing unit 205 instructs the inference unit 255 to infer whether predetermined features are included in the video data received via the communication unit 202. Upon receiving the instruction, the inference unit 255 inputs each frame image of the video data into the trained model and writes the feature amount to the RAM 203. Here, the feature amount is, for example, a combination of the detection result of a so-called highlight scene, which corresponds to a predetermined person present in the frame image, an event caused by that person, or a predetermined play action in sports in particular, and the like, and the inference score.

[0041] When the video processing unit 205 has executed the above-described inference processing for all frame images, it notifies the control unit 201 that the processing is complete. Upon receiving the notification, the control unit 201 reads information about the above-described feature amounts from the RAM 203, determines whether predetermined people and events exist, and determines whether the inference scores assigned to them exceed predetermined thresholds.

[0042] In S304, the control unit 201 writes information indicating which people, events, etc. are included in which sections of the video data based on the result of the determination to the HDD 204 as a table.

[0043] FIG. 4 is a diagram showing an example of table data indicating information about highlight scenes (partial moving images). In FIG. 4, 401 is a highlight ID assigned to the section of the detected highlight scene, and is assigned to information indicating the moving image section of the highlight scene (described later). 402 is information indicating the start time of the highlight scene, and 403 is information indicating the end time of the highlight scene. 404 indicates information indicating a predetermined person determined from the frame image and the face of that person. Here, IDs assigned in advance to predetermined face / person information are held, and information indicating a person included in the frame image and the face of that person is recorded here. 405 is information indicating an event determined from the frame image. Here, IDs assigned in advance to predetermined events are held, and information indicating an event included in each frame image or across multiple frame images is recorded here.

[0044] Next, in S305, the control unit 201 references the start time 402 and end time 403 in the table described above to determine the segment files included in the relevant section from the segment files written to the HDD 204. Then, a manifest file listing these segment files in playback order is written to the HDD 204. The control unit 201 executes this process for each highlight ID 401, and sequentially writes the manifest files corresponding to each highlight ID 401 to the HDD 204. Furthermore, a thumbnail image is generated from, for example, the first frame image of the video of each highlight scene, and the thumbnail image is written to the HDD 204 in association with the manifest file. The thumbnail image may be generated when the manifest file is written to the HDD 204, or may be generated when a thumbnail image is requested, as described below.

[0045] Furthermore, the control unit 101 writes to the HDD 204 a manifest file describing segment files of a predetermined section that are continuations of the segment file that was last played, described in each written manifest file. This is used when the tablet terminal 100, described below, plays back the continuation of a highlight scene after playing it. By having the tablet terminal 100 refer to this file, the control unit 201 writes to the HDD 204 a manifest file that lists predetermined sections from the segment files that are continuations of the segment file that was last played in the written manifest file. Note that the predetermined section can be any section, such as a section corresponding to a uniformly determined total real-time playback time or a section that changes depending on the highlight scene, but is not limited to any one of these sections.

[0046] The order of the inference process and encoding process for the video data uploaded by the user is not limited to the above, and the table data described above may be created after the processes are performed in the reverse order.

[0047] Through the above process, a manifest file, that is, a playlist file, from which highlight scenes are extracted is generated.

[0048] In addition, the server 200 also has a web server function for accepting requests for the generated data, and can transmit various information, image data, video data, etc. based on requests sent via HTTP / HTTPS via the communication unit 202.

[0049] ■ Highlight scene playback Next, with reference to FIGS. 5 and 6, a process of reproducing highlight scenes generated by the server 200 on the tablet terminal 100 in the first embodiment will be described.

[0050] 5 is a flowchart showing the flow of operations on the tablet terminal 100. This flowchart shows the operations when a user uploads a video to the server 200 in advance, launches a web browser on the tablet terminal 100, and accesses the server 200 in order to play back highlight scenes generated from the video.

[0051] First, in S501, the control unit 101 requests thumbnail images of highlight scenes (a list showing one or more partial videos) from the server 200 via the communication unit 102. Thereafter, the control unit 101 receives thumbnail image data via the communication unit 102, writes the received thumbnail image data to the RAM 103, and instructs the display control unit 107 to display the written thumbnail image data on the display unit 106.

[0052] 7 is a diagram showing an example of thumbnail images displayed on the display unit 106 of the tablet terminal 100. Here, four thumbnail images are requested and displayed. Note that the manner in which thumbnail images are displayed and the number of thumbnail images requested are not limited to this example, and for example, thumbnail images corresponding to all highlight scenes generated from a video uploaded to the server 200 by the user may be displayed.

[0053] After displaying the thumbnail images, in S502 the control unit 101 waits for the operation unit 105 to select one of the thumbnail images displayed on the display unit 106. When a thumbnail image is selected, the process proceeds to S503, where the control unit 101 transmits a request for a manifest file corresponding to the selected thumbnail image via the communication unit 102.

[0054] Then, in S504 and S505, the segment files that make up the highlight scenes described in the manifest file are played back in sequence. First, in S504, the control unit 101 reads the manifest file received via the communication unit 102 and written to the RAM 103, and transmits a request for segment files via the communication unit 102 based on the playback order described therein.

[0055] In S505, the control unit 101 instructs the decoder 108 to decode the segment file received via the communication unit 102 and written to the RAM 103. When the decoding by the decoder 108 is completed, the control unit 101 instructs the display control unit 107 to display on the display unit 106 based on the decoded video data.

[0056] Once all the segment files described in the manifest file have been played, the process proceeds to S506. Then, the control unit 101 instructs the display control unit 107 to display a UI (User Interface) for inquiring about whether to continue playback (playback of related videos), and displays it on the display unit 106. Figure 8 is a diagram showing an example of the last frame image of the played segment file and options for inquiring about whether to continue playback, and shows a continue viewing button 800 for continuing viewing and an end button 801 for ending viewing.

[0057] Then, in S507, the control unit 101 waits for a user input on the UI inquiring about the continuation of playback. When the user touches the continue viewing button 800, the control unit 101 receives the instruction via the display unit 106 and proceeds to S508. On the other hand, when the user touches the end button 801, the control unit 101 proceeds to S501 and again transmits a request for thumbnail images of highlight scenes to the server 200 via the communication unit 102.

[0058] In S508, the control unit 101 transmits, via the communication unit 102, a request for a manifest file that describes a group of segment files that are continuations of the segment files played back in S504 and S505.

[0059] Next, in steps S509 and S510, the segment files that make up the moving image of the highlight scenes, which are described in the manifest file, are played back in sequence. First, in S509, the control unit 101 reads the manifest file received via the communication unit 102 and written to the RAM 103, and transmits a request for segment files via the communication unit 102 based on the playback order described therein.

[0060] In S510, the control unit 101 instructs the decoder 108 to decode the segment file received via the communication unit 102 and written to the RAM 103. Subsequently, when the decoding by the decoder 108 is completed, the control unit 101 instructs the display control unit 107 to display on the display unit 106 based on the decoded video data.

[0061] When all the segment files described in the manifest file have been played back, the process proceeds to S511. Then, the control unit 101 instructs the display control unit 107 to display the end button 801 shown in Fig. 8 on the last frame image of the played segment file, displays it on the display unit 106, and the process proceeds to S512.

[0062] In S512, when the user touches the end button 801, the control unit 101 transitions to S501 and again transmits a request for thumbnails of highlight scenes to the server 200 via the communication unit .

[0063] On the other hand, if the user does not return to the thumbnail screen but instead issues an instruction to close the web browser via the operation unit 105 in S512, the control unit 101 ends the processing based on that instruction.

[0064] In the above example, the case where only the end button 801 is displayed in S511 has been described, but it is also possible to display the continue viewing button 800. In this case, when the continue viewing button 800 is selected, the process returns to S508, where a request for a manifest file describing a group of segment files corresponding to the continuation of the segment file that was played back is transmitted, and playback processing is performed.

[0065] Next, a description will be given of the operation of the server 200, which corresponds to the operation of the tablet terminal 100 described above. In S601, the control unit 201 waits for a request to be received via the communication unit 202. When a request is received via the communication unit 202, in S602 the control unit 201 determines whether the received request is a request for a thumbnail image of a highlight scene. If it is determined that the received request is a request for a thumbnail image of a highlight scene, the process proceeds to S603, where the control unit 201 reads out the corresponding thumbnail image from the HDD 204 and transmits it via the communication unit 202.

[0066] If the control unit 201 determines in S602 that the received request is not a request for a thumbnail image of a highlight scene, the control unit 201 determines in S604 whether the received request is a request for a manifest file. If the control unit 201 determines that the received request is a request for a manifest file, the process proceeds to S605, where the control unit 201 reads the corresponding manifest file from the HDD 204 and transmits it via the communication unit 202.

[0067] If it is determined in S604 that the received request is not a request for a manifest file, the control unit 201 determines in S606 whether the received request is a request for a segment file. If it is determined that the received request is a request for a segment file, the process proceeds to S607, where the control unit 201 reads the corresponding segment file from the HDD 204 and transmits it via the communication unit 202.

[0068] If it is determined in S606 that the received request is not a request for a segment file, the control unit 201 returns to S601 and waits for the next request.

[0069] In this way, it is assumed that the segment files are requested based on the contents described in the manifest file sent to the tablet terminal 100 in S605, and each time the communication unit 202 receives a request, the control unit 201 executes S607.

[0070] Also, when viewing of the video that continues from the highlight scene is selected in S507 of FIG. 5 and the corresponding manifest file is requested via the communication unit 202, the control unit 201 executes S605 and transmits the manifest file.

[0071] As described above, the tablet device 100 inquires whether or not the user wants to watch the rest of the video when playback of the specified highlight scene ends. By requesting the rest depending on the result, it becomes possible to watch the video of sections other than the highlight scene that were generated in advance by AI inference.

[0072] In this embodiment, an embodiment has been described in which an inquiry is made as to whether or not to view the video following the highlight scene, but the subject of the inquiry is not limited to this method. For example, a manifest file of a video (related video) corresponding to the part before the start position of the highlight scene may be generated in advance on the server 200, and an inquiry about viewing that video may be made. Furthermore, with regard to viewing the video corresponding to the part before the start position of the highlight scene, in addition to being presented at the end of playback, an inquiry about viewing may also be made during playback of the highlight scene.

[0073] In this embodiment, the HLS method is used for playing back highlight scenes, but the present invention is not limited to this method. For example, any protocol that allows streaming playback, such as MPEG-DASH (Moving Picture Expert Group-Dynamic Adaptive Streaming over HTTP), RTSP (Real Time Streaming Protocol), or RTMP (Real Time Messaging Protocol), can be applied.

[0074] The highlight scene video handled in this embodiment can be applied to content other than that shot by a single camera. For example, the viewing operations described in this embodiment may be applied to content suitable for playing back highlight scenes, such as content created using free viewpoint video technology that is generated by combining content shot by multiple cameras.

[0075] Furthermore, after viewing the video following the highlight scene, the user may be asked whether to view the highlight scene and the video following the highlight scene in succession. This allows the video to be viewed without interruption. In this case, the tablet device 100 transmits a request for the segment files described in the acquired manifest file to the server 200 via the communication unit 102, and the server 200 transmits the requested segment files to the tablet device 100 via the communication unit 202.

[0076] Alternatively, if the tablet terminal 100 is capable of storing decrypted video data, the stored decrypted video data may be played back.

[0077] As described above, according to the first embodiment, when a highlight scene is viewed, the video following the highlight scene can be viewed.

[0078] <Second embodiment> Next, a second embodiment of the present invention will be described.

[0079] In the first embodiment, a method was described in which, when playback of a highlight scene has ended, a user is inquired as to whether or not to watch the rest of the video, and based on the result, a video segment other than the highlight scene is viewed. In contrast, in the second embodiment, a method is described in which, when playback of a highlight scene has ended, a user is inquired as to whether or not to watch the rest of the video, and an option to view highlight scenes surrounding the highlight scene whose playback has ended is presented. Note that the configurations of the tablet terminal 100 and the server 200 in this embodiment are the same as those described in the first embodiment with reference to FIGS. 1 and 2, and therefore a description thereof will be omitted here. Furthermore, the following description will mainly focus on the operations of the tablet terminal 100 and the server 200 that are different from those in the first embodiment, and operations similar to those in the first embodiment will be omitted as appropriate.

[0080] Fig. 9 is a flowchart showing the operation of the tablet terminal 100. Note that the processes in S501 to S505 are the same as those in S501 to S505 in Fig. 5 of the first embodiment, so the same reference numerals are used and the description will be omitted.

[0081] When the playback of the highlight video ends in S501 to S505, the process proceeds to S906, where the control unit 101 transmits, via the communication unit 202, a request for a thumbnail image of the highlight video related to the highlight video whose playback has ended.

[0082] When a thumbnail image of a highlight video related to the highlight video whose playback has finished is received via the communication unit 202, in S907 the control unit 101 instructs the display control unit 107 to display a UI that inquires about whether to continue playback and suggests viewing of the related highlight video.

[0083] FIG. 11 is a diagram showing an example of the UI displayed in S907, in which thumbnail images 1102 to 1104 of related highlight videos (related videos) received from the server 200 are displayed in addition to the continue viewing button 800 and end button 801 shown in FIG.

[0084] Here, when the continue viewing button 800 or any of the thumbnail images 1102 to 1104 is selected in S908, the process proceeds to S909. In S909, the control unit 101 transmits, via the communication unit 102, a request for a group of segment files corresponding to continuations of the segment files played in S504 and S505, or a manifest file based on the selected thumbnail image.

[0085] The subsequent processing from S509 to S511 is the same as the processing from S509 to S511 in FIG. 5 of the first embodiment, so the same step numbers are used and the description will be omitted.

[0086] Next, the operation of the server 200 will be described. 10 is a diagram showing the operational flow of the server 200. Steps S1001 to S1003 are processes in response to the request for thumbnail images of the related highlight video in step S906 in FIG.

[0087] In S1001, the control unit 201 waits for a request for a related thumbnail image via the communication unit 202.

[0088] If the control unit 201 determines that a request for related thumbnail images has been received via the communication unit 202, the process proceeds to S1002, where the control unit 201 reads table data related to the highlight video from the HDD 204 and determines the related video.

[0089] As an example of the determination, for example, the control unit 201 determines and selects information about each highlight video shown in FIG. 4 that has a start time 402 or end time 403 that is close to the highlight video that has been played up until that point, and transmits the selected information to the tablet terminal 100.

[0090] Alternatively, the control unit 201 may determine and select an object whose face / person ID 404 or event ID 405 matches that of the highlight video that has been played up until that point, and transmit this to the tablet terminal 100.

[0091] Furthermore, in addition to the face / person ID and event ID mentioned above, in the case of sports, any object that can be inferred by AI is sufficient, such as a player's uniform number, team type, or the role or position of a player, as long as the relevance can be determined according to certain rules.

[0092] In S1002, when the control unit 201 determines and selects a highlight video related to the highlight video that has been played up until then, the process transitions to S1003, and the control unit 201 transmits a thumbnail image of the determined highlight video via the communication unit 202.

[0093] As described above, when playback of a highlight video ends, the user is prompted to continue watching, and the option to watch related highlight videos is presented, allowing the user to watch the scenes they desire.

[0094] <Other embodiments> The present invention may be applied to a system made up of a plurality of devices, or to an apparatus made up of a single device.

[0095] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0096] <Summary> The disclosure of this embodiment includes the following configuration.

[0097] (Configuration 1) A communication means for communicating with an external device that edits and distributes videos; A display means; a control unit that controls the communication unit and the display unit, The control means receiving a list indicating one or more partial moving images extracted from the moving image based on predetermined characteristics from the external device via the communication means, and displaying the list on the display means; receiving a selected partial moving image from the displayed list from the external device via the communication means and playing it on the display means; When the playback of the selected partial video is completed, an option is displayed on the display means to inquire whether or not the user would like to view a related video, which is a video of a part of the video that is related to the selected partial video but different from the selected partial video; When viewing of the related video is selected, the related video is received from the external device via the communication means and played on the display means. A playback device characterized by controlling the playback device to:

[0098] (Configuration 2) The playback device described in configuration 1 is characterized in that the related video is a video that continues from the selected partial video, a video that precedes the selected partial video, or a partial video that is related to the selected partial video among the one or more partial videos.

[0099] (Configuration 3) The playback device described in configuration 2 is characterized in that the control means controls the display means to display, among the options, an option for inquiring about viewing a video portion preceding the selected partial video during playback of the selected partial video.

[0100] (Configuration 4) The playback device according to configuration 2 or 3, wherein the video following the selected partial video and the video preceding the selected partial video are videos of a predetermined length.

[0101] (Configuration 5) The control means further When the playback of the subsequent video is completed, an option is displayed on the display means to inquire whether or not the user wants to watch the subsequent video; When viewing of the further video is selected, the further video is received via the communication means and played on the display means. 5. The playback device according to any one of configurations 2 to 4, characterized in that the playback device is controlled as follows.

[0102] (Configuration 6) The control means further When the playback of the subsequent video is completed, an option is displayed on the display means to inquire whether or not the selected partial video and the subsequent video should be played back consecutively; When continuous playback is selected, the selected partial video and the subsequent video are played back continuously on the display means. 6. The playback device according to any one of configurations 2 to 5, characterized in that the playback device is controlled as follows.

[0103] (Configuration 7) The playback device according to any one of configurations 1 to 6, wherein the one or more partial moving images are partial moving images extracted by performing inference on the moving images using an inference model.

[0104] (Configuration 8) 8. The playback device according to any one of configurations 1 to 7, wherein the list is a thumbnail image of each of the one or more partial moving images.

[0105] (Configuration 9) a playback device according to any one of configurations 1 to 8; a server, The server: an extraction means for extracting the one or more partial moving images from the input moving image based on predetermined characteristics; a communication means for transmitting, in response to a request from the playback device, a list indicating the one or more partial moving images and a partial moving image requested from the playback device among the one or more partial moving images to the playback device; When the related video is requested from the playback device, the related video is transmitted via the communication means. A system characterized by:

[0106] (Configuration 10) The system described in configuration 9, wherein the extraction means performs inference using an inference model to extract the one or more partial videos.

[0107] (Configuration 11) receiving a list indicating one or more partial moving images extracted from the moving image based on predetermined characteristics from an external device via communication means, and playing the list on display means; receiving a selected partial moving image from the displayed list from the external device via the communication means and playing it on the display means; When the playback of the selected partial video ends, displaying on the display means an option to inquire whether or not to view a related video, which is a video of a part of the video that is related to the selected partial video but different from the selected partial video; receiving the related video from the external device via the communication means and playing it on the display means when viewing the related video is selected; A regeneration method comprising the steps of:

[0108] (Configuration 12) A program for causing a computer to function as each means of the playback device according to any one of configurations 1 to 8.

[0109] (Configuration 13) A computer-readable storage medium storing the program according to configuration 12.

[0110] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, the following description is given to clarify the scope of the invention. [Explanation of symbols]

[0111] 100: tablet terminal, 101: control unit, 102: communication unit, 103: RAM, 105: operation unit, 106: display unit, 107: display control unit, 108: decoder, 200: server, 201: control unit, 202: communication unit, 203: RAM, 204: HDD, 205: video processing unit, 255: inference unit

Claims

1. A communication means for communicating with an external device that edits and distributes videos; A display means; a control unit for controlling the communication unit and the display unit, The control means receiving a list indicating one or more partial moving images extracted from a moving image based on a predetermined feature from the external device via the communication means, and displaying the list on the display means; receiving a selected partial moving image from the displayed list from the external device via the communication means and playing the selected partial moving image on the display means; When the playback of the selected partial video is completed, a selection is displayed on the display means to inquire whether or not to view a related video, which is a video of a part of the video that is related to the partial video and different from the partial video; When viewing of the related video is selected, the related video is received from the external device via the communication means and is played on the display means. The playback device is characterized by controlling the playback device to perform the following:

2. The playback device according to claim 1, characterized in that the related video is a video that continues from the selected partial video, a video that is a portion of the video preceding the selected partial video, or a partial video that is related to the selected partial video among the one or more partial videos.

3. The playback device according to claim 2, characterized in that the control means controls the display means to display, among the options, an option for inquiring about viewing a portion of the video preceding the selected partial video during playback of the selected partial video.

4. 3. The playback device according to claim 2, wherein the moving image following the selected partial moving image and the moving image preceding the selected partial moving image are moving images of a predetermined length.

5. The control means further comprises: When the playback of the subsequent video is completed, a choice is displayed on the display means to inquire whether or not the subsequent video is to be viewed; When viewing of the further video is selected, the further video is received via the communication means and played on the display means.

3. The reproducing apparatus according to claim 2, wherein the reproducing apparatus is controlled as follows.

6. The control means further comprises: displaying on the display means an option for inquiring whether or not to continuously play the selected partial video and the subsequent video when the playback of the subsequent video is completed; When the continuous reproduction is selected, the selected partial moving image and the subsequent moving image are reproduced continuously on the display means.

3. The reproducing apparatus according to claim 2, wherein the reproducing apparatus is controlled as follows.

7. The playback device according to claim 1 , wherein the one or more partial moving images are partial moving images extracted by performing inference on the moving images using an inference model.

8. 2. The playback device according to claim 1, wherein the list is a list of thumbnail images of the one or more partial moving images.

9. A playback device according to any one of claims 1 to 8; A system including: The server, an extraction means for extracting the one or more partial moving images from the input moving image based on a predetermined feature; a communication means for transmitting, in response to a request from the playback device, to the playback device, a list indicating the one or more partial moving images and a partial moving image requested from the playback device among the one or more partial moving images; When the related video is requested from the playback device, the related video is transmitted via the communication means. A system characterized by:

10. The system according to claim 9 , wherein the extraction means extracts the one or more partial videos by performing inference using an inference model.

11. receiving, via a communication means from an external device, a list indicating one or more partial moving images extracted from the moving image based on a predetermined feature, and displaying the list on a display means; receiving a partial moving image selected from the displayed list from the external device via the communication means and playing the partial moving image on the display means; displaying on the display means an option to inquire whether or not to view a related video, which is a video of a part of the video that is related to the selected partial video and different from the partial video, when playback of the selected partial video ends; receiving the related video from the external device via the communication means and playing the related video on the display means when viewing of the related video is selected; A regeneration method comprising the steps of:

12. A program for causing a computer to function as each of the means of the reproducing apparatus according to any one of claims 1 to 8.

13. A computer-readable storage medium storing the program according to claim 12.

Images