Content processing device, content processing program and content processing method

Abstract

This disclosure demonstrates that a simple mechanism can display a dynamic image that only reflects the tracked object T. The content processing apparatus (2) disclosed herein includes: a dynamic image acquisition unit (231) that acquires content data containing a dynamic image; an information generation unit (232) that detects a plurality of viewing objects from the dynamic image and generates position information for displaying the position of each feature of the plurality of viewing objects at a plurality of time points in the dynamic image, and size information for displaying the size of a portion of the viewing object that includes at least the feature; a dynamic image generation unit (233) that, based on the position information and size information, cuts out a portion of the viewing object corresponding to the plurality of viewing objects from each frame corresponding to the plurality of time points in the dynamic image, and generates a tracking dynamic image for each frame composed of the cut-out portion of the viewing object for each viewing object; an information acquisition unit that acquires selection information of the viewing object selected as the tracking object among the plurality of viewing objects; and an output unit (21) that, based on the selection information, outputs a tracking dynamic image of the tracking object reflected in the plurality of tracking dynamic images.

Description

[Technical Field]

[0001] This invention relates to a content processing device, a content processing program, and a content processing method. [Previous Technology]

[0002] Prior art has proposed various image processing techniques to provide customized live images for each viewer. For example, Patent Document 1 describes an image data processing apparatus comprising: an input means for inputting images captured by a plurality of cameras arranged in a venue; an analysis data acquisition means for acquiring music analysis data, i.e., the analysis results of the sound of music played in the venue; a switching means for switching the output image based on the music analysis data; and an output means for outputting the image switched by the switching means.

[0003] [Prior Art Documents][Patent Documents]Patent Document 1: Japanese Patent Application Publication No. 2021-153257 [Summary of the Invention]

[0004] (Problem to be solved by the invention) However, in the prior art described in Patent Document 1, multiple cameras must be used to photograph the object. Furthermore, since the switching is performed using sound analysis results, it cannot handle content that only contains moving images.

[0005] One aspect of the present invention was made in view of the above-mentioned problems, and its purpose is to display a dynamic image that only reflects the tracked object with a simple mechanism.

[0006] (Technical means to solve the problem) In order to solve the above-mentioned problem, the present invention provides a content processing apparatus related to a state, comprising: a dynamic image acquisition unit that acquires content data including a dynamic image; an information generation unit that detects a plurality of viewing objects from the aforementioned dynamic image and generates position information for displaying the position of each feature of the plurality of viewing objects at a plurality of time points in the dynamic image, and size information for displaying the size of at least a portion of the viewing object including the aforementioned feature; a dynamic image generation unit that, based on the aforementioned position information and the aforementioned size information, crops the aforementioned portion of each frame corresponding to the plurality of time points in the dynamic image, and generates a tracking dynamic image for each frame composed of the cropped plurality of the aforementioned portion of each viewing object; an information acquisition unit that acquires selection information for displaying the viewing object selected as the tracking object among the plurality of viewing objects; and an output unit that, based on the aforementioned selection information, outputs the aforementioned tracking dynamic image reflecting the aforementioned tracking object in the plurality of aforementioned tracking dynamic images.

[0007] Furthermore, the content processing program related to other aspects of the present invention causes the computer to execute: a dynamic image acquisition process, which acquires content data containing a dynamic image; a generation process, which detects a plurality of viewing objects from the aforementioned dynamic image and generates position information of the positions of each feature of the plurality of viewing objects at a plurality of time points for displaying the dynamic image, and size information of the size of at least a portion of the viewing object containing the aforementioned feature; a dynamic image generation process, which, based on the aforementioned position information and the aforementioned size information, crops out the aforementioned portion of each frame corresponding to the plurality of time points of the aforementioned dynamic image, corresponding to the plurality of viewing objects, and generates a tracking dynamic image for each frame composed of the cropped plurality of aforementioned portion of each viewing object; a result acquisition process, which acquires selection information of the viewing object selected as the tracking object among the plurality of viewing objects; and an output process, which, based on the aforementioned selection information, outputs the aforementioned tracking dynamic image reflecting the aforementioned tracking object in the plurality of aforementioned tracking dynamic images.

[0008] Furthermore, the content processing method related to other aspects of the present invention includes: a generation step, which uses a computer to detect a plurality of viewing objects from the dynamic image contained in the content, and generates position information for displaying the position of each feature of the plurality of viewing objects at a plurality of time points in the dynamic image, and size information for displaying the size of at least a portion of the viewing object containing the aforementioned feature; a dynamic image generation step, which uses a computer to crop out the aforementioned portion of each frame corresponding to the plurality of time points in the dynamic image based on the aforementioned position information and the aforementioned size information, and generates a tracking dynamic image for each frame composed of the cropped plurality of aforementioned portion of each viewing object; a result acquisition step, which uses a computer to acquire selection information of the viewing object selected as the tracking object among the plurality of viewing objects; and an output step, which uses a computer to output the aforementioned tracking dynamic image reflecting the aforementioned tracking object in the plurality of aforementioned tracking dynamic images based on the aforementioned selection information.

[0009] (Effects of the Invention) By means of one aspect of the present invention, it is possible to display a dynamic image that only reflects the tracked object through a simple mechanism.

Implementation Method

[0011] <First Embodiment of Content Playback System> Hereinafter, one embodiment of the present invention will be described in detail.

[0012] [Composition of Content Playback System 100] As shown in FIG1, the content playback system 100 includes a content server 1, a content processing device 2, and a terminal device 3. In addition, the content processing device 2 may be integrated with the content server 1.

[0013] [Content Server 1] The content server 1 communicates with the terminal device 3. In this embodiment, the content server 1 communicates wirelessly with the terminal device 3. Furthermore, the content server 1 in this embodiment also communicates with the content processing device 2. In this embodiment, the content server 1 also communicates wirelessly with the content processing device 2. Alternatively, the content server 1 may be configured to communicate wiredly with both the terminal device 3 and the content processing device 2.

[0014] Furthermore, the content server 1 stores content data of moving images for transmission to the content processing device 2 and the terminal device 3. In this embodiment, the content server 1 stores content data including sound. Also, in this embodiment, the content server 1 stores content data obtained from recording music activities.

[0015] Furthermore, upon receiving a transmission request signal from the content processing device 2, the content server 1 will send the content data corresponding to the transmission request signal to both the content processing device 2 and the terminal device 3. The terminal device 3, upon receiving the content data, will play a dynamic image based on the data. The operation of the content processing device 2 upon receiving the content data will be described later.

[0016] [Content processing device 2] The content processing device 2 includes a communication unit 21, a memory unit 22 and an arithmetic unit 23.

[0017] (Communication Unit 21) The communication unit 21 is the output unit in the first embodiment. The communication unit 21 communicates with the terminal device 3. The communication unit 21 in this embodiment communicates wirelessly with the terminal device 3. Furthermore, the communication unit 21 in this embodiment also communicates with the content server 1. The communication unit 21 in this embodiment also communicates wirelessly with the content server 1. The communication unit 21 in this embodiment is composed of a communication module. Alternatively, the communication unit 21 may also be composed of a terminal for connecting a cable used for wired communication with the terminal device 3 or the content server 1.

[0018] (Memory Unit 22) The memory unit 22 stores the content processing program. The content processing program is used to enable the computer to function as a content processing device 2. In this embodiment, the memory unit 22 is composed of semiconductor memory, hard disk, etc.

[0019] (Arithmetic Unit 23) The arithmetic unit 23 includes a motion image acquisition unit 231, an information generation unit 232, a motion image generation unit 233, and an information acquisition unit 234. In this embodiment, the arithmetic unit 23 also includes an output processing unit 235. The arithmetic unit 23 in this embodiment is composed of a processor. By executing the content processing program stored in the memory unit 22, the processor functions as the motion image acquisition unit 231, the information generation unit 232, the motion image generation unit 233, the information acquisition unit 234, and the output processing unit 235.

[0020] Motion Image Acquisition Unit 231 The motion image acquisition unit 231 performs motion image acquisition processing. During motion image acquisition processing, the motion image acquisition unit 231 acquires content data containing motion images. In this embodiment, the motion image acquisition unit 231 acquires content data received by the communication unit 21 from the content server 1. Furthermore, when the content processing device 2 and the content server 1 are integrated, the motion image acquisition unit 231 can also acquire data read from the memory unit 22, etc.

[0021] Information Generation Unit 232 performs information generation processing after the motion image acquisition unit 231 has acquired content data containing a motion image. In the information generation processing, the information generation unit 232 detects a plurality of viewing objects W from the motion image. Then, the information generation unit 232 generates position information and size information for the plurality of viewing objects W at a plurality of time points in the motion image. "Viewing object W" is, for example, a person. Alternatively, the viewing object W can be an animal or an object (such as a musical instrument). "Position information" displays the position of each feature of the plurality of viewing objects W. "Size information" displays the size of a partial region R. "Partial region R" is a region that at least contains the feature of the viewing object W. In the case where the viewing object W is a person, "feature" is, for example, a person's face. Alternatively, the feature can be an object (held by a person) attached to the viewing object W (such as a musical instrument). Furthermore, when the object of viewing W is a person, the partial area R includes an area containing only the face, an area containing the upper body, and an area containing the whole body. The shape of the partial area R is preferably similar to the shape of the display area of ​​the playback unit 32 of the terminal device 3 (for example, a rectangle with the same aspect ratio). "Time point" can be a moment (0 seconds after the start of the playback of the moving image) or a frame. The information generation unit 232 related to this embodiment generates size information as shown in FIG3, so that the ratio b / a of the size b of the feature part of the tracked object T to the size a (width, height, or area) of the partial area R is within a predetermined range (lower limit c ≦ b / a ≦ upper limit d).

[0022] Furthermore, the information generation unit 232 related to this embodiment uses a learned model to generate position information and size information. The learned model is constructed to take the content data of the viewed object W as input and output the position information and size information of the viewed object W. In this way, the position of the tracking object T can be detected without constructing a complex algorithm.

[0023] The motion image generation unit 233 performs motion image generation processing after the information generation unit 232 has generated position information and size information. In this process, the motion image generation unit 233 first crops out partial regions R corresponding to multiple viewing objects W from each frame corresponding to multiple time points of the motion image, based on the position information and size information. Then, the motion image generation unit 233 generates a tracking motion image for each frame, consisting of the cropped partial regions R, for each viewing object W. The motion image generation unit in this embodiment generates a tracking motion image where the partial regions R are enlarged to match the display area of ​​the motion image.

[0024] The motion picture generation unit 233 of this embodiment performs processing to emphasize the components in the sound contained in the content that are equivalent to the sound emitted by the person who is the tracking object T or the sound emitted by the instrument played by that person. Specifically, the motion picture generation unit 233 extracts the frequency components equivalent to the sound emitted by the person who is the tracking object T or the sound emitted by the instrument played by that person from the sound, amplifies the extracted frequency components, and synthesizes them with the original sound.

[0025] Information Acquisition Unit 234 The Information Acquisition Unit 234 performs information acquisition processing. During this processing, the Information Acquisition Unit 234 acquires selection information. The selection information displays information about the viewer's object W selected as the tracking object T from a plurality of viewable objects W. "Tracking object T" refers to the viewer's specific (only) desired viewable object W among the viewer's objects W (candidates for tracking objects) projected in the moving image, and is the viewer's object W selected by the viewer via the terminal device 3 described later. In this embodiment, the Information Acquisition Unit 234 acquires the selection information received by the Communication Unit 21 from the terminal device 3.

[0026] Output Processing Unit 235 performs output processing when the information acquisition unit 234 has acquired the selection information. During output processing, the output processing unit 235 controls the communication unit 21 (output unit). Thereby, the communication unit 21, based on the selection information, sends data from multiple tracking motion images reflecting the tracking object T to the terminal device 3 (outputting the tracking motion image). As described above, the motion image generation unit 233 performs processing to emphasize sound components contained in the content. Therefore, the communication unit 21, based on the information in this embodiment, outputs the tracking motion image data and the emphasized sound data.

[0027] [Terminal Device 3] The terminal device 3 includes a terminal communication unit 31, a playback unit 32, and an operation unit 33. The terminal device 3 according to this embodiment also includes a terminal memory unit 34 and a terminal arithmetic unit 35.

[0028] (Terminal Communication Unit 31) The terminal communication unit 31 communicates with the content processing device 2. In this embodiment, the terminal communication unit 31 communicates wirelessly with the content processing device 2. Furthermore, the terminal communication unit 31 in this embodiment also communicates with the content server 1. The terminal communication unit 31 in this embodiment also communicates wirelessly with the content server 1. The terminal communication unit 31 in this embodiment is composed of a communication module. Alternatively, the terminal communication unit 31 may also be configured to communicate wiredly with the content server 1 or the content processing device 2.

[0029] (Playback Unit 32) The playback unit 32 is configured to display various dynamic images. The playback unit 32 related to this embodiment is composed of a display device (e.g., a liquid crystal display, an organic EL display, etc.).

[0030] (Operation Unit 33) The operation unit 33 is operated by the user. The operation includes sending request operations, tracking object selection operations, etc. The sending request operation is used to request the content server 1 to send the content data desired by the user. When a sending request operation is performed, the terminal device 3 sends a sending request signal corresponding to the sending request operation to the content server 1. The tracking object T selection operation will be described later. The operation unit 33 may include, for example, physical buttons, touch panels, pointer devices (such as mice), etc.

[0031] (Terminal Memory Unit 34) The terminal memory unit 34 stores a content processing program (application program). The content processing program is used to enable the computer to function as a terminal device 3. The terminal memory unit 34 in this embodiment is composed of semiconductor memory, hard disk, etc.

[0032] (Terminal Computing Unit 35) The terminal computing unit 35 includes a terminal dynamic image acquisition unit 351, a receiving unit 352, a transmission control unit 353, and a playback control unit 354. In this embodiment, the terminal computing unit 35 is composed of a processor. By executing the content processing program stored in the terminal memory unit 34, the processor functions as the terminal dynamic image acquisition unit 351, the receiving unit 352, the transmission control unit 353, and the playback control unit 354.

[0033] Terminal Dynamic Image Acquisition Unit 351 The terminal dynamic image acquisition unit 351 is a dynamic image acquisition unit in the second embodiment. The terminal dynamic image acquisition unit 351 performs dynamic image acquisition processing. In the dynamic image acquisition processing, the terminal dynamic image acquisition unit 351 acquires content data. The dynamic image containing content can be an unprocessed (cropped) dynamic image, or it can be a tracking dynamic image in which a portion of the region R has been cropped. The terminal dynamic image acquisition unit 351 associated with this embodiment acquires the content data received by the terminal communication unit 31 from the content server 1.

[0034] When the operation unit 33 performs a tracking object selection operation, the receiving unit 352 performs a receiving process. In the receiving process, the receiving unit 352 accepts the selection of the tracking object T in the moving image based on the operation performed in the operation unit 33 and generates selection information. The receiving unit 352 related to this embodiment accepts the tracking object selection operation performed in the operation unit 33, which is "selecting any one of the viewing objects W from the plurality of viewing objects W displayed in the moving image", as the selection of the tracking object T. When the operation unit 33 is composed of a touch panel, "touching the viewing object W in the moving image" is the tracking object selection operation. Also, when the operation unit 33 is composed of an indicator device, "aligning the cursor with the viewing object W in the moving image and clicking" is the tracking object selection operation. As shown in FIG2, the receiving unit 352 related to this embodiment displays a box F for surrounding the selectable viewing object W reflected in the moving image. In this way, the viewer can easily select the tracking object T.

[0035] Furthermore, in the processing, the receiving unit 352 can be configured to display a list of multiple names on the playback unit 32. The "name" includes the name of the viewing object W, the name of the item attached to the viewing object W (such as a musical instrument held by a person), the name of the group to which the viewing object W belongs (such as a playing section), etc. Then, the receiving unit 352 can be configured to process the tracking object selection operation of "selecting any name from the list" performed in the operation unit 33 as a selection of the tracking object T. In this case, the receiving unit 352 will refer to, for example, the correspondence between the viewing object W in the moving image and the names on the list stored in the memory unit 22, to generate selection information for the viewing object W corresponding to the selected name.

[0036] When the receiving unit 352 has generated selection information, the sending control unit 353 will execute sending control processing. During sending control processing, the sending control unit 353 will control the terminal communication unit 31. Thereby, the terminal communication unit 31 will send selection information to the content processing device 2.

[0037] Playback Control Unit 354: When the terminal motion image acquisition unit 351 has acquired content data, the playback control unit 354 performs playback control processing. During playback control processing, the playback control unit 354 controls the playback unit 32. Thereby, the playback unit 32 displays motion images (including tracking motion images) in the display area. As described above, the motion image generation unit 233 of the content processing device 2 generates a magnified tracking motion image, aligning a portion of the area R with the display area of ​​the motion image. Therefore, the playback unit 32 uses the entire display area to display the tracking motion image. Thus, even when the camera zooms out and the tracking object T appears smaller, the viewer can still view a significantly magnified tracking motion image of the tracking object T, as shown in FIG. 4.

[0038] [Effects of Content Processing Device 2] In the content processing device 2 described above, the information generation unit 232 generates position information and size information, and the motion image generation unit 233 generates tracking motion images for each of the plurality of tracking objects T based on the position information and size information. Then, the communication unit 21 (output unit) sends the tracking motion image data corresponding to the selection information obtained by the information acquisition unit 234 to the terminal device 3. Therefore, according to the content processing device 2 or the content playback system 100 including the content processing device 2, the terminal device 3 can display a tracking motion image that only shows the tracking object T (partial area R) selected by the viewer in the display area. As a result, even if a plurality of viewing objects W are shown in the original motion image, the viewer can only view the tracking object T.

[0039] <Second Embodiment of the Content Playback System> Next, other embodiments of the present invention will be described. Furthermore, for ease of explanation, components that have the same function as those described in the above embodiments will be labeled with the same symbols, and their descriptions will not be repeated.

[0040] 〔Configuration of Content Playback System 100A〕 As shown in FIG5, the content playback system 100A includes a terminal device 3A in addition to the content server 1, which is the same as the content playback system 100 in the first embodiment.

[0041] [Terminal Device 3A] The terminal device 3A is a content processing device in the second embodiment. In addition to the terminal communication unit 31, playback unit 32 and operation unit 33 that are the same as those of the terminal device 3 in the first embodiment, the terminal device 3A also includes a terminal memory unit 34A and a terminal arithmetic unit 35A.

[0042] (Terminal Memory Unit 34A) The terminal memory unit 34A stores a content processing program (application program). The content processing program is used to enable the computer to function as a terminal device 3A. The terminal memory unit 34A related to this embodiment is composed of semiconductor memory, hard disk, etc.

[0043] (Terminal Computing Unit 35A) The terminal computing unit 35A, in addition to having the same terminal motion image acquisition unit 351 and receiving unit 352 as the terminal computing unit 35 of the first embodiment, and the same information generation unit 232 and motion image generation unit 233 as the computing unit 23 of the content processing apparatus 2 related to the first embodiment, also includes an information acquisition unit 234A and an output processing unit 235A. The terminal computing unit 35A related to this embodiment is composed of a processor. The processor executes the content processing program stored in the terminal memory unit 34A, and thus functions as the terminal motion image acquisition unit 351, receiving unit 352, information generation unit 232, motion image generation unit 233, information acquisition unit 234A, and output processing unit 235A.

[0044] . Information Acquisition Unit 234A: In this embodiment, the information acquisition unit 234A acquires selection information generated by the receiving unit 352.

[0045] Output Processing Unit 235A: In this embodiment, the output processing unit 235A controls the playback unit 32 when the information acquisition unit 234A has acquired selection information. Therefore, the playback unit 32 displays data of the tracking motion image showing the tracked object T among a plurality of tracking motion images based on the selection information. The motion image generation unit 233 in this embodiment, like the motion image generation unit 233 in the first embodiment, performs processing to emphasize the sound components contained in the content. Therefore, the playback unit 32 displays the tracking motion image along with the emphasized sound components. Furthermore, in this embodiment, the output processing unit 235A also controls the playback unit 32 (output unit) when the terminal motion image acquisition unit 351 has acquired content data. Therefore, the playback unit 32 plays the unprocessed (cropped) motion image.

[0046] [Effects of Terminal Device 3A] In the terminal device 3A described above, the information generation unit 232 generates position information and size information, and the motion image generation unit 233 generates tracking motion images for each of the plurality of tracked objects T based on the position information and size information. Then, the playback unit 32 (output unit) displays the tracking motion image corresponding to the selection information obtained by the information acquisition unit 234. Therefore, according to the terminal device 3A or the content playback system 100A equipped with the terminal device 3A, even if a plurality of viewing objects W are displayed in the original motion image, the viewer can only view the tracked object T.

[0047] <Embodiments of the Content Processing Method> Next, other embodiments of the present invention will be described. Furthermore, for ease of explanation, components that have the same function as those described in the above embodiments will be marked with the same symbols, and their descriptions will not be repeated.

[0048] 〔Flowchart of Content Processing Method S100〕 As shown in Figure 6, the content processing method S100 includes an information generation step S1, a dynamic image generation step S2, an information acquisition step S3, and an output step S4.

[0049] [Information Generation Step S1] In the initial information generation step S1, a plurality of viewing objects W are detected from the moving image. Then, position information of the positions of each feature of the plurality of viewing objects W at a plurality of time points in the moving image, and size information of the size of a portion region R containing at least the feature of the viewing object W, are generated. The detection of the viewing object W and the generation of the position and size information can be performed by the content processing device 2 associated with the first embodiment or the terminal device 3A associated with the second embodiment, or by other devices. Furthermore, the device for detecting the viewing object W and the device for generating the position and size information can also be different.

[0050] [Motion Image Generation Step S2] After generating position information and size information, motion image generation step S2 is performed. In motion image generation step S2, based on the position information and size information, partial regions R corresponding to multiple viewing objects W are cropped from each frame corresponding to multiple time points of the motion image. Then, for each viewing object W, a tracking motion image consisting of the cropped partial regions R is generated for each frame. The cropping of partial regions R and the generation of tracking motion images can be performed by the content processing device 2 related to the first embodiment or the terminal device 3A related to the second embodiment, or by other devices. Furthermore, the device for cropping partial regions R and the device for generating tracking motion images can be different.

[0051] [Information Acquisition Step S3] After generating the tracking motion image, the information acquisition step S3 is performed. In the information acquisition step S3, selection information of the viewing object W selected as the tracking object T from among the plurality of viewing objects is acquired. The acquisition of the selection information can be performed by the content processing device 2 of the first embodiment or the terminal device 3A of the second embodiment, or by other devices.

[0052] [Output Step S4] After accepting the selection of the tracking object T, the output step S4 is performed. In the output step S4, a tracking motion image showing the tracking object T is output from a plurality of tracking motion images based on the selection information. The output of the tracking motion image can be performed by the content processing device 2 associated with the first embodiment or the terminal device 3A associated with the second embodiment, or it can be performed by other devices.

[0053] [Effects of Content Processing Method S100] In the content processing method S100 described above, position information and size information are generated in the information generation step S1, and in the motion image generation step S2, tracking motion images of a plurality of tracking objects T are generated based on the position information and size information. Then, in the output step S4, the tracking motion image corresponding to the selection information obtained in the information acquisition step S3 is output. Therefore, according to the content processing method S100, it is possible to display a tracking motion image that only reflects the tracking object T (partial area R) selected by the viewer in the display area. As a result, even if a plurality of viewing objects W are reflected in the original motion image, the viewer can only view the tracking object T.

[0054] <Modifications> The present invention is not limited to the above embodiments. Various modifications can be made within the scope shown in the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0055] For example, the content processing device 2 or the terminal device 3A may be configured such that the information acquisition unit 234 acquires selection information before the motion picture generation unit 233 generates the tracking motion picture. Then, the motion picture generation unit 233 may also be configured to generate a tracking motion picture that only shows the tracked object T based on the selection information.

[0056] Furthermore, the content processing device 2 or the terminal device 3A can be configured to alternately generate position information and size information by the information generation unit 232 and generate tracking motion images by the motion image generation unit 233 frame by frame at the same period as the frame rate of the motion image. In this way, since the time delay from acquiring content data to playing the tracking motion image can be shortened, it can also be used for real-time video.

[0057] Furthermore, the content playback system 100 (content playback system 100A) may also have an authentication device for authenticating identification information. In this case, the terminal devices 3 and 3A may also be configured to accept the input of identification information and send it to the authentication device. Then, the authentication device may also be configured to send connection information for accessing the content server 1 to the terminal devices 3 and 3A if it is permitted to play content on the terminal devices 3 and 3A.

[0058] Furthermore, the content server 1 may also be configured to encrypt the content data and send the encrypted data to the content processing device 2 (terminal device 3A). In this case, the content playback system 100 (content playback system 100A) may have a key server that generates a decryption key for decrypting the encrypted data. Also, the content processing device 2 (terminal device 3A) may be configured to obtain the decryption key from the key server and use the decryption key to decrypt the encrypted data.

[0059] Furthermore, the aforementioned content processing program may also be recorded on one or more non-transitory, computer-readable recording media. The aforementioned device may or may not have such recording media. In the latter case, the aforementioned content processing program may be provided to the aforementioned device via any wired or wireless transmission medium.

[0060] Furthermore, some or all of the functions of the aforementioned control blocks can also be implemented through logic circuits. For example, integrated circuits that form logic circuits to perform the functions of the aforementioned control blocks are also included within the scope of this invention. Other examples include the functions of the control blocks being implemented by a quantum computer.

[0061] <Summary> The content processing apparatus related to the present invention, in aspect 1, comprises: a motion image acquisition unit that acquires content data including a motion image; an information generation unit that detects a plurality of viewing objects from the aforementioned motion image and generates position information for displaying the position of each feature of the plurality of viewing objects at a plurality of time points in the motion image, and size information for displaying the size of at least a portion of the viewing object containing the aforementioned feature; a motion image generation unit that, based on the aforementioned position information and the aforementioned size information, crops the aforementioned portion of each frame corresponding to the plurality of time points in the aforementioned motion image, corresponding to the plurality of viewing objects, and generates a tracking motion image for each frame composed of the cropped plurality of aforementioned portion of each viewing object; an information acquisition unit that acquires selection information for displaying the viewing object selected as the tracking object among the plurality of viewing objects; and an output unit that, based on the aforementioned selection information, outputs the aforementioned tracking motion image reflecting the aforementioned tracking object in the plurality of aforementioned tracking motion images.

[0062] The content processing apparatus related to the present invention in the above-mentioned state 1 may also be configured such that the aforementioned information generation unit takes the content data of the aforementioned viewing object as input and uses the learned model that outputs the aforementioned position information and the aforementioned size information of the viewing object to generate the aforementioned position information and the aforementioned size information.

[0063] The content processing apparatus related to the state 3 of the present invention, in the state 1 or 2 described above, may also be configured such that the object of viewing is a person and the feature part is the face of the person.

[0064] The content processing apparatus related to the state 4 of the present invention may be configured in any of the states 1 to 3 described above, such that the aforementioned information generation unit generates size information such that the ratio of the size of the aforementioned feature portion to the size of the aforementioned partial region is within a predetermined range.

[0065] The content processing apparatus related to the state 5 of the present invention, in the state 4 described above, may also be configured such that the aforementioned dynamic image generation unit generates the aforementioned tracking dynamic image, and the aforementioned tracking dynamic image is enlarged so that the aforementioned partial area is consistent with the display area of ​​the aforementioned dynamic image.

[0066] The content processing apparatus related to the state 6 of the present invention may be configured in any of the states 1 to 5 described above, wherein the aforementioned content includes sound.

[0067] The content processing apparatus related to the present invention in the above-described state 6 may also be configured such that: the viewing object is a person; and the content data includes data obtained by filming music activities.

[0068] The content processing apparatus related to the state 8 of the present invention, in the state 7 described above, may also be configured such that: the motion image generation unit performs processing to emphasize the components in the aforementioned sound that are equivalent to the sound emitted by the person who is the subject of the tracking or the sound emitted by an instrument played by that person; and the aforementioned output unit outputs the aforementioned tracking motion image together with the sound whose components have been emphasized.

[0069] The content processing program related to aspect 9 of the present invention is configured to cause a computer to perform: a motion image acquisition process, which acquires content data containing a motion image; a generation process, which detects a plurality of viewing objects from the aforementioned motion image and generates position information of the positions of each feature of the plurality of viewing objects at a plurality of time points for displaying the motion image, and size information of the size of at least a portion of the viewing object containing the aforementioned feature; a motion image generation process, which, based on the aforementioned position information and the aforementioned size information, crops out the aforementioned portion of each frame corresponding to the plurality of time points of the aforementioned motion image, corresponding to the plurality of viewing objects, and generates a tracking motion image for each frame composed of the cropped plurality of aforementioned portion of each viewing object; a result acquisition process, which acquires selection information of the viewing object selected as the tracking object among the plurality of viewing objects; and an output process, which, based on the aforementioned selection information, outputs the aforementioned tracking motion image reflecting the aforementioned tracking object in the plurality of aforementioned tracking motion images.

[0070] The content processing method related to aspect 10 of the present invention includes: a generation step, which uses a computer to detect a plurality of viewing objects from a dynamic image contained in the content, and generates position information of the positions of each feature of the plurality of viewing objects at a plurality of time points for displaying the dynamic image, and size information of the size of at least a portion of the viewing object containing the aforementioned feature; a dynamic image generation step, which uses a computer to crop out the aforementioned portion of each frame corresponding to the plurality of time points of the dynamic image based on the aforementioned position information and the aforementioned size information, and generates a tracking dynamic image for each frame composed of the cropped plurality of aforementioned portion of each viewing object; a result acquisition step, which uses a computer to acquire selection information of the viewing object selected as the tracking object among the plurality of viewing objects; and an output step, which uses a computer to output the aforementioned tracking dynamic image reflecting the aforementioned tracking object in the plurality of tracking dynamic images based on the aforementioned selection information. [Simplified Explanation of the Diagram]

[0010] Figure 1 is a block diagram showing an example of the functional configuration of a content playback system related to the first embodiment of the present invention. Figure 2 is an image diagram showing an example of the function of a receiving unit in a terminal device of the same system. Figure 3 is an image diagram showing an example of the function of an information generation unit in a content processing device of the same system. Figure 4 is a schematic diagram showing the tracking image status of a terminal device of the same system. Figure 5 is a block diagram showing an example of the functional configuration of a content playback system related to the second embodiment of the present invention. Figure 6 is a flowchart showing an example of the flow of a content processing method related to an embodiment of the present invention. [Biomaterial Storage]

[0072] None.

Claims

1. A content processing apparatus comprising: a motion image acquisition unit that acquires content data including a motion image; an information generation unit that detects a plurality of viewing objects from the motion image and generates position information for displaying the position of each feature of the plurality of viewing objects at a plurality of time points in the motion image, and size information for displaying the size of at least a portion of the viewing object including the aforementioned feature; a motion image generation unit that, based on the position information and the size information, crops the aforementioned portion of each frame corresponding to the plurality of time points in the motion image, and generates a tracking motion image for each frame consisting of the cropped plurality of the aforementioned portion of each viewing object; an information acquisition unit that acquires selection information for displaying a viewing object selected as a tracking object among the plurality of viewing objects; and an output unit that, based on the selection information, outputs the aforementioned tracking motion image reflecting the aforementioned tracking object in a plurality of the aforementioned tracking motion images. The aforementioned information generation unit will take the content data of the aforementioned viewing object as input, and use the learned model that outputs the aforementioned position information and the aforementioned size information of the viewing object to generate the aforementioned position information and the aforementioned size information.

2. The content processing apparatus as described in claim 1, wherein, The aforementioned object of observation is a person; the aforementioned feature is the face of the aforementioned person.

3. The content processing apparatus as described in claim 1, wherein, The aforementioned information generation unit generates size information so that the ratio of the size of the aforementioned feature to the size of the aforementioned partial region is within a predetermined range.

4. The content processing apparatus as described in claim 3, wherein, The aforementioned dynamic image generation unit generates the aforementioned tracking dynamic image, and enlarges the aforementioned tracking dynamic image so that the aforementioned partial area is consistent with the display area of ​​the aforementioned dynamic image.

5. The content processing apparatus as described in claim 1, wherein, The aforementioned content includes audio.

6. The content processing apparatus as described in claim 5, wherein, The aforementioned subject of observation is people; the aforementioned content includes information obtained from filming music events.

7. The content processing apparatus as described in claim 6, wherein, The motion image generation unit performs processing to emphasize components in the aforementioned sound that are equivalent to the sound emitted by the person being tracked or the sound emitted by an instrument played by that person; the aforementioned output unit outputs the aforementioned tracking motion image together with the sound whose components have been emphasized.

8. A content processing program that causes a computer to perform: a motion image acquisition process that acquires content data containing a motion image; a generation process that detects a plurality of viewing objects from the aforementioned motion image and generates position information of the positions of each feature of the plurality of viewing objects at a plurality of time points for displaying the motion image, and size information of the size of a portion of the viewing object containing at least the aforementioned feature; a motion image generation process that, based on the aforementioned position information and the aforementioned size information, crops the aforementioned portion of each frame corresponding to the plurality of time points of the aforementioned motion image, corresponding to the plurality of viewing objects, and generates a tracking motion image for each frame composed of the cropped plurality of aforementioned portion of each viewing object; a result acquisition process that acquires selection information of the viewing object selected as the tracking object among the plurality of viewing objects; and an output process that, based on the aforementioned selection information, outputs the aforementioned tracking motion image reflecting the aforementioned tracking object in the plurality of aforementioned tracking motion images; In the aforementioned generation process, the computer takes the content data of the object being viewed as input and uses the learned model that outputs the position information and size information of the object being viewed to generate the position information and size information.

9. A content processing method, comprising: a generation step, wherein a computer detects a plurality of viewing objects from a moving image contained in the content, and generates position information of the positions of each feature of the plurality of viewing objects at a plurality of time points for displaying the moving image, and size information of the size of a portion of the viewing object containing at least the aforementioned feature; a moving image generation step, wherein a computer, based on the aforementioned position information and the aforementioned size information, respectively crops the aforementioned portion of each frame corresponding to the plurality of time points of the moving image, corresponding to the plurality of viewing objects, and generates a tracking moving image for each frame composed of the cropped plurality of aforementioned portion of each viewing object; a result acquisition step, wherein a computer acquires selection information of the viewing object selected as the tracking object among the plurality of viewing objects; and an output step, wherein a computer, based on the aforementioned selection information, outputs the aforementioned tracking moving image reflecting the aforementioned tracking object in the plurality of aforementioned tracking moving images; In the aforementioned generation step, the computer takes the content data of the object being viewed as input and uses the learned model that outputs the position information and size information of the object being viewed to generate the position information and size information.

Images