Information processing device and information processing method

Abstract

The present invention enables emotion data, which represents user emotion for each scene of moving image content, to be effectively used.　A representative emotion scene is extracted by an extraction unit on the basis of emotion metadata having user emotion information for each scene of the moving image content. On the basis of the extracted representative emotion scene, playing back a portion of the moving image content or editing for taking out a portion of the moving image content can be effectively performed. For example, the extraction unit extracts the representative emotion scene on the basis of the type or degree of user emotion.

Description

Technical Field

[0001] The present technology relates to an information processing apparatus and an information processing method, and more particularly to an information processing apparatus that processes information related to video content.

Background Art

[0002] Conventionally, various techniques have been proposed for generating emotion data indicating user emotions for each scene of video content based on a user's face image, biometric information of the user, and the like (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present technology is to effectively utilize emotion data indicating user emotions for each scene of video content.

Means for Solving the Problems

[0005] The concept of the present technology is an information processing apparatus including an extraction unit that extracts an emotion representative scene based on emotion metadata having user emotion information for each scene of video content. In the present technology, the extraction unit extracts an emotion representative scene based on emotion metadata having user emotion information for each scene of video content. For example, the extraction unit may be configured to extract an emotion representative scene based on the type of user emotion.

[0006]

[0007] ​Furthermore, for example, the extraction unit may extract emotionally representative scenes based on the degree of user emotion. In this case, for example, the extraction unit may extract scenes where the degree of user emotion exceeds a threshold as emotionally representative scenes. Alternatively, in this case, for example, the extraction unit may extract emotionally representative scenes based on statistical values ​​of the degree of user emotion in the entire video content. Here, the statistical values ​​may include, for example, the maximum value, sorting result, mean, or standard deviation.

[0008] In this technology, representative emotional scenes are extracted based on emotional metadata that contains user emotion information for each scene of the video content. This makes it possible to effectively utilize emotional data that indicates user emotions for each scene of the video content during video playback and editing.

[0009] Furthermore, this technology may include, for example, a playback control unit that plays back the extracted emotional representative scenes from the video content. This would allow the user to view only the extracted emotional representative scenes.

[0010] Furthermore, this technology may also include an editing control unit that extracts emotionally representative scenes from video content and generates new video content. This allows the user to obtain new video content that includes only the extracted emotionally representative scenes.

[0011] Furthermore, this technology may also include, for example, a display control unit that displays the time position of the extracted emotional representative scene relative to the entire video content. This makes it easy for the user to recognize the time position of the extracted emotional representative scene relative to the entire video content.

[0012] In this case, for example, the display control unit may display the type and degree of user emotion in the extracted emotional representative scene at the time position corresponding to the extracted emotional representative scene on a time axis slider bar corresponding to the entire video content. In this case, the user can recognize the time position of the extracted emotional representative scene relative to the entire video content by the position on the time axis slider bar, and can also easily recognize the type and degree of user emotion in the extracted emotional scene.

[0013] Here, for example, the display control unit may display the type of user emotion using a mark. This allows the user to intuitively recognize the type of emotion from the mark. [Brief explanation of the drawing]

[0014] [Figure 1] This block diagram shows an example configuration of an information processing device that generates emotion metadata. [Figure 2] This block diagram shows another example configuration of an information processing device that generates emotional metadata. [Figure 3] This is a block diagram showing an example configuration of an information processing device that utilizes emotion metadata. [Figure 4] This diagram illustrates how scenes where the degree of user emotion exceeds a threshold are extracted as representative emotional scenes. [Figure 5] This diagram illustrates how to extract emotionally representative scenes based on statistical values ​​of the overall level of user emotion in video content. [Figure 6] This diagram illustrates an example of how emotionally representative scenes are positioned within the overall video content. [Figure 7] This block diagram shows another example configuration of an information processing device that utilizes emotional metadata. [Modes for carrying out the invention]

[0015] Hereinafter, embodiments for implementing the invention (hereinafter referred to as "embodiments") will be described. The description will be given in the following order. 1. Embodiment 2. Variation

[0016] <1. Embodiment> [Configuration Example of Information Processing Apparatus for Generating Emotion Metadata] FIG. 1 shows a configuration example of an information processing apparatus 100A that generates emotion metadata. This information processing apparatus 100A includes a content database (content DB) 101, a content playback display unit 102, a face image capture camera 103, a biometric information sensor 104, a user emotion analysis unit 105, a metadata generation unit 106, and a metadata rewriting unit 107.

[0017] The content database 101 stores a plurality of video content files. When a playback video file name is input, the content database 101 supplies the video content file corresponding to the playback video file name to the content playback display unit 102. Here, the playback video file name is specified, for example, by the user of this information processing apparatus 100A.

[0018] During playback, the content playback display unit 102 plays the video content included in the video content file supplied from the content database 101 and displays the video on a display unit (not shown). Also, during playback, the content playback display unit 102 supplies a frame number (time code) to the metadata generation unit 106 in synchronization with the playback frame. This frame number is information that can identify the scene of the video content.

[0019] The face image capture camera 103 is a camera that captures the face image of the user viewing the video displayed on the display unit by the content playback display unit 102. The face images of each frame obtained by capturing with this face image capture camera 103 are sequentially supplied to the user emotion analysis unit 105.

[0020] The biometric information sensor 104 is attached to the user who is watching the video displayed on the content playback display unit 102, and is a sensor for acquiring biometric information such as heart rate, respiratory rate, and sweating amount. The biometric information for each frame acquired by this biometric information sensor 104 is sequentially supplied to the user emotion analysis unit 105.

[0021] The user emotion analysis unit 105 analyzes the degree of a predetermined type of user emotion for each frame based on the face images of each frame sequentially supplied from the face image capture camera 103 and the biometric information of each frame sequentially supplied from the biometric information sensor 104, and supplies the user emotion information to the metadata generation unit 106.

[0022] Furthermore, the types of user emotions are not limited to secondary information obtained by analyzing facial images and biometric data, such as information on "joy," "anger," "sadness," and "happiness," but may also be primary information such as heart rate, respiratory rate, and sweating volume.

[0023] The metadata generation unit 106 associates the user sentiment information for each frame obtained by the user sentiment analysis unit 105 with the frame number (time code) to generate sentiment metadata containing user sentiment information for each frame of the video content, and supplies this sentiment metadata to the metadata rewriting unit 107.

[0024] If the video content file corresponding to the playback video file name does not yet have emotion metadata attached, the metadata rewriting unit 107 attaches the emotion metadata supplied by the metadata generation unit 106 as is. If the video content file corresponding to the playback video file name already has emotion metadata attached, the metadata rewriting unit 107 updates it with the emotion metadata supplied by the metadata generation unit 106.

[0025] Alternatively, if the video content file corresponding to the playback video file name already has sentiment metadata attached, the metadata rewriting unit 107 updates it with sentiment metadata obtained by combining the already attached sentiment metadata with the sentiment metadata supplied by the metadata generation unit 106. A weighted average is a possible method of combination, but it is not limited to this and other methods may be used. In the case of a weighted average, if the already attached sentiment metadata pertains to m users, the already attached sentiment metadata and the sentiment metadata supplied by the metadata generation unit 106 are weighted m:1 and averaged.

[0026] When updating with sentiment metadata obtained through this synthesis process, the more users who view the video content, the more the sentiment metadata will be updated, resulting in more accurate sentiment metadata that will be more useful when playing and editing video content.

[0027] As shown in Figure 1, the information processing device 100A generates emotion metadata containing user emotion information for each frame of the video content, and attaches this emotion metadata to the video content file. This emotion metadata can then be used when playing and watching the video content or when editing the video content.

[0028] Figure 2 shows an example configuration of the information processing device 100B that generates emotion metadata. In Figure 2, parts corresponding to those in Figure 1 are denoted by the same reference numerals, and detailed explanations are omitted where appropriate.

[0029] This information processing device 100B includes a content database (content DB) 101, a content playback and display unit 102, a face image capture camera 103, a biometric information sensor 104, a user emotion analysis unit 105, a metadata generation unit 106, and a metadata database (metadata DB) 108.

[0030] The metadata generation unit 106 associates the user sentiment information for each frame obtained by the user sentiment analysis unit 105 with the frame number (time code) to generate sentiment metadata containing user sentiment information for each frame of the video content, and supplies this sentiment metadata to the metadata database 108.

[0031] The metadata database 108 stores sentiment metadata corresponding to multiple video content files. The metadata database 108 stores the sentiment metadata supplied from the metadata generation unit 106 in a database along with the video file name so that it can identify which video content file the sentiment metadata corresponds to; in other words, it stores it linked to the video file name. If the metadata database 108 has not yet stored sentiment metadata corresponding to the playback video file name, it stores the sentiment metadata supplied from the metadata generation unit 106 as is. If the metadata database 108 has already stored sentiment metadata corresponding to the playback video file name, it updates it with the sentiment metadata supplied from the metadata generation unit 106.

[0032] Alternatively, if the metadata database 108 already stores emotion metadata corresponding to the playback video file name, it updates the existing emotion metadata with emotion metadata obtained by combining it with emotion metadata supplied from the metadata generation unit 106. While a detailed explanation is omitted, the method of combination is the same as in the case of the metadata rewriting unit 107 in the information processing device 100A shown in Figure 1 above.

[0033] In the illustrated example, the association between sentiment metadata stored in the metadata database 108 and video content files stored in the content database 101 is shown using the video file name. However, it is also possible to link them using other methods, such as link information such as URLs. In this case, for example, the link is established by recording link information, such as a URL for accessing the sentiment metadata stored in the metadata database 108, as metadata within the corresponding video content file in the content database 101.

[0034] The other components of the information processing device 100B shown in Figure 2 are configured in the same way as those of the information processing device 100A shown in Figure 1.

[0035] As shown in Figure 2, the information processing device 100B generates emotion metadata containing user emotion information for each frame of the video content, and stores this emotion metadata in the metadata database 108, linked to the video content file. This makes it possible to use this emotion metadata when playing and watching the video content or when editing the video content.

[0036] Also 、 In this information processing device 100B, emotion metadata corresponding to multiple video content files is stored in the metadata database 108, and as shown in the information processing device 100A in Figure 1, Tsude ta base Compared to adding sentiment metadata to video content files stored in 101, this method eliminates the need to extract sentiment metadata from video content files, making it possible to process data efficiently, especially when performing some kind of analysis using only sentiment metadata.

[0037] [Example configuration of an information processing device that utilizes emotional metadata] Figure 3 shows an example configuration of an information processing device 200A that utilizes emotion metadata. This information processing device 200A includes a content database (content DB) 201, a content playback / editing unit 202, a metadata extraction unit 203, and an emotion representative scene extraction unit 204.

[0038] Content database 201 corresponds to content database 101 shown in Figure 1, and stores multiple video content files. Each video content file also has sentiment metadata attached to it, which contains user sentiment information for each frame of the video content.

[0039] The content database 201, upon receiving the filename of a video to be played, supplies the video content file corresponding to that filename to the content playback / editing unit 202 and the metadata extraction unit 203. Here, the filename of the video to be played is specified, for example, by the user of this information processing device 200A.

[0040] The metadata extraction unit 203 extracts emotion metadata from video content files supplied from the content data database 201 and supplies it to the emotion representative scene extraction unit 204. The emotion representative scene extraction unit 204 extracts emotion representative scenes from the emotion metadata supplied from the metadata extraction unit 203.

[0041] For example, the emotion representative scene extraction unit 204 extracts emotion representative scenes based on the type of user emotion. In this case, for example, if the emotion metadata contains information on "joy," "anger," "sadness," and "happiness" as user emotion information for each frame of the video content, it selects one of these emotions and extracts scenes where the degree (level) of that emotion is above a threshold as emotion representative scenes. Here, the selection of emotions and the setting of thresholds can be arbitrarily performed, for example, by user operation.

[0042] Furthermore, for example, the emotion representative scene extraction unit 204 extracts emotion representative scenes based on the degree of user emotion. In this case, possible methods include (1) extracting scenes where the degree of user emotion exceeds a threshold as emotion representative scenes, or (2) extracting emotion representative scenes based on a statistical value of the overall degree of user emotion in the video content.

[0043] First, we will explain the case where (1) scenes in which the degree of user emotion exceeds a threshold are extracted as representative emotional scenes. In this case, for example, if the emotion metadata contains information on "joy," "anger," "sadness," and "happiness" as user emotion information for each frame of the video content, then for each emotion, scenes in which the degree (level) exceeds the threshold will be extracted as representative emotional scenes. Here, the threshold can be set arbitrarily, for example, by user operation.

[0044] Figure 4(a) shows an example of the frame-by-frame change in a given degree (level) of user emotion. Here, the horizontal axis represents the frame number fr, and the vertical axis represents the degree of user emotion Em(fr). In this example, since the degree Em(fr_a) exceeds the threshold th at frame number fr_a, frame number fr_a is stored as emotion representative scene information L(1), and since the degree Em(fr_b) exceeds the threshold th at frame number fr_b, frame number fr_b is stored as emotion representative scene information L(2).

[0045] The flowchart in Figure 4(b) shows an example of the processing procedure of the emotion representative scene extraction unit 204 when extracting scenes in which the degree of user emotion exceeds a threshold as emotion representative scenes.

[0046] First, the emotion representative scene extraction unit 204 starts processing in step ST1. Next, in step ST2, the emotion representative scene extraction unit 204 initializes frame numbers fr=1 and n=1.

[0047] Next, in step ST3, the emotion representative scene extraction unit 204 determines whether the degree Em(fr) is greater than the threshold th. If Em(fr) > th, in step ST4, the emotion representative scene extraction unit 204 stores the emotion representative scene information, that is, the frame number fr as the emotion representative scene L(n). Also in step ST4, the emotion representative scene extraction unit 204 increments n to n+1.

[0048] Next, in step ST5, the emotion representative scene extraction unit 204 updates the frame number fr to fr = fr + 1. Similarly, if Em(fr) > th is not true in step ST3, the frame number fr is updated in step ST5.

[0049] Next, in step ST6, the emotion representative scene extraction unit 204 determines whether the frame number fr is greater than the last frame number fr_end, that is, it determines the end of the process. If fr > fr_end is not true, the emotion representative scene extraction unit 204 returns to the process in step ST3 and repeats the same process as described above. On the other hand, if fr > fr_end, the emotion representative scene extraction unit 204 terminates the process in step ST7.

[0050] Next, we will explain (2) the case where representative emotional scenes are extracted based on statistical values ​​of the overall degree of user emotion in the video content. In this case, the statistical values ​​may be the maximum value, sorting result, mean, or standard deviation.

[0051] When the statistical value is at its maximum, for example, if the emotion metadata contains information on user emotions such as "joy," "anger," "sadness," and "happiness" for each frame of the video content, the scene where the degree (level) of each emotion is at its maximum will be extracted as the representative scene for that emotion.

[0052] Furthermore, when statistical values ​​are sorting results, for example, if the emotion metadata contains information on user emotions such as "joy," "anger," "sadness," and "happiness" for each frame of video content, then for each emotion, not only the scene with the highest degree (level) but also the scenes ranked second, third, and so on will be extracted as representative scenes of that emotion.

[0053] Furthermore, when statistical values ​​are the mean or standard deviation, for example, if the emotional metadata contains information on user emotions such as "joy," "anger," "sadness," and "happiness" for each frame of video content, then scenes where the degree (level) of each emotion deviates significantly from the average (for example, three times the standard deviation) will be extracted as representative scenes of that emotion.

[0054] Figure 5(a) shows an example of the frame-by-frame change in a given degree (level) of user emotion. Here, the horizontal axis represents the frame number fr, and the vertical axis represents the degree of user emotion Em(fr). In this example, the degree Em(fr_a) for frame number fr_a becomes the maximum value em_max, so frame number fr_a is stored as the emotion representative scene information L.

[0055] The flowchart in Figure 5(b) shows an example of the processing procedure of the emotion representative scene extraction unit 204 when extracting a scene that represents the maximum level of user emotion in the overall video content as an emotion representative scene.

[0056] First, the emotion representative scene extraction unit 204 starts processing in step ST11. Next, in step ST12, the emotion representative scene extraction unit 204 initializes the frame number fr=1 and the maximum value em_max=0.

[0057] Next, in step ST13, the emotion representative scene extraction unit 204 determines whether the degree Em(fr) is greater than the maximum value em_max. If Em(fr) > em_max, in step ST14, the emotion representative scene extraction unit 204 stores the emotion representative scene information, that is, stores the frame number fr as the emotion representative scene L. Also in step ST14, the emotion representative scene extraction unit 204 updates em_max to Em(fr).

[0058] Next, in step ST15, the emotion representative scene extraction unit 204 updates the frame number fr to fr = fr + 1. Similarly, if Em(fr) > em_max is not true in step ST13, the frame number fr is updated in step ST15.

[0059] Next, in step ST16, the emotion representative scene extraction unit 204 determines whether the frame number fr is greater than the last frame number fr_end, that is, it determines the end of the process. If fr > fr_end is not true, the emotion representative scene extraction unit 204 returns to the process in step ST13 and repeats the same process as described above. On the other hand, if fr > fr_end, the emotion representative scene extraction unit 204 terminates the process in step ST17.

[0060] Returning to Figure 3, the emotion representative scene extraction unit 204 supplies emotion representative scene information to the content playback / editing unit 202. The content playback / editing unit 202 plays the video content contained in the video content file supplied from the content database 201.

[0061] In this case, the content playback / editing unit 202 can play a portion of the video content contained in the video content file supplied from the content database 201, either in response to user input or automatically.

[0062] When playback is automatic, for example, a control unit (not shown) controls the playback to play back the emotional representative scenes extracted by the emotional representative scene information extraction unit 204 based on the emotional representative scene information. This allows the user to view only the extracted emotional representative scenes.

[0063] Furthermore, when playback is performed in response to user input, for example, a control unit (not shown) controls the system to display the position of the emotional representative scene extracted by the emotional representative scene information extraction unit 204 relative to the entire video content, for the user's convenience. This allows the user to easily recognize the time position of the extracted emotional representative scene relative to the entire video content, enabling efficient playback operations, and for example, allowing efficient playback of only the extracted emotional representative scene.

[0064] Furthermore, the content playback / editing unit 202 edits the video content contained in the video content files supplied from the content database 201, either in response to user input or automatically, to generate new video content.

[0065] When editing automatically, for example, a control unit (not shown) controls the system to extract the representative emotional scenes extracted by the representative emotional scene information extraction unit 204 based on the representative emotional scene information, and generate new video content. This makes it possible to automatically obtain new video content that includes only the extracted representative emotional scenes.

[0066] Furthermore, when editing in response to user operations, for example, for the convenience of the user, how the emotional representative scenes extracted by the emotional representative scene information extraction unit 204 relate to the entire video content timeA control unit (not shown) is used to indicate the location of the extracted emotional representative scene. This allows the user to easily recognize the time position of the extracted emotional representative scene within the overall video content, enabling efficient editing operations. For example, it becomes possible to efficiently obtain new video content containing only the extracted emotional representative scene.

[0067] Figure 6(a) shows an example of how the emotional representative scene extracted by the emotional representative scene information extraction unit 204 is positioned within the overall video content. In this example, a time axis slider bar 301 indicating the progress of video content playback is displayed at the bottom, and the playback video 302 is displayed at the top.

[0068] This time axis slider 301 corresponds to the entire video content, and at the time position on this time axis slider 301 corresponding to the emotional representative scene extracted by the emotional representative scene information extraction unit 204, the type and degree of user emotion in that emotional representative scene is displayed. In this case, the user can recognize the time position of the extracted emotional representative scene relative to the entire video content by the position on the time axis slider, and the extracted emotional scene to The type and degree of user emotion can also be easily recognized.

[0069] In this example, the type is indicated by a mark (icon) so that the user can intuitively recognize it, and the degree is indicated by a numerical value, but the display method is not limited to this.

[0070] Furthermore, at the time position corresponding to the representative emotion scene extracted by the emotion representative scene information extraction unit 204, the type and degree of user emotion in that representative emotion scene are displayed. ofAlternatively, as shown in Figure 6(b), user emotion information for each frame of the video content can be displayed directly. In the illustrated example, only "sad" and "happy" information is shown for the sake of simplicity. In this case, as shown by the dashed line in Figure 3, the emotion metadata extracted by the metadata extraction unit 203 is supplied to the content playback / editing unit 202, and the display is performed based on this emotion metadata.

[0071] As shown in Figure 3, the information processing device 200A extracts representative emotional scenes using the emotion representative scene information extraction unit 204 based on emotion metadata that contains user emotion information for each frame of the video content. This makes it possible to effectively utilize emotion data that indicates user emotions for each frame of the video content during playback and editing of the video content.

[0072] Figure 7 shows an example configuration of the information processing device 200B that utilizes emotion metadata. In Figure 7, parts corresponding to those in Figure 3 are denoted by the same reference numerals, and detailed explanations are omitted where appropriate.

[0073] This information processing device 200B includes a content database (content DB) 201, a content playback / editing unit 202, a metadata database (metadata DB) 205, and an emotion representative scene extraction unit 204.

[0074] Metadata database 205 corresponds to metadata database 108 shown in Figure 2 and stores sentiment metadata associated with multiple video content files stored in content database 201. In this example, the association is shown as being done by the video file name.

[0075] The metadata database 205 receives the same playback video file name as that entered into the content database 201, and then supplies the emotion metadata associated with the video content file supplied from the content database 201 to the content playback / editing unit 202 to the emotion representative scene extraction unit 204.

[0076] The emotion representative scene extraction unit 204 extracts emotion representative scenes from emotion metadata supplied from the metadata database 205 and supplies the emotion representative scene information to the content playback / editing unit 202.

[0077] The other components of the information processing device 200B shown in Figure 7 are configured in the same way as those of the information processing device 200A shown in Figure 3. The same effects can be obtained with this information processing device 200B as with the information processing device 200A shown in Figure 3.

[0078] <2. Variant> In the embodiment described above, an example was shown where the emotion metadata contained user emotion information for each frame of the video content. In other words, an example was shown where each scene consisted of one frame. However, it is also conceivable that the emotion metadata could be configured to contain user emotion information for multiple frames, rather than per frame. In this case, each scene would consist of multiple frames. This would make it possible to reduce the amount of emotion metadata data.

[0079] Furthermore, in the above-described embodiment, it was explained that when generating emotional metadata, more accurate emotional metadata can be obtained by having multiple users sequentially view video content and update the emotional metadata. However, it is also conceivable that highly accurate emotional metadata can be obtained all at once by inputting facial images and biometric information related to multiple users into the user emotion analysis unit 105 and performing the analysis.

[0080] Furthermore, while sentiment metadata generated by the observation of a single user contains the sentiment information of that single user, sentiment metadata generated by the observation of multiple users contains sentiment information that is statistically representative of the sentiment responses of those other users.

[0081] Furthermore, although not mentioned above, it is also conceivable that sentiment metadata could be generated separately for each generation, gender, country, etc., and that the differences between these attributes could be used for playback and editing.

[0082] Furthermore, while preferred embodiments of this disclosure have been described in detail with reference to the accompanying drawings, the technical scope of this disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of this disclosure that various modifications or alterations may be conceived within the scope of the technical idea set forth in the claims, and these too will naturally fall within the technical scope of this disclosure.

[0083] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0084] Furthermore, this technology can also be configured as follows: (1) The video content includes an extraction unit that extracts representative emotional scenes based on emotional data representing user emotions for each scene. Information processing device. (2) The extraction unit extracts the representative scene of the emotion based on the type of user emotion. The information processing device described in (1) above. (3) The extraction unit extracts the emotional representative scene based on the degree of the user's emotion. The information processing device described in (1) above. (4) The extraction unit extracts scenes in which the degree of user emotion exceeds a threshold as the representative emotion scene. The information processing device described in (3) above. (5) The extraction unit extracts the emotional representative scenes based on the statistical value of the degree of user emotion of the entire video content. The information processing device described in (3) above. (6) The statistical values ​​include the maximum value, sorting result, mean, or standard deviation. The information processing device described in (5) above. (7) Further comprising a playback control unit that plays the extracted emotional representative scene from the video content An information processing device according to any one of (1) to (6) above. (8) The video content further comprises an editing control unit that extracts the emotional representative scenes from the video content and generates new video content. An information processing device according to any one of (1) to (7) above. (9) The system further includes a display control unit that displays the time position of the extracted emotional representative scene relative to the entire video content. An information processing device according to any of (1) to (8) above. (10) The display control unit displays the type and degree of user emotion in the extracted emotional representative scene at the time position corresponding to the extracted emotional representative scene on the time axis slider bar corresponding to the entire video content. The information processing device described in (9) above. (11) The display control unit displays the type of user emotion with a mark. The information processing device described in (10) above. (12) Having a procedure for extracting emotionally representative scenes based on emotional data that represents user emotions for each scene of video content. Information processing methods. [Explanation of symbols]

[0085] 100A, 100B... Information processing equipment 101. Content Database (Content DB) 102...Content Playback Display Unit 103...Face image capture camera 104... Biometric Information Sensor 105...User Sentiment Analysis Department 106...Metadata generation unit 107...Metadata rewriting section 108. Metadata Database (Metadata DB) 200A, 200B... Information processing equipment 201...Content Database (Content DB) 202...Content Playback / Editorial Department 203...Metadata Extraction Section 204... Emotional Representative Scene Extraction Section 205. Metadata Database (Metadata DB)

Claims

1. An extraction unit that extracts emotionally representative scenes from the video content based on emotional data containing user emotional information for each scene of the video content, The system includes a display control unit that displays the time position of the extracted emotional representative scene relative to the entire video content, The user sentiment information includes information on the type and degree of user sentiment, The display control unit displays the type and degree of user emotion in the extracted emotional representative scene at the time position corresponding to the extracted emotional representative scene on the time axis slider bar corresponding to the entire video content. Information processing device.

2. The user emotion is obtained by analyzing the facial image and biometric information of the user viewing the video content. The information processing apparatus according to claim 1.

3. The display control unit displays the type of user emotion with a mark. The information processing apparatus according to claim 1.

4. The extraction unit extracts the representative emotion scene based on the type of user emotion. The information processing apparatus according to claim 1.

5. The extraction unit extracts the emotional representative scene based on the degree of the user's emotion. The information processing apparatus according to claim 1.

6. The extraction unit extracts scenes in which the degree of user emotion exceeds a threshold as the representative emotion scene. The information processing apparatus according to claim 5.

7. The extraction unit extracts the emotional representative scenes based on statistical values ​​of the degree of user emotion in the entire video content. The information processing apparatus according to claim 5.

8. The aforementioned statistical values ​​include the maximum value, sorting result, mean, or standard deviation. The information processing apparatus according to claim 7.

9. The system further comprises a playback control unit that plays the extracted emotional representative scenes from the video content. The information processing apparatus according to claim 1.

10. The system further includes an editing control unit that extracts the emotionally representative scenes from the aforementioned video content and generates new video content. The information processing apparatus according to claim 1.

11. An extraction procedure for extracting emotionally representative scenes from video content based on emotional data containing user emotional information for each scene of the video content, The display procedure includes indicating the time position of the extracted emotional representative scene within the entire video content, The user sentiment information includes information on the type and degree of user sentiment, In the display procedure described above, the type and degree of user emotion in the extracted emotional representative scene are displayed at the time position corresponding to the extracted emotional representative scene on the time axis slider bar corresponding to the entire video content. Information processing methods.

Images