Information processing apparatus and non-volatile storage medium

Abstract

The present application provides an information processing apparatus and a program capable of effectively generating time-shortened content. The information processing apparatus receives content data and processes the content data. The information processing apparatus comprises: an acquisition part, which stores sound data acquired from the content data into a storage area; an inference part, which performs sound source separation processing by using the sound data stored in the storage area, and outputs sound source data; and a generation part, which generates, on the basis of the outputted sound source data, time-shortened content in which the playback time of the content data is shortened.

Description

Information processing devices and non-volatile storage media

[0001] Cross-references to related applications

[0002] This application claims priority to Japanese Patent Application No. 2024-201983, filed on November 20, 2024, entitled "Information Processing Apparatus and Program", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to information processing apparatus and non-volatile storage media. Background Technology

[0004] It is known that content such as animated images can be edited based on information collected from external sources, thereby enabling the creation of time-lapse content that shortens the playback time.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent document 1: Japanese Patent Application Publication No. 2017-212626. Summary of the Invention

[0008] However, as in Patent Document 1, when time-lapse content is generated using information provided by multiple users who viewed the content after its publication has ended, it can sometimes take time to generate the time-lapse content. Additionally, sometimes content cannot be published to multiple viewers due to copyright issues.

[0009] The problem this application aims to solve is to provide an information processing device and program that can effectively generate time-lapse content.

[0010] The information processing apparatus of the embodiment receives and processes content data. The information processing apparatus includes: an acquisition unit that stores sound data acquired from the content data into a storage area; an inference unit that performs sound source separation processing using the sound data stored in the storage area and outputs sound source data; and a generation unit that generates time-lapse content that shortens the playback time of the content data based on the output sound source data. Attached Figure Description

[0011] Figure 1 is a diagram illustrating an example of the overall configuration of an information processing apparatus including an embodiment;

[0012] Figure 2 is a diagram illustrating the sound source separation and playlist of the implementation method;

[0013] Figure 3 is a schematic diagram illustrating the generated playlist of the implementation method;

[0014] Figure 4 is a diagram illustrating the gain adjustment of the sound source data in the implementation method;

[0015] Figure 5 is a flowchart illustrating an example of the process for generating a playlist in an embodiment;

[0016] Figure 6 is a flowchart illustrating an example of the gain adjustment process in an implementation.

[0017] Explanation of reference numerals in the attached figures: 1…Information processing device, 100…Dynamic image stream parsing unit, 103…Metadata parsing unit, 200…Sound source separation AI processing unit, 201…DNN input buffer management unit (acquisition unit), 202…Sound source separation DNN inference unit (inference unit), 203…Sound source separation result cache, 204…Sound source separation result parsing unit (generation unit), 205…Playlist DB, 206…DNN model storage unit, 300…Playback control unit, 301…Video decoder, 302…Audio decoder, 303…Special playback control unit (control unit), 304…Audio mixer, 400…Content DB, 500…UI setting unit. Detailed Implementation

[0018] Hereinafter, the information processing apparatus and program of this embodiment will be described with reference to the accompanying drawings.

[0019] Figure 1 is a diagram showing an example of the overall configuration of the information processing apparatus 1 including the embodiment. As shown in Figure 1, the information processing apparatus 1 includes a dynamic image stream parsing unit 100, a sound source separation AI (artificial intelligence) processing unit 200, a playback control unit 300, and a UI (User Interface) setting unit 500. In addition, the information processing apparatus 1 is connected to the content DB 400.

[0020] In this embodiment, the information processing device 1 will be described as an example of a device built into an edge device such as a television device.

[0021] The dynamic image stream parsing unit 100, for example, decrypts or parses video data, audio data, and service information (SI), subtitle data, etc., contained in content data received using a receiver for receiving broadcast waves (not shown) or a communication device (not shown). Furthermore, the content data may also be a broadcast wave.

[0022] Here, "video data" refers to, for example, video data that includes both moving image data (such as sports or movies) and sound data. "Sound data" refers to, for example, sound data that is linked to moving image data, such as cheers, dialogue, or instrument sounds. "Service information" refers to, for example, program arrangement information such as the start time of content or the type of content as an attribute. "Subtitle data" refers to, for example, data containing cheers or dialogue within the sound data displayed as text.

[0023] The dynamic image stream parsing unit 100 includes a video decoder 101, an audio decoder 102, and a metadata parsing unit 103.

[0024] Specifically, for example, if the received content data is encrypted, the video decoder 101 decrypts the video data contained in the content data.

[0025] Specifically, for example, if the received content data is encrypted, the audio decoder 102 decrypts the sound data contained in the content data. The audio decoder 102 then sends the decrypted sound data to the DNN (Deep Neural Network) input buffer management unit 201. Conversely, if the received content data is not encrypted, the sound data is sent to the DNN input buffer management unit 201 without decryption.

[0026] Metadata parsing unit 103, for example, obtains service information contained in content data and uses the obtained service information to determine the type of content.

[0027] The metadata parsing unit 103 sends the determined content type, etc., to the sound source separation DNN inference unit 202. Additionally, the metadata parsing unit 103 sends the determined content type, etc., to the sound source separation result parsing unit 204. The content type, etc., includes, for example, sports, movies, dramas, music, etc.

[0028] Next, the sound source separation AI processing unit 200 performs sound source separation processing on the sound data sent from the dynamic image stream parsing unit 100, separating it into multiple sound source data. Furthermore, the sound source separation AI processing unit 200 uses information such as the sound pressure level of the separated sound source data to generate time-lapsed content (hereinafter referred to as a playlist) that can shorten the playback time of the content data.

[0029] The sound source separation AI processing unit 200 includes a DNN input buffer management unit 201, a sound source separation DNN inference unit 202, a sound source separation result cache 203, a sound source separation result parsing unit 204, a playlist DB (Data Base) 205, and a DNN model storage unit 206.

[0030] The DNN input buffer management unit 201 acquires sound data from the audio decoder 102. Furthermore, the acquired sound data is stored in a storage area (not shown) provided in the DNN input buffer management unit 201. The DNN input buffer management unit 201 is also referred to as the acquisition unit.

[0031] The sound source separation DNN inference unit 202 acquires sound data from a storage area (not shown) provided in the DNN input buffer management unit 201. Additionally, it acquires information such as the content type from the metadata parsing unit 103. Based on the acquired content type, the sound source separation DNN inference unit 202 selects and acquires an inference model corresponding to the content type from the DNN model storage unit 206. Furthermore, the inference model is also referred to as a DNN model.

[0032] The sound source separation DNN inference unit 202 uses the selected inference model and the acquired sound data to generate multiple sound source data after separating the acquired sound data according to the specified sound sources.

[0033] In addition, the sound source separation DNN inference unit 202 outputs the separated multiple sound source data to the sound source separation result buffer 203. Furthermore, the sound source separation DNN inference unit 202 is also referred to as the inference unit.

[0034] To ensure sufficient separation performance of the DNN inference model, a constant amount of sound data needs to be collected and applied to the DNN inference model. Therefore, after the sound source separation DNN inference unit 202 accumulates a predetermined amount of sound data in the storage area (not shown) of the DNN input buffer management unit 201, it uses the DNN inference model to separate the sound source data into at least one sound source. Then, the separated at least one sound source data is output to the sound source separation result buffer 203.

[0035] The phrase "the acquired audio data is above a specified amount" refers to, for example, the amount of audio data at a point in time after the content data has been published. Specifically, it could be the amount of audio data at a point in time after the reception has been completed (e.g., 8 seconds or more).

[0036] In addition, the sound source separation DNN inference unit 202 is executed by a CPU (Central Processing Unit), which uses parameters stored in the DNN model storage unit 206 and the like to perform prescribed calculation and control processing.

[0037] The sound source separation result cache 203 stores the sound source data separated by the sound source separation DNN inference unit 202. Furthermore, the separated sound source data is also stored after the playlist is generated.

[0038] The sound source separation result parsing unit 204 obtains the sound source data stored in the sound source separation result cache 203. Additionally, it obtains the content type from the metadata parsing unit 103. Furthermore, although not shown, the sound source separation result parsing unit 204 can also obtain an inference model from the DNN model storage unit 206 based on the obtained content type.

[0039] Furthermore, the sound source separation result analysis unit 204 generates a playlist, for example, based on the acquired sound source data and service information. For instance, it generates a playlist based on the types of content included in the content's service information. Then, the generated playlist is output to the playlist DB205. Additionally, the sound source separation result analysis unit 204 is also referred to as the generation unit.

[0040] Playlists, for example, can be exemplified by time-lapse content that shortens the playback time of the content data. Playlists can also be exemplified by extracting important scenes from the content data and identifying these important scenes along the timeline of the content data. Alternatively, time-lapse content that shortens the duration of the content data can be used as an example.

[0041] Specifically, for example, in the case of sports content, the playlist uses multiple sound source data separated from the audio data to determine information about exciting scenes. Exciting scenes can be determined, for example, based on the amplitude of sound source data separated as cheers.

[0042] Specifically, the sound source separation result analysis unit 204 generates a playlist, for example, using the separated sound source data. Alternatively, for example, the playlist may be generated based on the types of content contained in at least one sound source data and service information.

[0043] The sound source separation result analysis unit 204 uses sound source data to generate a playlist, wherein the sound source data is output through sound source separation processing performed by the sound source separation DNN inference unit 202. Therefore, compared with the case of generating a playlist using sound data or dynamic image data, a playlist can be generated efficiently within an edge device.

[0044] Furthermore, the sound source separation result analysis unit 204 applies different rules according to the type of content, thereby generating a playlist corresponding to the type. Also, because the sound data is separated into sound source data after a predetermined time, a playlist can be generated during the publication of content data.

[0045] In addition, the sound source separation result analysis unit 204 is executed by the CPU, which uses the parameters stored in the DNN model storage unit 206 and the like to perform the prescribed calculation and control processing.

[0046] Playlist DB205 stores the playlist sent from the sound source separation result parsing unit 204. Multiple playlists can also be stored in playlist DB205 for each piece of content data.

[0047] The DNN model storage unit 206 stores, for example, at least one inference model. Additionally, the DNN model storage unit 206 stores, for example, multiple inference models based on service information. Specifically, for example, multiple inference models are stored according to each type of content contained in the service information.

[0048] The DNN model storage unit 206 stores, for example, inference models for sports, movies, and music. The DNN model storage unit 206 may be a non-volatile memory, such as flash memory, hard disk, or ROM (read-only memory).

[0049] Next, the playback control unit 300 will be described using a tracking playback control unit as an example. For example, it can be used when tracking playback of published content data. Here, tracking playback is exemplified by the following situation: while recording published content data, playback is resumed at any point within the range from the start of the recording of the published content data to the current position in the recording.

[0050] For example, in the case of tracking playback, the playback control unit 300 can play content in a shorter time than usual by using a playlist that shortens the content data to a specific scene, thus catching up with the published content data. Furthermore, even in the case of non-tracking playback, i.e., when playing recorded content after its publication, the playlist can also play the content in a shorter time than the recorded content would normally take.

[0051] The playback control unit 300 includes a video decoder 301, an audio decoder 302, a special playback (trick play) control unit 303, and an audio remixer 304.

[0052] The video decoder 301 acquires video data from the recorded content of the content DB400, for example. Specifically, it acquires a playlist from the special playback control unit 303 and outputs video data based on the playlist. Furthermore, for example, when the playlist is being played, it outputs video data for each period of the acquired recorded content, determined by the playlist, based on the playlist.

[0053] The audio decoder 302 obtains sound source data, for example, from the sound source separation result buffer 203. Additionally, it obtains a playlist from the special playback control unit 303 and outputs the separated sound source data based on the playlist.

[0054] The audio decoder 302 can also acquire and play audio data from the recorded content of the content DB400. Additionally, it plays audio source data that has undergone gain adjustment by the audio mixer 304.

[0055] The audio decoder 302, for example, can output only specific sound source data not only when outputting based on a playlist, but also when playing normally based on the content DB400, because sound source data is stored in the sound source separation result buffer 203. Furthermore, for example, during playlist playback, the audio decoder 302 outputs sound source data for each period determined by the playlist from the acquired sound source data.

[0056] The special playback control unit 303 reads the playlist from the playlist DB205. It controls the video decoder 301 and the audio decoder 302 to focus on important scenes based on the playlist.

[0057] Specifically, the special playback control unit 303 controls playback by displaying a playlist, for example, based on video data of the sound source data and content data output from the sound source separation DNN inference unit 202. Furthermore, the special playback control unit 303 is also referred to as a control unit.

[0058] In this embodiment, the audio mixer 304 obtains the separated sound source data from the sound source separation result buffer 203, thereby adjusting the gain of the sound data during playback.

[0059] Additionally, for example, when playing content DB400 and playlist DB205, audio mixer 304 adjusts the gain of the content data or the multiple sound source data contained in the playlist based on multiple sound source data output from the sound source separation DNN inference unit 202.

[0060] In addition, specifically, the audio mixer 304 obtains information from the UI setting unit 500, such as the type of content selected by the user's operation and the sound source data that needs to be emphasized, and controls the audio decoder 302 in a way that adjusts the gain of multiple sound source data based on the obtained information.

[0061] The content DB (Data Base) 400, for example, includes auxiliary storage devices such as HDD (Hard Disk Drive) and SSD (Solid State Drive) to store various information.

[0062] Alternatively, the content DB400 may be an external storage device separate from the information processing device 1, as an example. However, it can also be included within the information processing device 1. The content DB400 stores content data such as published video recordings.

[0063] The UI setting unit 500 is a unit that receives user operations. It may include input devices such as a keyboard and mouse. In addition, for example, when playing content DB400, the UI setting unit 500 sends information related to the type of content selected by the user and the sound source data to be emphasized to the special playback control unit 303.

[0064] Next, Figure 2 will be used to illustrate the sound source separation and playlist generation. Figure 2 is a diagram used to illustrate the sound source separation and playlist generation of the implementation method.

[0065] The sound source separation in this embodiment refers to, for example, the following situation: using an inference model, outputting at least one sound source data from the sound data contained in the content data.

[0066] As shown in Figure 2, for example, the sound source separation DNN inference unit 202 uses sound data obtained from content data and an inference model corresponding to the type of content to output sound source data.

[0067] When the content is sports, the sound source separation DNN inference unit 202 separates the sound sources according to sound source data such as "dialog", "cheer", "music" and "effect".

[0068] In addition, when the content is of the type of movie / drama, the sound source separation DNN inference unit 202 separates the sound sources according to sound source data such as "dialogue" and "Other".

[0069] Furthermore, when the content is music, the sound source separation DNN inference unit 202 separates the sound sources according to sound source data such as "drums," "bass," "others," and "vocals." Moreover, the above separation example is just one example; it could also output at least one sound source data.

[0070] In this way, the sound source separation DNN inference unit 202 can output only one sound source data by separating the sound source data. In addition, the gain can be adjusted according to each sound source data.

[0071] Specifically, for example, when the content is music, the sound source separation DNN inference unit 202 may output only "singing". Alternatively, it may output only "singing". Furthermore, when the content is sports, it may reduce the amplitude of "cheering" and increase the amplitude of "dialogue". That is, it may suppress cheering and emphasize the announcer's voice.

[0072] Next, the generation of playlists using sound source data will be explained. The sound source separation result analysis unit 204 generates playlists using sound source data separated according to content type. For example, a playlist is generated using an inference model of at least one sound source data separated according to content type and each content type.

[0073] When the content is sports, the sound source separation result analysis unit 204 may, for example, regard the range of "cheers" in the sound source data that is above a constant sound pressure level as a "passionate" scene and generate a playlist that extracts the "passionate" scene.

[0074] In addition, when the content is a movie or drama, the sound source separation result analysis unit 204 may, for example, consider the range of "dialogue" in the sound source data that is above a constant sound pressure level as a "scene with dialogue" and generate a playlist that extracts the "scene with dialogue".

[0075] In addition, when the content is music, the sound source separation result analysis unit 204 may, for example, regard the range of sound source data other than "singing" that is above a constant sound pressure level as a "performance" scene and generate a playlist that extracts the "performance" scene.

[0076] Furthermore, the different extraction conditions for each type of content described above are just one example. Extraction conditions can also be preset and changed each time. Moreover, inference models can be used to set appropriate extraction conditions corresponding to the type of content.

[0077] Next, using Figure 3, an example of content data and playlist playback will be illustrated. Figure 3 is a diagram schematically showing the generated playlist of an implementation method. Figure 3 shows an example of published content data and a playlist generated from the content data.

[0078] In the content data shown in Figure 3, for example, the playback time of one piece of recorded content data is shown.

[0079] Next, the playlist shown below the content data in Figure 3 serves as an example, using data from the timeline of the content data to determine specific periods (important scenes or performance scenes, etc.). Specifically, the playlist, as shown in Figure 3, determines specific periods (a), (b), (c), and (d) from the timeline of the content data.

[0080] The playlist shown below in Figure 3 illustrates the playback time of specific periods (a), (b), (c), and (d) of the playlist generated from that content.

[0081] As shown in Figure 3, by playing a playlist, playback can be completed in a shorter time compared to the playback time of the content data. Furthermore, the playlist, as shown in Figure 3, can also be data pre-edited only for specific periods (important scenes, performance scenes, etc.).

[0082] Next, Figure 4 is a diagram illustrating the gain adjustment of the sound source data in the implementation method. The upper part of Figure 4 shows an example of playing back the sound source data separated according to each sound, stored in the sound source separation result buffer 203. The lower part of Figure 4 shows an example of adjusting the gain of the separated sound source data and playing it back.

[0083] Here, for example, when playing the sound source data stored in the sound source separation result buffer 203, the audio mixer 304 of the playback control unit 300 can adjust the gain according to each separated sound source data. In addition, the separated sound source data can be mixed.

[0084] Here, for example, as shown in the lower part of Figure 4, the case where the sound source data is "drum sound", "bass", "other" and "singing" from top to bottom is used as an example for illustration.

[0085] The audio mixer 304 of the playback control unit 300 will play the original sound from time 0 to t1 as an example. Specifically, it represents the case where the sound source data of "drum sound", "bass", "other" and "singing" are all multiplied by a coefficient of 1 and then added together.

[0086] Next, we will illustrate the case where the audio mixer 304 of the playback control unit 300 plays karaoke music from time t1 to t2 as an example. Specifically, this means that, except for "singing", all sound source data such as "drum sound", "bass", and "other" are multiplied by a coefficient of 1 and then added together.

[0087] Furthermore, the audio mixer 304 of the playback control unit 300 will play a song with emphasis after time t2 as an example. Specifically, this means multiplying the sound source data of "singing" by a coefficient of 1, and multiplying the sound source data of "drum sound", "bass", and "other" by a coefficient of 0.1 and then adding them together.

[0088] Furthermore, even during playback, such as when the playback mode is changed by the UI setting unit 500, the audio mixer 304 can perform gain adjustment. For example, gain adjustment can be performed at free moments during playback, such as time t1, t2, etc.

[0089] Thus, because the separated sound source data is stored in the sound source separation result cache 203, the audio mixer 304 can perform gain adjustment during playlist playback. Furthermore, after the playlist is generated, gain adjustment can be performed even while the content is being played, for example, because the sound source data is stored in the sound source separation result cache 203.

[0090] Next, Figure 5 is a flowchart illustrating an example of the process for generating a playlist according to the embodiment. As shown in Figure 5, the audio decoder 102 of the dynamic image stream parsing unit 100 receives content data (step S100).

[0091] Additionally, the audio decoder 102 sends the sound data contained in the content data to the DNN input buffer management unit 201 of the sound source separation AI processing unit 200 (step S101).

[0092] Next, the DNN input buffer management unit 201 stores the acquired sound data in the storage area (step S102).

[0093] Next, when the sound data accumulated in the DNN input buffer management unit 201 reaches a predetermined amount or more, the sound source separation DNN inference unit 202 acquires the sound data that has reached a predetermined amount or more (step S103).

[0094] Here, the metadata parsing unit 103 of the dynamic image stream parsing unit 100 obtains the service information contained in the received content data. Then, it determines the type of content, etc., from the service information. The type of content, etc., is sent to the sound source separation DNN inference unit 202 (step S104).

[0095] The sound source separation DNN inference unit 202 selects and obtains an inference model from the DNN model storage unit 206 based on the type of content obtained (step S105).

[0096] Additionally, the sound source separation DNN inference unit 202 uses the acquired sound data and the selected inference model to separate the sound source data into multiple sound source data. Furthermore, at least one sound source data can also be output through sound source separation processing (step S106).

[0097] Additionally, the sound source separation DNN inference unit 202 outputs the separated sound source data to the sound source separation result buffer 203. Then, the sound source separation result buffer 203 stores the separated sound source data (step S107).

[0098] Next, the sound source separation result analysis unit 204 obtains the information that the sound source data is stored in the sound source separation result cache 203 (step S108).

[0099] Additionally, the sound source separation result analysis unit 204 obtains service information from the metadata analysis unit 103. Furthermore, it obtains the types of content included in the service information (step S109).

[0100] In addition, the sound source separation result analysis unit 204 obtains the separated multiple sound source data from the sound source separation result cache 203 (step S110).

[0101] In addition, the sound source separation result analysis unit 204 generates a playlist from the separated sound source data using a content-based inference model (step S111).

[0102] Furthermore, the sound source separation result analysis unit 204 may select a content-based inference model from the DNN model storage unit 206, for example. Alternatively, extraction conditions for the content-based playlist may be preset, and the playlist may be generated using the extraction conditions.

[0103] In addition, the sound source separation result analysis unit 204 sends the generated playlist to the playlist DB205 (step S112).

[0104] In addition, the sound source separation AI processing unit 200 repeatedly processes the sound data according to the sound source data and generates a playlist until the content is published.

[0105] Next, the playback process of the playlist will be explained using FIG6. FIG6 is a flowchart showing an example of the gain adjustment related processing of the embodiment. As shown in FIG6, the special playback control unit 303 of the playback control unit 300 obtains information on the playlist selection by the user regarding content data from the UI setting unit 500 (step S200).

[0106] Additionally, the special playback control unit 303 reads the playlist from the playlist DB205 (step S201).

[0107] In addition, the special playback control unit 303 sets the input source based on the playlist in the video decoder 301 (step S202).

[0108] In addition, the special playback control unit 303 sets the input source based on the playlist in the audio decoder 302 (step S203).

[0109] Next, the video decoder 301 obtains content data matching the playlist from the content DB400 (step S204).

[0110] Next, the audio decoder 302 retrieves the separated sound source data that matches the playlist from the sound source separation result cache 203 (step S205).

[0111] Furthermore, the audio mixer 304 adjusts the gain of the sound source data included in the playlist based on the information set by the user from the UI setting unit 500. Specifically, the gain adjustment of the audio decoder 302 is performed by obtaining information from the UI setting unit 500 through the special playback control unit 303 (step S206).

[0112] Then, the video decoder 301 outputs motion picture data based on the playlist. Additionally, the audio decoder 302 outputs sound source data based on the playlist. (Step S207).

[0113] In addition, the audio mixer 304 may also make appropriate gain adjustments when outputting playlist-based data in step S207.

[0114] The special playback control unit 303 obtains the playback position of the playlist (step S208).

[0115] In step S208, for example, taking tracked playback as an example, if the playback position of the playlist catches up with the published content data, the special playback control unit 303 ends the playback of the playlist. Alternatively, taking the playback of content data as an example, if the playback position of the content data is at the end, the playback of the playlist ends.

[0116] Alternatively, in step S208, for example, taking tracking playback as an example, if the playback position of the playlist does not catch up with the published content, the special playback control unit 303 may return to step S206, and use the stored content data and sound source data to output the playlist by the video decoder 301 and the audio decoder 302.

[0117] Additionally, in step S208, for example, taking tracked playback as an example, if the playback position of the playlist does not catch up with the published content data, the special playback control unit 303 returns to step S204 after reading the playlist again, obtains the content data, and outputs the playlist by the video decoder 301 and the audio decoder 302.

[0118] As described above, the information processing apparatus of this embodiment is an information processing apparatus that receives and processes content, including: an acquisition unit that stores sound data acquired from content data in a storage area; an inference unit that performs sound source separation processing using the sound data stored in the storage area and outputs sound source data; and a generation unit that generates time-lapse content that shortens the playback time of the content data based on the output sound source data.

[0119] Therefore, by generating time-lapse content based on sound source data, time-lapse content can be generated effectively. Furthermore, compared to generating time-lapse content using moving image data or sound data, the generation time can be shortened. Additionally, by generating time-lapse content, users can effectively play the content while tracking playback.

[0120] Furthermore, the inference unit in the information processing apparatus of this embodiment outputs sound source data based on service information regarding content data. Therefore, different sound source data can be generated based on the service information regarding content data. Additionally, it is possible to output only specific sound source data.

[0121] Furthermore, the inference unit in the information processing apparatus of this embodiment outputs sound source data using an inference model corresponding to service information about the content data. Thus, by using an inference model corresponding to the service information, sound source data for each service can be generated. That is, multiple sound source data can be appropriately generated from the sound data depending on the type of content.

[0122] Furthermore, the generation unit in the information processing apparatus of this embodiment generates time-lapse content based on sound source data and service information. Therefore, appropriate time-lapse content can be generated based on service information. That is, when the content is sports-related, time-lapse content that gathers together enthusiastic scenes with lots of cheers can be generated.

[0123] Furthermore, the information processing apparatus in this embodiment also includes a control unit that plays time-lapse content based on sound source data and content data. Thus, for example, the information processing apparatus can use the control unit to play time-lapse content using only specific sound source data from the sound source data.

[0124] Furthermore, the information processing apparatus in this embodiment also includes an audio mixer that, during playback of content data or time-lapse content, performs gain adjustment on the sound source data contained in the content data or time-lapse content based on the sound source data. Thus, by adjusting the gain of specific sound source data from the sound source data, it is possible to play the sound source data that the user wants to hear.

[0125] Furthermore, the information processing apparatus of this embodiment includes an audio decoder that, during playback of time-lapse content, outputs sound source data for a specific period based on the acquired sound source data from the time-lapse content. Therefore, since the time-lapse content is played using the sound source data output by the inference unit, gain adjustment can be performed during playback.

[0126] Furthermore, the information processing apparatus of this embodiment includes a video decoder that, during playback of time-lapse content, outputs dynamic image data within the acquired content data, arranged according to a specific period, based on the time-lapse content. Therefore, since content data stored in the content database can be played, it is unnecessary to store the large amount of dynamic image data in the sound source separation AI processing unit. Additionally, dynamic image data can be played using the video decoder.

[0127] The information processing apparatus described in this embodiment is exemplified by a case where it is built into an edge device that does not send the acquired content to an external device. The edge device is exemplified by a television device, which includes at least a dynamic image stream resolution unit 100, an audio source separation AI processing unit 200, and a playback control unit 300. Alternatively, the edge device could be a Blu-ray recorder, a DVD (Digital Versatile Disk) recorder, or the like. Furthermore, because the information processing apparatus can generate time-lapse content within the edge device, it does not need to send the content to an external device.

[0128] This embodiment describes sound source separation and playlist generation using received content data. However, it is not limited to this; for example, it may also include sound source data after sound source separation of the broadcast wave. Alternatively, the generated playlist may be broadcast as a broadcast wave.

[0129] The program for implementing the functions of the information processing device 1 described above can also be recorded in a computer-readable storage medium such as a CD-ROM, floppy disk (FD), CD-R, or DVD (Digital Versatile Disk) and provided as a file that can be installed on a computer or is executable. Alternatively, the program can be configured to be stored on a computer connected to a network such as the Internet and provided by downloading it from the network. Furthermore, the program can be configured to be provided or distributed via a network such as the Internet.

[0130] The above describes the embodiments of this application. However, these embodiments are presented as examples and are not intended to limit the scope of the application. These new embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the application. These embodiments and their variations are included in the scope and spirit of the application, and are included in the technical solutions described in the protection scope and their equivalents.

Claims

1. An information processing apparatus that receives and processes content data, wherein, The information processing device includes: The acquisition unit stores the audio data acquired from the content data in the storage area; The inference unit performs sound source separation processing using the sound data stored in the storage area and outputs sound source data; as well as The generation unit generates time-lapse content that shortens the playback time of the content data based on the output sound source data.

2. The information processing apparatus according to claim 1, wherein, The inference unit outputs the sound source data based on service information related to the content data.

3. The information processing apparatus according to claim 1 or 2, wherein, The inference unit uses an inference model corresponding to the service information related to the content data to output the sound source data.

4. The information processing apparatus according to claim 2, wherein, The generation unit generates the time-lapse content based on the sound source data and the service information.

5. The information processing apparatus according to claim 1, wherein, The information processing device further includes a control unit, which controls the playback of the time-lapse content based on the sound source data and the content data.

6. The information processing apparatus according to claim 1, wherein, The information processing device also includes an audio mixer, which, when the content data or the time-lapse content is played, performs gain adjustment on the sound source data contained in the content data or the time-lapse content based on the sound source data.

7. The information processing apparatus according to claim 1, wherein, The information processing device also includes an audio decoder, which, when the time-lapse content is played, outputs sound source data from the acquired sound source data according to a specific period based on the time-lapse content.

8. The information processing apparatus according to claim 1 or 7, wherein, The information processing device also includes a video decoder, which, when the time-lapse content is played, outputs dynamic image data of the acquired content data according to a specific period based on the time-lapse content.

9. A non-volatile storage medium storing a program for controlling an information processing device via a computer, the program causing the computer to function as: The acquisition unit stores the audio data acquired from the content data into the storage area; The inference unit performs sound source separation processing using the sound data stored in the storage area and outputs sound source data. as well as The generation unit generates time-lapse content that shortens the playback time of the content data based on the output sound source data.

Images