Digest video generation system, digest video generation device, and digest video generation method

Abstract

To provide a digest video generation system capable of easily generating a digest video in line with a user's intention.SOLUTION: According to an embodiment, a digest video generation device includes a communication unit, a broadcast data acquisition unit, a meta information extraction unit, a user interface unit, and a digest video generation unit. The communication unit accesses a database that stores broadcast data to be aired. The broadcast data acquisition unit acquires the broadcast data from the database. The meta information extraction unit extracts, from the acquired broadcast data, meta information of the broadcast data. The user interface unit provides a user interface that presents pick-up information that can be specified by a user based on the meta information. The digest video generation unit generates a digest video based on the specified pick-up information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to a digest video generation system, a digest video generation device, and a digest video generation method. [Background technology]

[0002] There has been active development of technologies related to the secondary use of video content. For example, broadcasting stations are considering services that use social media to distribute video summaries of programs. As a result, attention is now being focused on video summarization technology, which generates videos that can be viewed efficiently in a short amount of time from large amounts of long video data.

[0003] This type of technology is highly compatible with AI (Artificial Intelligence) and has been developed primarily for sports, concerts, and other video content. Further development is needed to establish automatic summarization technology for broadcast video of various genres, particularly to increase the appeal to younger generations and boost program exposure. Digest video generation, in particular, is a technology that can uniformly identify important scenes while preserving the overall content of the video, and is expected to be applied primarily to story-based content such as dramas and movies. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-181790 [Patent Document 2] Patent Publication No. 2021-170724 [Patent Document 3] Special Publication No. 2020-516107 [Patent Document 4] Japanese Patent Application Publication No. 2024-65703 [Non-patent literature]

[0005] [Non-Patent Document 1] “Latest Trends in Automatic Video Summarization Technology”, [online], NHK STRL R&D 2020 Summer Issue No. 182, [Retrieved May 24, 2024], Internet,<URL:https: / / www.nhk.or.jp / strl / publica / rd / 182 / 2.html> Summary of the Invention [Problem to be solved by the invention]

[0006] Services that generate and quickly distribute digest videos are becoming increasingly important in video production sites, including broadcast stations. However, some production sites do not always have sufficient budgets or personnel to produce digest videos. Furthermore, when producing digest videos, the scenes to be cut out are determined from the video producer's perspective, making it difficult for editors to generate the digest video after the fact. Even when digest videos are automatically generated using AI, it is unclear whether the creator's intentions and story are properly reflected. There is a demand for technology that can generate digest videos that match the diverse needs of various users with simple operations.

[0007] Therefore, an object of the present invention is to provide a digest video generation system, a digest video generation device, and a digest video generation method that can easily generate a digest video that meets the user's intentions. [Means for solving the problem]

[0008] According to an embodiment, a digest video generation device includes a communication unit, a broadcast data acquisition unit, a meta information extraction unit, a user interface unit, and a digest video generation unit. The communication unit accesses a database that stores broadcast data to be aired. The broadcast data acquisition unit acquires the broadcast data from the database. The meta information extraction unit extracts meta information of the acquired broadcast data from the broadcast data. The user interface unit provides a user interface that presents pick-up information that can be specified by a user based on the meta information. The digest video generation unit generates a digest video based on the specified pick-up information. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a system diagram showing an example of a digest video generation system according to an embodiment. [Figure 2] FIG. 2 is a functional block diagram illustrating an example of a digest video generation system according to the embodiment. [Figure 3] FIG. 3 is a flowchart showing an example of a processing procedure of the digest video generation device 200. [Figure 4] FIG. 4 is a diagram for explaining scoring based on audience rating information. [Figure 5] FIG. 5 is a diagram showing an example of a GUI window displayed on the user terminal 50. As shown in FIG. [Figure 6] FIG. 6 is a diagram showing another example of a GUI window displayed on the user terminal 50. In FIG. DETAILED DESCRIPTION OF THE INVENTION

[0010] FIG. 1 is a system diagram showing an example of a digest video generation system according to an embodiment. In FIG. 1, a broadcasting station 100 equipped with a master system MS transmits broadcast data to a transmitting station 600 via an STL (Studio to Transmitter Link) / TTL (Transmitter to Transmitter Link) network according to an on-air schedule. The transmitting station 600 generates an OFDM (Orthogonal Frequency Division Multiplex) modulated wave for terrestrial digital broadcasting from the broadcast data and broadcasts it within its service area. In this way, terrestrial digital broadcasting is realized. There is also a system for BS (Broadcasting Satellite) broadcasting, but this is omitted for simplicity of explanation.

[0011] The broadcast station 100 stores broadcast data such as raw footage, scripts, and program operation information in an in-station database (broadcast data 800). Examples of broadcast data include products, shops, areas, public information, and disaster information broadcast on television programs. The broadcast station 100 also provides broadcast data based on a contract with a service provider, for example, and stores the data in a database 300 in the cloud 400 (broadcast data 300a). In the embodiment, a system that encourages secondary use of broadcast data and can provide convenient services to a variety of users will be described.

[0012] The cloud 400 is a service platform managed and operated by, for example, a service provider. The cloud 400 includes a digest video generation device 200 and a database 300, and receives access from users 900. The users 900 can use various data related to the broadcast data 800, for example, in the form of SaaS (Software as a Service). In particular, cloud services that are closely related to the broadcasting business are often referred to as BaaS (Broadcast as a Service).

[0013] The broadcast station 100 can access the cloud 400 to receive the desired service. In addition, users 900, such as content distributors, local governments, regional public organizations, corporations, retail stores, or individuals, can also access the cloud 400 to receive the service they desire. The service provider and the user agree in advance on the specifications (required specifications) of the secondary use data they desire, the details of charges, and so on.

[0014] The broadcast station 100 can also receive services as a user. In short, the term "user" is not limited to the above-mentioned entities, but may include all entities that "want to use broadcast data," including secondary users who use the generated and provided digest video.

[0015] Fig. 2 is a functional block diagram showing an example of a digest video generation system according to an embodiment. In Fig. 2, the digest video generation device 200 is a computer and includes a processor 21, a read-only memory (ROM) 23, and a random access memory (RAM) 24. The digest video generation device 200 further includes a storage unit 22 such as a hard disk drive (HDD), a communication unit 25, and an optical media drive 26.

[0016] The ROM 23 stores basic programs such as a Basic Input Output System (BIOS) and a Unified Extensible Firmware Interface (UEFI), various setting data, etc. The RAM 24 temporarily stores programs and data loaded from the storage unit 22.

[0017] The optical media drive 26 reads digital data recorded on a recording medium such as a CD-ROM 27. Various programs executed by the digest video generation device 200 are recorded on, for example, the CD-ROM 27 and distributed. The programs stored on the CD-ROM 27 are read by the optical media drive 26 and installed in the storage unit 22. The communication unit 25 processes communication between the digest video generation device 200 and, for example, the broadcast station 100 and the cloud 400. For example, the communication unit 25 controls access by the digest video generation device 200 to the database 300.

[0018] Various programs executed by the digest video generation device 200 can be downloaded from a server via the communication unit 25, for example, and installed in the storage unit 22. Also, the latest programs can be downloaded from the cloud 400 via the communication unit 25, and installed programs can be updated.

[0019] The database 300 stores and accumulates broadcast data 30a to be aired, meta information 30b extracted from the broadcast data 30a, and secondary use data 30c generated based on these data / information. In this embodiment, digest video is generated as the secondary use data 30c. The broadcast data 30a and the secondary use data 30c may be stored in, for example, the Material Exchange Format (MXF) file format, which is advantageous for data distribution. The meta information 30b may be in, for example, the general-purpose CSV format or text data.

[0020] The processor 21 is, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and executes an OS (Operating System) and various application programs.

[0021] The processor 21 also includes, as processing functions according to the embodiment, a broadcast data acquisition unit 21a, a meta information extraction unit 21b, a user interface unit 21c, a digest video generation unit 21d, a scoring processing unit 21e, and a providing unit 21f. These functional blocks can be understood as processes that are generated when a program 22a stored in the storage unit 22 is loaded into the RAM 24 and executed. In other words, the program 22a causes the computer to function as the broadcast data acquisition unit 21a, the meta information extraction unit 21b, the user interface unit 21c, the digest video generation unit 21d, the scoring processing unit 21e, and the providing unit 21f, thereby realizing the digest video generation device 200.

[0022] The broadcast data acquisition unit 21a acquires broadcast data 30a to be aired from the database 300. The broadcast data 30a to be aired may be broadcast data before airing, broadcast data currently airing, or broadcast data after airing. The meta information extraction unit 21b extracts the acquired broadcast data 30a and meta information from the broadcast data 30a. For example, in the case of commercial message (CM) content, meta information may be the broadcast date and time of the CM content. Other meta information may include attributes such as product information for the promotion of the CM content, the sponsor, the cast, the age, gender, and generation of the promotion target, the start and end times of the CM broadcast, whether or not subtitles are present, the industry, the product name, the content, and the image resolution (e.g., low-res, HD, 4K image). In particular, when handling data in the MXF file format, both commercial content and program content are accompanied by detailed meta information. Furthermore, broadcast operation data for operating the broadcasting system is also an example of meta information.

[0023] The user interface unit 21c creates a GUI (Graphical User Interface) window, including a dialog box and a pull-down window, based on the extracted meta information. That is, the user interface unit 21c provides a user interface that presents pick-up information that can be specified by the user based on the meta information. That is, the user interface unit 21c displays the created GUI window, for example, in a web page format, on the display of the user terminal 50, and prompts the user to specify pick-up information.

[0024] The featured information is presented to the user in the form of, for example, meta information converted into text, such as hot words. In other words, the meta information can include hot words. By selecting and specifying the featured information on the GUI window, the user can instruct the system on "what kind of digest video they want to create."

[0025] The digest video generator 21d generates a digest video based on the specified pick information. For example, suppose that trending words (meta information) such as "delicious" and "great value for money" are extracted from the same video material. The digest video generator 21d generates a digest video including tasty ingredients for a user who selected "delicious," and creates a digest video including reasonable and useful products for a user who selected "great value for money." In this way, by constructing a user interface that allows users to arbitrarily select pick information from the meta information, users can easily create the digest video they want.

[0026] The scoring processor 21e assigns a score between 0 and 100 to the extracted meta information, for example, to convert it into a score. For example, it is possible to obtain viewership rating information for the attention words in the program after the broadcast data 800 is aired, and weight the scenes in which the attention words appear based on the viewership rating information, thereby converting the attention words into scores. The user interface unit 21c presents the attention words and the weighting results in a GUI window of the user terminal 50.

[0027] The providing unit 21f provides the generated digest video to the secondary use destination, for example, in a download format. Furthermore, in Fig. 2, a billing server 10 may be provided that charges the secondary use destination, for example, a flat-rate service (subscription format) according to the rank of the contract content. Next, the operation of the above configuration will be described.

[0028] 3 is a flowchart showing an example of the processing procedure of the digest video generation device 200. The digest video generation device 200 acquires broadcast data 30a to be aired from the database 300 (step S1), and extracts meta information such as program information and product information (step S2). In particular, known AI can be applied to extract the meta information.

[0029] Next, the digest video generation device 200 scores the extracted information (step S3). FIG. 4 is a diagram for explaining scoring based on audience rating information. For example, the same attention word may appear in multiple places in a program. In this case, by obtaining the (instantaneous) audience rating calculated after broadcast via the provider's API (Application Programming Interface) or the like and using it as a criterion for judgment, a more effective digest range can be specified. In other words, even if the attention word is the same, specifying a word associated with a scene with a high audience rating can increase the effectiveness of that attention word.

[0030] 3, the digest video generating device 200 further displays a GUI window on the user terminal 50 to accept the selection / designation of information to be picked up (step S4). This will be described in detail with reference to FIGS.

[0031] Fig. 5 is a diagram showing an example of a GUI window displayed on the user terminal 50. The window shown in Fig. 5 will be referred to as a "content list screen" in the following description. The content list screen is a user interface for accepting the designation of content (for example, provided program information, broadcast material, etc.) that will be the source of generating a digest video.

[0032] The content list screen is equipped with various input fields, dialogue windows, pull-down menus, etc., and allows users to search content not only by free word, but also by content name, program name, broadcast date, etc. Furthermore, searches can be performed using extracted meta information (people, highlights, etc.). This user interface allows inexperienced editors to search for programs while fully reflecting the user's intentions, even when generating digest videos. Clicking the [Edit] button on the content list screen jumps to the [Edit Screen] shown in Figure 6.

[0033] 6 is a diagram showing an example of an "editing screen" for generating a digest video. The digest video generating device 200 receives a selection / designation of information that the user wants to pick up via the "editing screen."

[0034] In Figure 6, "moist," "delicious," and "white" are the words with the highest scores from the trending words analyzed by AI. By selecting one of these three words, it is possible to extract scenes that display a lot of information related to the selected word. In other words, scenes from a program can be extracted using information scored by meta information. The extracted images can be used as promotional materials, for example, by creating push ads.

[0035] Additionally, on the Editing Screen, meta information can be used to narrow down scenes, and the digest range can be determined accordingly, allowing for the automatic generation of digest videos. For example, if you enter a product name or the word "delicious," the digest range is generated, and a digest video highlighting the "delicious" element is automatically generated. Furthermore, depending on the digest selected, not only the digest video but also the document within the digest range (such as the script or broadcast audio) can be summarized.

[0036] At the bottom of the Editing Screen, for example, a list of scenes divided by meta information is displayed. For example, if the word "delicious" is broadcast in the audio of the program, that scene can be extracted. It is also possible to search by meta information, making it easy to narrow down scenes when editing.

[0037] When the "Execute" button on the editing screen is clicked, the digest video generation device 200 generates a digest video (step S5 in FIG. 3). Then, the digest video generation device 200 provides the generated digest video to the secondary use destination (step S6) and notifies the billing server 10 of this. The billing server 10 charges the secondary use destination at the set timing.

[0038] In this way, users can specify the content they want to highlight from the meta information extracted by AI, and a digest video that matches the specified content will be generated. For example, when providing digest videos as promotional materials for retailers, they can generate digest videos by entering the target product and keywords. In other words, rather than creating a fixed digest, it is possible to create digest videos that are customized according to the user's needs. This allows for effective advertising of the target product, which is expected to increase sales.

[0039] In addition to inputting keywords related to the target product, digest videos can be extracted more effectively by using viewer rating information. The keywords you input may have been used multiple times within the program. Therefore, by using viewer rating information, it is possible to weight the points that are attracting the most attention from viewers (scenes where the keywords appear) and omit unimportant scenes, thereby generating a digest video that meets the user's intentions.

[0040] As described above, in the embodiment, the content of broadcast data is analyzed using AI, and trending words and minute-by-minute (instantaneous) viewer rating information, etc., are acquired and scored. Here, not only the video content itself but also accompanying information (meta information) is used, so a digest video that is more in line with the user's intentions and story can be generated. The scored analysis information allows the user to freely specify the information they want to highlight, making it easy for even inexperienced users to narrow down scenes and easily create a digest video that meets their intentions.

[0041] With existing technology, editing digest videos depends on the producer's perspective, making it difficult for new editors to create digest videos. In contrast, in this embodiment, the contents of the broadcast data are scored, making it easy to identify where important scenes were located within the program. This allows users with little editing knowledge to create and edit digest videos as intended.

[0042] Furthermore, according to the embodiments, it is possible to provide and utilize digest videos as a service business. In other words, it is possible to realize the digest video generation system according to the embodiments as a service on a platform that automatically meta-data, stores, analyzes, and provides data on products, performers, keywords, etc. from raw video used for broadcasting.

[0043] Many existing digest video generation devices are related to sports, and there is no known method for creating digest videos from extracted raw video related to products, performers, keywords, etc., and utilizing the digest videos for business purposes. According to the embodiment, it becomes possible to automatically generate digests of information programs, product introduction programs, etc., and utilize the digest videos for service businesses.

[0044] As a result, according to the embodiments, it is possible to provide a digest video generation system, a digest video generation device, and a digest video generation method that can easily generate digest videos that meet the user's intentions, and further, it is possible to effectively utilize broadcast data.

[0045] The present invention is not limited to the above-described embodiments. For example, the embodiments have described a digest video generation device that edits raw video to generate a digest video. Also, a database has been disclosed that stores and accumulates raw video from which a digest video is generated. In addition to these, the system may further include storage means for managing raw video for each program.

[0046] Furthermore, the digest video generator 21d may automatically generate digest videos by providing a pre-trained AI (Artificial Intelligence) model with featured information designated by a user. For example, by using AI generation technology, which has been attracting attention in recent years, it is quite possible to generate digest videos from the designated featured information.

[0047] Furthermore, the scoring processor 21e may use trend information on a social networking service (SNS) instead of viewer rating information. That is, trend information on the SNS for a predetermined period including the time when the broadcast data 800 is on air (for example, up to 24 hours after airing) may be acquired, and scenes in which attention words appear in the program may be weighted based on this trend information.

[0048] The system may also be provided with means for storing and accumulating production information that stores production rules to be added to the digest video, viewpoint information that indicates viewpoints for generating the digest video, and text information that stores transcriptions (text) of the raw video, as well as means for storing and accumulating minute-by-minute (instantaneous) viewership ratings (viewership rating information).

[0049] The system may also be provided with a means for selecting a priority order according to the attention words and the scores assigned to them.Furthermore, the system may also be provided with a means for storing and accumulating analysis information that stores the analysis results of the video material.

[0050] Furthermore, for example, some or all of the functions of the digest video generation device 200 can be implemented in the cloud 400. While the embodiment has been described as being in the form of SaaS, other known cloud forms include PaaS (Platform as a Service), which provides a platform for running applications as a service, and IaaS (Infrastructure as a Service), which provides resources such as server devices, central processing units, and storage as a service (public cloud). The digest video generation system of the embodiment can be realized in either form.

[0051] The digest video generation device 200 and database 300 do not need to be physical entities, and can be realized as resources on a virtualized platform. The instance type and virtual processor type can be fixedly specified, and the processing power of the virtual server can be freely scaled up or down depending on the needs and number of users.

[0052] Furthermore, a program for realizing the digest video generation system according to the embodiment can be recorded on a computer-readable recording medium. The digest video generation system can be realized by loading and executing the program recorded on the recording medium into a computer system. The "computer system" may also include hardware such as an OS and peripheral devices.

[0053] Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This novel embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its modifications are included within the scope and spirit of the invention, and are also included in the invention described in the claims and their equivalents. [Explanation of symbols]

[0054] 10...billing server, 21...processor, 21a...broadcast data acquisition unit, 21b...meta information extraction unit, 21c...user interface unit, 21d...digest video generation unit, 21e...scoring processing unit, 21f...providing unit, 22...storage unit, 22a...program, 23...ROM, 24...RAM, 25...communication unit, 26...optical media drive, 30a...broadcast data, 30b...meta information, 30c...secondary use data, 50...user terminal, 100...broadcast station, 200...digest video generation device, 300...database, 400...cloud, 600...transmitting station, 800...broadcast data, 900...user.

Claims

1. A means for acquiring broadcast data to be aired; means for extracting meta information of the broadcast data from the broadcast data; a means for providing a user interface that presents pick-up information that can be specified by a user based on the meta information; a means for generating a digest video based on the specified pickup information; A digest video generation system comprising:

2. the means for acquiring broadcast data acquires broadcast data before it is aired; The digest video generation system according to claim 1 .

3. 2. The digest video generation system according to claim 1, wherein the means for generating the digest video provides the specified pickup information to a pre-trained AI (Artificial Intelligence) model to automatically generate the digest video.

4. 2. The digest video generation system according to claim 1, wherein the means for providing a user interface presents the user with a menu screen including a field for specifying the meta information.

5. The method further includes a means for scoring the extracted meta information, The digest video generation system according to claim 1 , wherein the means for providing a user interface presents the meta information and a score of the meta information on the user interface.

6. the meta information includes a word of interest included in the broadcast data, The means for scoring the meta information acquires audience rating information after the broadcast data is aired, and weights scenes in which the attention word appears based on the audience rating information; 6. The digest video generation system according to claim 5, wherein the means for providing a user interface presents the attention words and the weighting results thereof on the user interface.

7. the meta information includes a word of interest included in the broadcast data, The means for scoring the meta information acquires trend information on a social networking service (SNS) for a predetermined period including the time when the broadcast data is on air, and weights scenes in which the attention word appears based on the trend information; 6. The digest video generation system according to claim 5, wherein the means for providing a user interface presents the attention words and the weighting results thereof on the user interface.

8. 8. The digest video generation system according to claim 1, further comprising: a means for providing the digest video to a secondary user.

9. 9. The digest video generation system according to claim 8, further comprising a billing server that bills the secondary users for using the digest video.

10. a communication unit that accesses a database that stores broadcast data to be aired; a broadcast data acquisition unit that acquires the broadcast data from the database; a meta information extraction unit that extracts meta information of the broadcast data from the acquired broadcast data; a user interface unit that provides a user interface for presenting pick-up information that can be specified by a user based on the meta information; a digest video generating unit that generates a digest video based on the specified pickup information.

11. The process of acquiring broadcast data to be aired, extracting meta information of the broadcast data from the broadcast data; providing a user interface that presents pick-up information that can be specified by a user based on the meta information; and generating a digest video based on the specified pickup information.

Images