Video content providing apparatus, video content retrieval method, and program

Abstract

To easily retrieve a partial scene including a content desired by a user across multiple video contents from among the multiple video contents in an environment where the user cannot be involved in.SOLUTION: A video content providing apparatus 110 includes: a retrieval information generation unit 112 which has a video content dividing function to divide a video content including a voice data stream and a video data stream into a plurality of scene groups, and a retrieval keyword generation function to generate, for each of the scene groups, retrieval keywords for retrieving the scene groups; a retrieval record registration unit 114 which generates and registers a plurality of retrieval records in which information on the scene groups is associated with the retrieval keywords; and an information providing unit 115 which specifies, during retrieval of the video content, a retrieval keyword that satisfies a retrieval condition from the multiple retrieval records, and provides information on the scene group identified from the specified retrieval keyword, to a client terminal 120.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a video content providing device, a video content search method, and a program, and more particularly to automatic keyword generation for video content and video content search utilizing the same. [Background technology]

[0002] In recent years, with the spread of social networking services (SNS), a lot of video content is scattered across the Internet, etc. Video content refers to video data that has been packaged in a form that allows users to play it on the Internet or other environments, and there are various types of forms and data formats, etc., and corresponding utilization technologies (video generation and video playback technologies) have been developed and are in general use.

[0003] These environments have made it possible for ordinary users, who do not have any particular knowledge or skills regarding videos, to freely play and view video content available in open environments such as the Internet. As a result, users no longer just watch videos or search for and find out what they want to know, but also search for, play, and use video content to gain knowledge and know-how.

[0004] However, since there is a large amount of video content available in environments such as the Internet, it takes a considerable amount of time to find video content containing desired content and play back a desired scene.

[0005] In order to satisfy such a demand, for example, Patent Document 1 proposes a technique for easily searching for a part of a scene in a video content. Specifically, Patent Document 1 discloses a video service providing method that utilizes audio data included in the video content to separate the video content into a plurality of unit sections, recognizes the audio included in the unit sections to generate a script character string, and generates keywords corresponding to the unit sections using the generated script character string. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 6824332 Summary of the Invention [Problem to be solved by the invention]

[0007] According to the video service providing method described in Patent Document 1 mentioned above, when playing a piece of video content, it is possible to quickly play back a scene containing the desired content by referring to keywords associated with each unit section. However, for example, when using video content not for watching videos but for gaining knowledge or know-how, there is a demand for users to be able to pick out and play only partial scenes that contain content that interests them from multiple video contents.

[0008] The present invention has been made in consideration of the above circumstances, and aims to provide a video content providing device, a video content search method, and a program that enable a user to easily search for partial scenes containing content desired by a user across multiple video contents that are in an environment where the user cannot be involved. [Means for solving the problem]

[0009] One aspect of the present invention is a video content providing device comprising: a video content division means for dividing video content including an audio data stream and a video data stream into a plurality of scene groups; a search keyword generation means for generating, for each scene group, a search keyword for searching the scene group; a search record registration means for generating and registering a plurality of search records that associate information about the scene group with the search keywords; and an information provision means for identifying search keywords that match search conditions from the plurality of search records when searching the video content, and providing a client terminal with information about the scene group identified from the identified search keywords.

[0010] One aspect of the present invention is a video content search method in which a computer executes a video content division process that divides video content including an audio data stream and a video data stream into a plurality of scene groups; a search keyword generation process that generates, for each scene group, a search keyword for searching the scene group; a search record registration process that generates and registers a plurality of search records that associate information of the scene group with the search keywords; and an information provision process that, when searching the video content, identifies search keywords that match search conditions from the plurality of search records, and provides information of the scene group identified from the identified search keywords to a client terminal.

[0011] One aspect of the present invention is a program for causing a computer to function as the video content providing device. Effect of the Invention

[0012] According to the present invention, it is possible to easily search for partial scenes containing content desired by a user across multiple video contents from among a large number of video contents in an environment where the user cannot be involved. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram showing a schematic configuration of a video content providing device according to an embodiment of the present invention. [Diagram 2] 1 is a diagram illustrating an example of a hardware configuration of a video content providing device according to an embodiment of the present invention. [Diagram 3] FIG. 11 is a diagram showing an example of a video content search screen according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing an example of a structure of a search management table according to an embodiment of the present invention. [Diagram 5] 11 is a flowchart showing an example of a processing procedure of a content collection process executed by a content information collection unit according to an embodiment of the present invention. [Figure 6] 11 is a flowchart showing an example of a processing procedure of a search record generation process according to an embodiment of the present invention. [Figure 7] 11 is a flowchart showing an example of a processing procedure of a video content division process according to an embodiment of the present invention. [Figure 8] 1 is a diagram for explaining a process of dividing an audio data stream according to an embodiment of the present invention; [Figure 9] FIG. 1 is a diagram for explaining natural language processing. [Figure 10] FIG. 1 is a diagram for explaining natural language processing. [Figure 11] FIG. 1 is a diagram for explaining natural language processing. [Figure 12] 11 is a flowchart illustrating an example of a processing procedure of a search keyword generating process according to an embodiment of the present invention. [Figure 13] FIG. 1 is an explanatory diagram for explaining ranking of text data according to an embodiment of the present invention. [Figure 14] 1 is a flowchart showing an example of a processing procedure of a video content search method according to an embodiment of the present invention. [Figure 15] 11 is a diagram showing an example of JSON data transmitted to a client terminal by a video content providing device according to an embodiment of the present invention. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a video content providing device, a video content search method, and a program according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of a video content providing device 110 according to an embodiment of the present invention.

[0015] As shown in FIG. 1, a video content providing device 110 is connected to a video content holding system 100 and a client terminal 120 via a network, and is configured to be able to send and receive information between them. The video content holding system 100 holds a plurality of video contents. In this embodiment, the video content holding system 100 is assumed to be an open system connected to the Internet and accessible to anyone, but is not limited to this. For example, the video content holding system 100 may be a closed system whose access is limited to within a company. Furthermore, the video content holding system 100 may be a simple storage system, or a system that provides a function for managing video content.

[0016] The client terminal 120 is an information processing device used by a user to view video content. Examples of the client terminal include a notebook PC, a desktop PC, a tablet terminal, a smartphone, etc. The client terminal 120 is assumed to have an environment setting that allows access to the video content holding system 100 and the video content providing device 110, such as the Internet or an in-house system.

[0017] The video content providing device 110 is a so-called server that performs a search based on search conditions received from the client terminal 120 and provides the video content desired by the user, more specifically, a scene group that is a unit section that constitutes the video content, to the client terminal 120. The video content providing device 110 may be a so-called on-premise server on which a system is built, or may be virtualized and built on a cloud system.

[0018] The video content providing device 110 includes, for example, a content information collecting unit 111, a search information generating unit 112, a storage unit 113, a search record registering unit 114, an information providing unit 115, and a communication unit . The content information collection unit 111 collects information on video content in order to make the large number of unspecified video contents present in the video content holding system 100 searchable.

[0019] The search information generating unit 112 analyzes the video content collected by the content information collecting unit 111 and generates a search record which is search information used to search the video content. The search information generating unit 112 includes, for example, a video content dividing unit (not shown) which divides video content including an audio data stream and a video data stream into a plurality of scene groups, and a search keyword generating unit (not shown) which generates, for each scene group, a search keyword for searching the scene group.

[0020] The search record registration unit 114 generates a plurality of search records that associate scene group information with search keywords, and registers the search records in the storage unit 113 . 4, a search management table in which a plurality of search records are registered is stored in the storage unit 113. The search management table and the search records will be described in detail later.

[0021] When the information providing unit 115 receives search conditions from the client terminal 120, it identifies search keywords that match the search conditions from the search management table stored in the memory unit 113, and provides information on scene groups associated with the identified search keywords to the client terminal 120. The communication unit 116 communicates with the client terminal 120 and the like.

[0022] FIG. 2 is a diagram showing an example of a hardware configuration of the video content providing device 110. As shown in FIG. As shown in Fig. 2, the video content providing device 110 according to this embodiment is a so-called computer, and includes, for example, a CPU 201, a main memory 202 that functions as a work area when each program is executed, an auxiliary storage device 203 for storing the programs executed by the CPU 201, and a communication interface 204 for connecting to a network. These components are connected, for example, via a bus. Examples of the auxiliary storage device 203 include a magnetic disk, a magneto-optical disk, and a semiconductor memory. Furthermore, the video content providing device 110 may include an input device such as a keyboard and a display device for displaying settings, etc., as necessary.

[0023] A series of processes for realizing the functions of the above-mentioned units of the video content providing device 110 are stored in the auxiliary storage device 203 or the like in the form of a program, and the CPU 201 reads this program into the main memory 202 or the like and executes information processing and arithmetic processing to realize various functions. The program may be installed in the auxiliary storage device 203 in advance, may be provided in a state stored in a computer-readable storage medium, or may be distributed via wired or wireless communication means. Examples of the computer-readable storage medium include a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, and a semiconductor memory.

[0024] Next, a video content search screen, which is a user interface used to search for video content according to this embodiment, will be described with reference to the drawings. Fig. 3 is a diagram showing an example of a video content search screen 300. 3, this video content search screen 300 may be constructed as a Web application on the video content providing device 110, or may be constructed as a native application on the client terminal 120. In either case, the processing of the information providing unit 115 slightly changes, which will be described later.

[0025] The category selection area 301 functions as a category input section for the user to specify the field of the video content to be searched for. In this embodiment, a category is always present, but if the user wishes to search for video content in all fields regardless of classification, the user should simply input "All" in this area.

[0026] Categories are used to improve search accuracy and efficiency. For this reason, it is preferable that category information is registered in advance in the video content providing device 110. For example, category information may be managed by the search record registration unit 114, and may be assigned to each record by the search record registration unit 114 when a search list is registered in the storage unit 113. By using categories, it is possible to improve the efficiency of searching video content.

[0027] The keyword input area 302 is for the user to input any word. Fig. 3 shows an example in which the keyword "image processing" is input by the user. In response to this input, the video content providing device 110 executes a search process, which will be described later, and a list of search keywords is displayed in the search candidate display area 303.

[0028] The user selects the desired keyword from the displayed list of search keywords. Figure 3 shows an example in which "Overview of AI used for image processing" has been selected. After the user selects any keyword, he or she presses search button 307. This causes the video content providing device to search for scene groups based on the keyword selected by the user, and representative images 305 associated with the scene groups are displayed in a list in representative image display area 304. When the user selects any of the representative images 305 in this representative image display area 304, a scene group of the selected video content is displayed in video playback area 306, and when the user performs a playback operation, the video of the scene group is played. The user can then select multiple representative images displayed in representative image display area 304 one after another, thereby efficiently viewing video of partial scenes containing desired content in video playback area 306.

[0029] Next, the structure of the search management table stored in storage unit 113 will be described with reference to Fig. 4. This search management table has a plurality of search records (corresponding to each row shown in Fig. 4). The search records are generated by search information generation unit 112 and registered by search record registration unit 114.

[0030] The search management table includes items such as "category", "search keyword", "content URL", "start timestamp", "end timestamp", and "thumbnail image". As mentioned above, the "category" is an item used to improve the accuracy and efficiency of searches.

[0031] The item "content URL" registers a URL that is information indicating the storage location of the target video content. Since the video content should exist within the video content holding system 100, this address URL must be a valid address therein and must naturally be accessible from the client terminal 120. Here, there may be cases where the video content holding system 100 is divided into multiple areas, but even in such cases, as long as it is within a range accessible from the client terminal 120, there is no need to set common rules for the format of the address URL, etc.

[0032] The "start timestamp" and "end timestamp" register the timestamp at which a scene group associated with a search keyword starts and the timestamp at which that scene group ends in each video content. In this embodiment, the time elapsed from the beginning of the video content is measured in milliseconds (ms), but this is not limiting and other methods may be used to manage the information as long as it can identify a scene group. For example, instead of a timestamp, a frame number from the beginning of the video content may be used. This makes it possible to specify a position within the video content in more detail.

[0033] Information indicating the storage location of a still image representative of each scene group is registered in the "thumbnail image." There is no particular limitation on the storage location of the "thumbnail image."

[0034] As shown in FIG. 4, each search record constituting the search management table stores information that identifies a scene group that is part of the video content, making it possible to play back only the scenes with the desired content.

[0035] Furthermore, when multiple search keywords are generated for the same scene group, a search record is generated for each search keyword. For example, three search records 401, 402, and 403 having a "search keyword" of "overview of AI used for image processing" are registered in the search management table illustrated in Fig. 4. Of these, search records 401 and 402 have the same "content URL" but different "start timestamp" and "end timestamp", indicating that different scenes of the same video content are registered. On the other hand, search record 403 has a different "content URL" from search records 401 and 402, indicating that it is a different content from search records 401 and 402.

[0036] In this way, instead of generating a keyword for each video content, a search keyword is generated for each scene portion (scene group) included in the video content, and a search record is registered for each search keyword. This makes it possible to search for each scene portion (scene group) regardless of whether the video content is the same or not. In addition, since multiple video contents are managed as search records of the same format, management is simplified and operational costs can be reduced.

[0037] Next, a description will be given of various processes executed by the video content providing device 110 illustrated in Fig. 1. The following describes an example in which video content is searched for on an open system such as the Internet.

[0038] The content information collection unit 111, for example, monitors an unspecified number of video contents existing in the video content holding system 100, and when a video content is added, detects it and additionally registers search information in the search management table. This process enables the user to always search for the latest information without having to worry about changes in the video content holding system 100.

[0039] In the case of an open system, it is difficult to control the entire video content holding system 100, so when video content is added from somewhere unspecified, it is difficult to detect it almost in real time. For this reason, it is recommended that the content information collection unit 111 adopts a method such as crawling to periodically search the Internet and collect information. On the other hand, in the case of a system that is closed to a certain extent, such as an in-house system, it is relatively easy to detect the addition of video content to video content holding system 100. In this case, since video content holding system 100 itself can often be operated under complete control, it is advisable to provide a mechanism for transmitting an event when additional content occurs between video content holding system 100 and content information collecting unit 111. This makes it possible to easily detect the addition of video content, and reduce the system load.

[0040] FIG. 5 is a flowchart showing an example of a processing procedure of the content collection process executed by the content information collection unit 111 according to the present embodiment. First, in the automatic registration setting process (S601), the conditions required for performing a monitoring action (for example, the crawling described above) are set. Specifically, the target range for the crawling process, the information to be collected, the time interval for crawling, etc. are set. This process is performed by setting various conditions input by the user, for example.

[0041] Next, a timer is started (S602). Next, it is determined whether the processing timing based on the crawling time interval set in the automatic registration setting process (S601) has been reached (S603). As a result, if it is determined that the processing timing has not been reached, that is, the predetermined time has not elapsed since the timer was started (S603: NO), the process waits until the predetermined time arrives. On the other hand, if it is determined that the processing timing has arrived (S603: YES), the crawling process is executed to collect information on video content (S604). For example, video content information (e.g., URL of video content) is collected within the target range set in the automatic registration setting process (S601). In this collection process, the collected video content information (e.g., URL) is stored in a predetermined storage device held by the content information collection unit 111.

[0042] Next, new video content is extracted from the collected video content information (S605). Specifically, the URL of the current video content collected by the crawling process is compared with an analyzed history list in which information (e.g., URL) of video content for which search records have already been created is registered, and video content that does not exist in the analyzed history list is extracted as new video content.

[0043] Next, information on the new video content (e.g., URL) is stored as an analysis target list in a predetermined storage device provided in the content information collection unit 111 (S606). Next, it is determined whether or not to end the monitoring process of the video content holding system 100 (S607). For example, it is determined whether or not the end condition set in the automatic registration setting process (S601) is satisfied. As a result, if the end condition is not satisfied (S607: NO), the process returns to step S603 and waits until the next processing start timing arrives. On the other hand, if the end condition is satisfied (S607: YES), this process is terminated.

[0044] Next, the search record generation process executed by the search information generation unit 112 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the processing procedure of the search record generation process. The series of processes described below are executed for each new video content extracted in Fig. 5, for example.

[0045] First, the analysis target list stored in the storage device in step S606 of FIG. 5 is referenced, the URL of the new video content is accessed, and the video content is acquired (S701). Next, a video content division process is executed to divide the acquired video content into a plurality of scene groups (S702: video content division means). Next, a search keyword generation process is executed for each scene group to generate a search keyword for searching each scene group (S703: search keyword generation process).

[0046] Next, a search record that associates the scene group with a search keyword for searching the scene group is generated, and the generated search record is added to the search management table of the storage unit 113, thereby registering the search record (S704: search record generation means). In addition to the search keyword, the search record may include, for example, a storage address where a representative image (thumbnail image) representing the scene group is stored, a storage address (content URL) of the video content of the scene group, and a start timestamp and end timestamp of the scene group. In addition, category information may be included.

[0047] Next, the information of the video content for which the search record has been registered is deleted from the analysis target list, and the information of the video content is added to the analyzed history list, thereby updating the list (S705), and the process ends.

[0048] The above series of processes are then repeated until all video content information registered in the analysis target list is exhausted, resulting in the creation of search records for newly added video content and adding them to the management list.

[0049] Next, the video content division process executed in step S702 in Fig. 6 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the processing procedure of the video content division process according to this embodiment. First, the acquired video content is separated into a video data stream (hereinafter referred to as the "video data stream") and an audio data stream (hereinafter referred to as the "audio stream") (S801).

[0050] Next, the video stream is divided at the timing when the visual characteristics of the video stream change, thereby generating a plurality of scene groups (S802). Specifically, the system calculates information on changes between frames of the frame images that make up the video stream, calculates the positions where the screen of the video data changes significantly, in other words, the positions where the scenes change significantly, and then divides the video stream into multiple scene groups by adding timestamps to the timing of the scene changes.

[0051] Here, various methods for detecting scene changes have been proposed, and these known techniques may be appropriately adopted. For example, an image processing library such as OpenCV, which is provided as an open source, may be used to calculate inter-frame differences, and the calculated inter-frame differences may be compared with a threshold value to determine whether a scene has changed.

[0052] Here, the threshold value for determining frame transitions may affect search performance, so the threshold value may be adjusted depending on the content (e.g., category) of the video content to be targeted. For example, in the case of video content such as sports videos, in which scene changes are relatively large and occur frequently, a relatively large value may be set as the threshold value for determining scene changes. Also, in the case of video content such as explanatory video content, in which the screen is displayed large like a presentation material and the scene changes are not so large consistently, a relatively small value may be set as the threshold value.

[0053] Next, for each scene group created in S802, a representative image that represents the respective scene group is determined, and the determined representative image is stored in the storage unit 113 (S803). Here, each scene group is composed of a group of frames in which the scenes do not change significantly. For this reason, for example, the first frame image of each scene group is set as the representative image.

[0054] Next, the voice stream is converted into text data using voice recognition processing (S804). Various methods have been proposed for voice recognition processing, and these well-known techniques may be appropriately adopted. For example, AI (Artificial Intelligence) may be utilized. Specifically, Whisper provided by OpenAI, Inc. may be utilized.

[0055] Next, the audio stream is divided based on the time timing at which the video stream was divided (S805), and the process ends. Specifically, the audio data stream is divided based on the scene group created in S802 and the text data of the audio stream converted in S804. At this time, if the audio text data is simply divided based on the scene group, there is a risk that it will be divided at the timing when the audio is interrupted. This is because the video stream and the audio data stream are generally independent. Therefore, in this embodiment, the audio data stream is divided at the boundary of a sentence.

[0056] For example, consider the case where, as shown in Figure 8, the first scene group 1 of the video stream is 0 ms-3000 ms and the next scene group 2 is 3000 ms-10000 ms, while the text data of the audio stream has the first sentence from 0 ms-4000 ms and the next sentence from 4400 ms-13000 ms. In this case, the first sentence, 0 ms-4000 ms, is associated with the first scene group 1 (0 ms-3000 ms), and the next sentence, 4400 ms-13000 ms, is associated with the next scene group 2 (3000 ms-10000 ms).

[0057] In this way, the audio stream is divided based on the time when the scene groups of the video stream are divided and the sentences in the text data of the audio stream. More specifically, when dividing the audio stream, the sentences of the text data are analyzed, and the audio stream corresponding to each scene group is divided at the sentence breaks of the text data. This makes it possible to avoid unnatural interruptions in the audio when searching for video content.

[0058] Next, the search keyword generating process executed in step S703 of FIG. 6 will be described.

[0059] Search keywords for each scene group are generated based on the text data of the audio data associated with each scene group. For example, natural language processing (NLP) is used to generate search keywords. Specifically, by using a process called generative AI in NLP, it is possible to summarize text sentences converted from audio data streams and generate important noun phrases. Here, when generating search keywords using generative AI, the instructions given to the generative AI are important.

[0060] For example, Figure 9 shows an example of the input and results when a generative AI is asked to summarize a sentence. The upper part of Figure 9 is the original sentence, and the lower part is the result of instructing the AI ​​to summarize the original sentence in 50 characters. Figure 10 shows the same sentence as Figure 9, but with the addition of instructions to summarize with an emphasis on "data sharing among multiple medical institutions." Comparing these two results, we can see that even when summarizing the same sentence, the results are completely different depending on the instructions. In other words, by giving accurate instructions, the direction of the results can be characterized. As such, we can see that giving accurate instructions is important in order to generate accurate keywords.

[0061] Furthermore, when making a summary, not only the content of the instructions but also the length of the given text is important. For example, Figure 11 shows the results when the generative AI was instructed to divide the same text as in Figures 9 and 10 into three parts and create a summary for each part. In this case, the source text from which the summary is generated is shorter, so it can be seen that the content of each part is more accurately summarized than before it was divided.

[0062] Considering the characteristics of such a generative AI, a search keyword generation process for generating search keywords corresponding to each scene group will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of a processing procedure of the search keyword generation process.

[0063] First, character recognition processing (e.g., OCR: Optical Character Recognition) is performed on the representative image of each scene group determined in step S803 in Fig. 7 to obtain text data from the image data (S901). Attribute information (visual features) such as the position of the text in the entire image and the size or area per character of the text data (hereinafter referred to as "character size") is added to this text data.

[0064] Next, the acquired plurality of text data are ranked based on the attribute information of the text data (S902). Specifically, the ranking of each text data is performed using at least one of the character size of the text data and the position of the text data in the entire image.

[0065] FIG. 13 is an explanatory diagram for explaining the ranking of text data. For example, assume that a representative image 305 as shown in Fig. 13 is subjected to character recognition processing to obtain a plurality of text data 1201, 1202, 1203, etc. Each text data has attribute information such as text position information and character size added thereto. In Fig. 13, the text data 1201 is located at the top center of the entire screen, and the character size is larger than the other text data 1202 and 1203. For this reason, the text data 1201 can be considered to be the most significant in the representative image. Furthermore, the text data 1202 and 1203 have smaller character sizes than the text data 1201, and therefore can be considered to be less important than the text data 1201. Furthermore, the first character positions of the text data 1202 and 1203 are left-justified in the entire image, and the character sizes are the same, so these can be considered to be of the same importance. For this reason, the text data 1202 and 1203 can be considered to have a subtitle-like meaning. When ranking text data, information such as a table correlating the position of the text data in the entire image and the character size with the rank, and conditional expressions is stored, and the rank for each text data can be determined by referring to this information.

[0066] Next, search keywords are generated using the text data of the audio stream associated with each scene group and each text data and rank in the representative image (S903), and the process ends. For example, the text data of the audio stream associated with each scene group, each text data in the representative image and its rank are given to the generation system AI, and the AI ​​is instructed to generate a keyword within a predetermined number of characters based on these data. In this way, a keyword is generated for each scene group. Although the keyword may be generated by breaking it down into words as in morphological analysis, it is preferable to generate the keyword as a noun phrase so that it is a keyword that describes an individual scene. In this way, it is possible to generate a keyword that better reflects the content of the scene group, such as "Examples of more effective economies of scale" rather than a list of nouns such as "scale" and "economy". It should be noted that a plurality of search keywords may be generated for one scene group.

[0067] In this way, by periodically performing the above-mentioned series of processes in video content providing device 110, information on video content newly added to video content holding system 100 is collected by content information collecting unit 111 at a predetermined timing, and search records for each scene group of the collected new video content are added to the search management table stored in memory unit 113. This makes it possible to update the search management table stored in memory unit 113 at a predetermined timing, and to use the latest information in the search management table to search for video content, specifically, scene groups that are part of the video content.

[0068] Next, a video content search method using a search management table executed by the video content providing device 110 will be described with reference to the drawings. Fig. 14 is a flow chart showing an example of a processing procedure of the video content search method according to this embodiment. In the following description, a case where a screen is displayed on a web page using languages ​​(scripts) such as HTML, JavaScript, and CSS will be described as an example. The following search process is mainly executed by the information providing unit 115.

[0069] First, a user operates the client terminal 120 (see FIG. 1), and on the video content search screen illustrated in FIG. 3, the user selects a category in the category selection area 301, and then inputs a keyword in the keyword input area 302 and presses the search button 307. Then, the category and keyword information is transmitted from the client terminal to the video content management server.

[0070] When the information providing unit 115 of the video content management server receives information on search conditions consisting of, for example, a category and a search keyword (S1001), it searches the search management table using the acquired search conditions. Specifically, it compares the acquired category and search keyword with the categories and search keywords registered in each search management table to extract search records that match the search conditions (S1002). At this time, if the user inputs "all" in the category selection area 301, the category is ignored. As for the comparison method, any known technology may be appropriately adopted, such as full match, partial match, matching process using machine learning, etc.

[0071] Next, the search keywords written in each extracted search record are obtained (S1003), and the obtained search keywords are displayed as a list in the search candidate display area 303 (S1004). As a result, as shown in Fig. 3, the search keywords registered in each search record are displayed as a list of selection candidates in the search candidate display area 303.

[0072] Next, when the user selects one of the search keywords displayed in the search candidate display area 303 and presses the search button 307, information on the selected search keyword is transmitted to the video content providing device 110.

[0073] When the video content providing device 110 receives the search keyword information (S1005), it extracts search records having the search keyword (S1006), obtains representative images (still images) from the storage destination information (URL) of the "thumbnail image" registered in each extracted search record, and displays a list of the obtained representative images in the representative image display area 304 (S1007). As a result, a plurality of representative images 305 are displayed in the representative image display area 304, as shown in Fig. 3.

[0074] Next, when one of the representative images displayed in the representative image display area 304 is selected by the user, information on the selected representative image 305 is transmitted to the video content providing device 110 . The video content providing device 110 references the "content URL," "start timestamp," and "end timestamp" in the search record of the selected representative image 305, acquires the video content from the content URL, and identifies a scene group based on the start timestamp and end timestamp. Then, the information on this scene group is displayed in the video playback area 306 (S1008).

[0075] Next, when the user performs an operation to play the video, the video content providing device 110 plays the video of the scene group (S1009). The user can then successively select multiple representative images displayed in the representative image display area 304, thereby efficiently viewing a video of a partial scene containing the desired content in the video playback area 306.

[0076] The video content search method according to this embodiment may be performed on the Web as described above, or may be displayed on a screen at a client terminal as described below. In this case, the information providing unit 115 generates data as shown in FIG. 15 based on the information of each search record acquired in step S1006 of FIG. 14, for example, and transmits this to the client terminal 120. FIG. 15 is an example of JSON (JavaScript Object Notation) data. Here, script 501 is JSON data used for instructions from the client terminal 120 to the video content providing device 110, and script 502 is JSON data transmitted from the video content providing device 110 to the client terminal 120. The client terminal 120 updates the video content search screen based on the received data.

[0077] As described above, the video content providing device 110, video content search method, and program according to this embodiment analyze video content including an audio data stream and a video data stream to divide the video content into a plurality of scene groups, and generate a search keyword for searching the scene group for each of the divided scene groups. Then, a plurality of search records are generated that associate information about the scene group with the search keywords, and these are registered. Then, when searching for video content, a search keyword that matches the search conditions is identified from the plurality of search records, and information about the scene group identified from the identified search keyword is provided to a client terminal.

[0078] In this way, search keywords are generated for each scene group, and search records of a common format are generated among multiple video contents, making it possible to centrally manage scene group information whether the scene groups are the same video content or different video contents. This has the effect of allowing a user to easily search for a partial scene that includes a desired content across multiple video contents from among a large number of video contents in an environment where the user cannot participate. Also, the creator of the video can search for and play a specific scene of the video content desired by the user without having to perform a search operation in advance when creating the video content.

[0079] Furthermore, according to this embodiment, the video content information collection by the content information collection unit 111, the video content division process, the search keyword generation process, and the search record registration process by the search information generation unit 112 and the search record registration unit 114 are configured to operate at scheduled timings regardless of the timing of the generation of the video content (i.e., the timing when new video content is registered in the video content retention system) or the timing of the video content search and provision process by the information provision unit 115 of the video content provision device 110. As a result, whereas previously a video content search service itself could only be constructed if the creator of the video content at the time of creation, or if the video content user had to generate search keywords when using the content, any third party can now construct a video search service at any time, greatly improving the convenience of video search services and the freedom to build the service.

[0080] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various modifications or improvements can be made to the above embodiments without departing from the gist of the invention, and the forms in which such modifications or improvements are made are also included in the technical scope of the present invention. In addition, the above embodiments may be combined as appropriate. Furthermore, the processing flow described in the above embodiment is merely an example, and unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the present invention. [Explanation of symbols]

[0081] 110: Video content providing device 111: Content Information Collection Department 112: Search information generation unit 113: Storage section 114: Search record registration section 115: Information provision department 116: Communications Department 120: Client terminal

Claims

1. a video content division means for dividing a video content including an audio data stream and a video data stream into a plurality of scene groups; a search keyword generating means for generating, for each of the scene groups, a search keyword for searching the scene group; a search record registration means for generating and registering a plurality of search records in which the scene group information and the search keyword are associated with each other; an information providing means for, when searching for the video content, identifying a search keyword that matches a search condition from a plurality of the search records, and providing information on the scene group identified from the identified search keyword to a client terminal; A video content providing device comprising:

2. The search keyword generating means Detecting video features of the video data streams constituting the scene groups; 2. The video content providing device according to claim 1, wherein the search keyword is generated using video features of the video data stream and text data of an audio data stream that constitutes the scene group.

3. The video content division means includes: Splitting the video content into an audio data stream and a video data stream; generating a plurality of scene groups by dividing the video data stream at timings when a video characteristic of the video data stream changes; The video content providing device of claim 1, wherein the audio data stream is divided based on the time timing at which the video data stream is divided and the sentences in the text data of the audio data stream, and the divided audio data streams are associated with each of the scene groups.

4. The video content providing device of claim 1, wherein the video content division means, the search keyword generation means, and the search record registration means are configured to operate at scheduled timings independent of the timing of generation of the video content or the timing of searching for the video content by the information providing means.

5. A video content division process for dividing video content including an audio data stream and a video data stream into a plurality of scene groups; a search keyword generation process for generating, for each of the scene groups, a search keyword for searching the scene group; a search record registration process for generating and registering a plurality of search records in which the scene group information and the search keyword are associated with each other; an information providing process for identifying a search keyword that matches a search condition from a plurality of the search records when searching for the video content, and providing information on the scene group identified from the identified search keyword to a client terminal; The computer performs the video content search method.

6. A program for causing a computer to function as the video content providing device according to any one of claims 1 to 4.

Images