Information processing system, information processing method, and program

Abstract

To enable effective selection of videos. [Solution] An information processing system comprising: a video data storage unit that stores multiple video data; a question acquisition unit that acquires questions from users; and a playback video determination unit that provides a prompt to a large-scale language model including information about the video data, the question, and an instruction to select video data to provide in response to the question, thereby determining from the video data the video data to answer the question.

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] There is provided a system that answers questions from users (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-039881 A Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, there has been a demand to provide information through video rather than text, but it is difficult to provide appropriate answers to user questions through video.

[0005] The present invention has been made in view of the above background, and has an object to provide a technique that enables effective selection of videos. [Means for solving the problem]

[0006] The main invention of the present invention for solving the above problem is an information processing system comprising a video data storage unit that stores multiple video data, a question acquisition unit that acquires a question from a user, and a playback video determination unit that provides a prompt to a large-scale language model including information about the video data, the question, and an instruction to select the video data to be provided in response to the question, and determines the video data from the video data to answer the question.

[0007] Other problems and solutions disclosed in this application will be made clear in the description of the preferred embodiments of the invention and the drawings. Effect of the Invention

[0008] According to the present invention, moving images can be effectively selected. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of an overall configuration of an information processing system. [Diagram 2] 2 is a diagram illustrating an example of a hardware configuration of a management server 2. FIG. [Diagram 3] 2 illustrates an example of the software configuration of a management server 2. FIG. [Figure 4] FIG. 13 is a diagram illustrating the operation of the management server 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <System Overview> An information processing system according to an embodiment of the present invention will be described below. The information processing system of this embodiment is configured to provide answers to questions from a user using videos, and uses a large language model (LLM) to determine which videos from a plurality of prepared videos should be provided to the user and in what order.

[0011] 1 is a diagram showing an example of the overall configuration of an information processing system. The information processing system of this embodiment is configured to include a management server 2. The management server 2 is communicably connected to a user terminal 1 via a communication network. The communication network is, for example, the Internet, and is constructed by a public telephone line network, a mobile phone line network, a wireless communication path, Ethernet (registered trademark), etc.

[0012] The user terminal 1 is a computer operated by a user. The user terminal 1 may be, for example, a smartphone, a tablet computer, or a personal computer.

[0013] The management server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0014] <Management Server> FIG. 2 is a diagram showing an example of the hardware configuration of the management server 2. Note that the illustrated configuration is an example, and other configurations may be used. The management server 2 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 is, for example, a hard disk drive, a solid state drive, or a flash memory that stores various data and programs. The communication interface 204 is an interface for connecting to a communication network, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone line network, a wireless communication device for wireless communication, a USB (Universal Serial Bus) connector or an RS232C connector for serial communication, etc. The input device 205 is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, etc. that input data. The output device 206 is, for example, a display, a printer, a speaker, etc. that output data. Each functional unit of the management server 2 described below is realized by the CPU 201 reading a program stored in the storage device 203 into the memory 202 and executing it, and each storage unit of the management server 2 is realized as part of the storage area provided by the memory 202 and the storage device 203.

[0015] 3 is a diagram showing an example of the software configuration of the management server 2. The management server 2 includes a video data storage unit 231, a question acquisition unit 211, a playback video determination unit 212, and a playback processing unit 213.

[0016] <Storage section> The video data storage unit 231 stores video data. In this embodiment, the video data storage unit 231 stores information about the video data (hereinafter, video information) together with the video data. The video information can include any information (metadata) that indicates the contents and attributes of the video data, and can include, for example, information that identifies the video (for example, a video ID), the title of the video, a description of the video, video data, and the like. The metadata of the video information can also include the playback time of the video, the date and time of the video's creation, the creator of the video, the attributes of the viewers targeted by the video, the category of the video, and keywords related to the video. The metadata of the video information can also include a thumbnail image that indicates the contents of the video, subtitle data of the video, and transcribed text data generated from the audio data of the video. Note that the video information may not include the contents of the video data, and may store the video data as, for example, a file, and store information for acquiring the file (hereinafter, referred to as file acquisition information. For example, the information may be a path on a file system or a URL that identifies data in cloud storage), and acquire the video data based on the file acquisition information. A plurality of pieces of video data (and video information related to the video data) are stored in the video data storage unit 231. The title of the video information may be set in the file name. The description of the video is assumed to be text data, but may be video, audio, or the like.

[0017] In this embodiment, it is assumed that a large number of short video data (for example, about 15 seconds to 1 minute) are registered in the video data storage unit 231. For example, one video data can be registered for one topic or subtopic. In this embodiment, as described below, these short video data are linked together to dynamically generate answers.

[0018] <Functional section> The question acquisition unit 211 acquires a question from a user. The question from the user can be received from, for example, the user terminal 1. Also, an input of the question from the user together with the user's designation may be accepted from the user terminal 1 of the staff member who received the question from the user. The question acquisition unit 211 can acquire, for example, a message input by the user to a BOT or the like as a question.

[0019] The playback video determination unit 212 determines video data to be played back as a response to a question from a user. The playback video determination unit 212 provides a prompt including information on the video data, the question, and an instruction to select video data to be provided in response to the question to the large-scale language model, and determines video data to be used to respond to the question from among the video data.

[0020] The information on the video data may be at least one of the title and the description of the video included in the video information. The information on the video data may include the various metadata described above. The information on the video data included in the prompt may be, for example, information on all the video data stored in the video data storage unit 231.

[0021] The large-scale language model may be provided, for example, in the management server 2. In this case, the management server 2 stores a program for executing the large-scale language model in the storage device 203, and the CPU 201 executes the program to realize the large-scale language model. The large-scale language model may also be provided in a server other than the management server 2. In this case, the management server 2 can access the large-scale language model using an external API (Application Programming Interface). For example, the management server 2 can send an HTTP (Hypertext Transfer Protocol) request to a predetermined URL (Uniform Resource Locator) and obtain the processing result of the large-scale language model as a response. As the large-scale language model, GPT-3 (Generative Pre-trained Transformer 3), PaLM (Pathways Language Model), Chinchilla, OPT (Open Pre-trained Transformer), etc. may be used. Fine-tuned versions of these large-scale language models may also be used.

[0022] The playback moving image determination unit 212 can determine the playback order as well as the moving image data to be played back.

[0023] The playback video determination unit 212 can use various methods when determining the playback order of video data to be provided in response to a question. For example, the playback video determination unit 212 can determine the playback order by comparing the contents of the selected video data with the contents of the question and arranging the video data in the order in which matters related to the contents of the question are explained. The playback video determination unit 212 can also determine the playback order by comparing the contents of the selected video data with each other and arranging the video data of related contents so as to be consecutive. Furthermore, the playback video determination unit 212 can also determine the playback order based on the priority or importance previously set for the video data.

[0024] In this embodiment, the playback video determination unit 212 determines the playback order using a large-scale language model. For example, the playback video determination unit 212 can provide a prompt including information about video data, a question, and an instruction to select video data to be provided in response to the question and to determine the playback order of the video data, to the large-scale language model, and determine the video data to be provided in response to the question from among the video data and the playback order of the video data.

[0025] The playback order determined by the playback moving image determination unit 212 can be expressed in various formats. For example, the playback order can be expressed as the order of numbers assigned to the video data. In this case, the playback moving image determination unit 212 can assign numbers to the selected video data in the order in which they should be played. The playback order can also be expressed as a list in which the file names of the video data are arranged. In this case, the playback moving image determination unit 212 can generate a list in which the file names of the selected video data are arranged in the order in which they should be played. Furthermore, the playback order can also be expressed as a list indicating the playback start time and playback end time of the video data. In this case, the playback moving image determination unit 212 can determine the playback start time and playback end time for each selected video data, and generate a list in which they are arranged in the order in which they should be played.

[0026] The playback video determination unit 212 may select video data to be played back and generate a reason for selecting the video (a reason for viewing the video in response to a question). The playback video determination unit 212 may generate a reason by providing a prompt including an instruction to explain a reason for viewing the video to the large-scale language model. In this case, the playback video determination unit 212 may provide a prompt including information on the video data, a question, and an instruction to consider the video data to be provided in response to the question and the reason. After the video is selected, the reason may be generated by another prompt. In this case, the playback video determination unit 212 may provide a prompt including a question, information on the selected video data, and an instruction to explain the reason for selecting the video data to the large-scale language model. The playback video determination unit 212 may select video data and include an instruction to consider the reason for the selection and the playback order in the prompt.

[0027] The playback video determination unit 212 may also instruct the large-scale language model to analyze the relevance between keywords included in the title or description of the selected video data and keywords included in the question, and generate a sentence including a highly relevant keyword as a part of the reason. The playback video determination unit 212 may also instruct the large-scale language model to generate text data summarizing the contents of the selected video data, compare the text data with the question, extract a part suitable for explaining the contents of the question, and generate the extracted part as a part of the reason. The playback video determination unit 212 may also instruct the large-scale language model to generate a reason why the selected video data is suitable as an answer to the question by referring to the titles and descriptions of other video data related to the selected video data and analyzing the relevance of them with the question.

[0028] The playback processing unit 213 performs processing for playing back the video data in the playback order. The playback processing unit 213 can output (for example, transmit to the user terminal 1) a list of the video data determined by the playback video determination unit 212. The playback processing unit 213 may output the list of the determined video data in the determined playback order. The playback processing unit 213 may arrange the list of the determined video data in the determined playback order and may also attach to the video data a reason why the video data should be viewed.

[0029] The playback processing unit 213 can control the video data determined by the playback video determination unit 212 to be played back continuously in the determined playback order. The playback processing unit 213 can, for example, transmit the determined video data to the user terminal 1 continuously in the determined playback order so that the data is played back continuously in the user terminal 1. For example, the playback processing unit 213 can stream the video data. As a streaming delivery method, for example, various streaming protocols such as HTTP Live Streaming (HLS), Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), MPEG-DASH, etc. can be used. The playback processing unit 213 can also download and deliver the video data. In this case, the playback processing unit 213 can control the video data to be downloaded to the user terminal 1 and then played back in the local storage of the user terminal 1. The playback processing unit 213 may transmit the determined video data to the user terminal 1 with a playback order attached, so that the video data is played back in the user terminal 1 in the playback order.

[0030] Furthermore, the playback processing unit 213 may transmit the reason why the video should be viewed together with the video data, and cause the user terminal 1 to display the reason before, during, and / or after playback of the video data.

[0031] <Operation> FIG. 4 is a diagram illustrating the operation of the management server 2. As shown in FIG.

[0032] The management server 2 receives a question from a user (S301), provides the question and the titles / descriptions of the videos to be played to the LLM to generate the videos to be played and their order (S302), and controls the playback of the selected videos in the playback order (S303).

[0033] As described above, the information processing system of this embodiment can play back video data that is appropriate as an answer to a question from a user from among the video data prepared in advance. The information processing system of this embodiment can also play back video data that is appropriate as an answer to a question in an appropriate order. The information processing system of this embodiment can also display the reason why the suggested video data should be viewed.

[0034] Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention, and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and equivalents thereof are also included in the present invention.

[0035] For example, the processes performed by each of the functional units of the management server 2 described above may be executed by any of the functional units. Also, a different functional unit that executes part of the processes performed by each of the functional units described above may be added. Also, the functional units of the management server 2 may be provided in a distributed manner on multiple computers.

[0036] Furthermore, the information stored in each storage unit of the management server 2 may be stored by any storage unit. That is, the information stored in the above-mentioned multiple storage units may be stored by one storage unit, or part of the information stored in one of the above-mentioned storage units may be stored by another storage unit.

[0037] <Variation 1> Information about video data can include a wider variety of information, such as the date and time the video was created, the creator, the playback time, and the attributes of the target audience.

[0038] For example, the video information may include timestamp data indicating the date and time when the video was created. In response to a question, the video may be selected based on the date and time when the video was created, for example, by preferentially selecting the most recent video or, conversely, by selecting an older video.

[0039] For example, the video information may include creator information indicating the creator of the video. Creator information may include, for example, the creator's name, affiliation, title, specialty, etc. In response to a question, videos may be selected based on the creator of the video, such as by preferentially selecting videos by a specific creator or by selecting videos by multiple creators in a balanced manner.

[0040] For example, the video information may include playback time information indicating the playback time of the video. In response to a question, videos may be selected based on their playback time, such as by preferentially selecting videos with long or short playback times, or by selecting videos whose total playback time falls within a predetermined time.

[0041] For example, the video information may include viewer attribute information indicating the attributes of the target viewer. The viewer attribute information may include, for example, the expected age group, gender, occupation, interests, etc. A video may be selected according to the attributes of the user who asked the question, and video information related to the selected video data may be provided to a prompt to select video data to be proposed to the user from the selected video data.

[0042] <Variation 2> The playback video determination unit 212 may select video data without using a large-scale language model. For example, the playback video determination unit 212 generates data obtained by vectorizing the title and description of the video data in advance, and calculates the cosine distance between the data and the question acquired by the question acquisition unit 211, thereby selecting video data related to the question.

[0043] For example, the playback video determination unit 212 can perform morphological analysis on the title and description of each video data, and vectorize each video data based on the frequency of occurrence of each morpheme. The playback video determination unit 212 can also perform morphological analysis and vectorize the question acquired by the question acquisition unit 211 in the same manner. The playback video determination unit 212 can then calculate the cosine distance between the vector of the question and the vector of each video data, and select the video data in ascending order of cosine distance. This allows the playback video determination unit 212 to appropriately select video data related to the content of the question.

[0044] Furthermore, the playback video determination unit 212 may use information other than the title and description when vectorizing the video data. For example, the playback video determination unit 212 may vectorize the video data using metadata (keywords, category, etc.) added to the video data, or user evaluations of the video data (number of views, number of likes, etc.). This allows the playback video determination unit 212 to select video data that is more relevant to the question.

[0045] Furthermore, the playback video determination unit 212 may select video data related to a question using a method other than the cosine distance. For example, the playback video determination unit 212 may calculate the Euclidean distance between the vector of the question and the vector of each video data, and select video data in the order of smallest Euclidean distance. The playback video determination unit 212 may calculate the inner product between the vector of the question and the vector of each video data, and select video data in the order of largest inner product. By appropriately selecting one of these methods, the playback video determination unit 212 can select video data that is more related to the question.

[0046] <Variation 3> The playback video determination unit 212 can determine the playback order of video data provided in response to a question based on the relevance of the contents of the video data. For example, the playback video determination unit 212 can extract keywords contained in the titles and descriptions of the video data, and determine a playback order such that video data containing many common keywords are played back consecutively. The playback video determination unit 212 can also refer to metadata (e.g., tags representing objects and scenes shown in the video) that represent the contents of the video data, and determine a playback order such that video data having high similarity in metadata are played back consecutively. This allows video data having high relevance in content to be played back consecutively.

[0047] Furthermore, the playback video determination unit 212 can determine the playback order of video data provided in response to a question based on the degree of relevance between the question and the contents of the video data. For example, the playback video determination unit 212 can compare keywords included in the question text with keywords included in the title or description of the video data, and determine the playback order to play video data with higher keyword match in descending order. The playback video determination unit 212 can also calculate the similarity between the content of the question text and metadata representing the content of the video data, and determine the playback order to play video data with higher similarity in descending order. This allows video data with higher relevance to the content of the question to be preferentially played.

[0048] Furthermore, the playback video determination unit 212 can determine the playback order by combining the relevance of the contents of the video data with the relevance between the question and the contents of the video data. For example, the playback video determination unit 212 can first select video data that is highly relevant to the question, and then determine the playback order based on the relevance between the selected video data.

[0049] <Variation 4> When generating a reason for selecting a video, the playback video determination unit 212 may generate the reason by comparing the content of the selected video with the content of the question and extracting a highly related part. For example, the playback video determination unit 212 can generate a reason for selecting a video by comparing text data such as the description and subtitle data of the selected video with the text data of the question, extracting words that appear in common between the two and words that are synonymous or similar, extracting sentences, paragraphs, etc. containing the extracted words from the text data of the video and the question, and summarizing them using a text summarization algorithm.

[0050] Furthermore, the playback video determination unit 212 may add the playback start time and end time of a part of the selected video that is highly relevant to the question, to the reason for selecting the video. Based on the playback start time and end time information, the playback processing unit 213 may play only the relevant part of the video, or may play the entire video before or after playing the relevant part.

[0051] <Variation 5>

[0052] The playback processing unit 213 can play back the video data in various ways. For example, the playback processing unit 213 can stream the video data. In this case, the playback processing unit 213 can transmit the video data to the user terminal 1 in real time and play the video data in the user terminal 1 as it receives it.

[0053] The playback processing unit 213 can also download the video data and then play it. In this case, the playback processing unit 213 transmits the video data to the user terminal 1, and the user terminal 1 receives and stores the video data. The user terminal 1 can play the stored video data.

[0054] The playback processing unit 213 can also play back a combination of streaming and downloading of video data. For example, the playback processing unit 213 can download a portion of the video data, start playback, and stream the remaining portion.

[0055] <Disclosures> The present disclosure also includes the following configurations. [Item 1] a video data storage unit for storing a plurality of video data; a question acquisition unit for acquiring a question from a user; a playback video determination unit that provides a prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question to a large-scale language model, and determines the video data to be provided in response to the question from among the video data; An information processing system comprising: [Item 2] The information processing system according to item 1, the playback video determination unit includes in the prompt an instruction for creating the video data and a playback order to be provided in response to the question, selects the video data to answer the question from among the video data, and determines the playback order of the selected video data; An information processing system comprising: [Item 3] Item 2. An information processing system according to the present invention, the playback video determination unit further provides the prompt, including an instruction to explain a reason for selecting the video data, to the large-scale language model to generate the reason; An information processing system comprising: [Item 4] Item 2. An information processing system according to the present invention, a playback processing unit that plays back the video data in the playback order; An information processing system comprising: [Item 5] storing a plurality of video data; obtaining a question from a user; providing a prompt to a large-scale language model, the prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question, to determine the video data to be provided from among the video data to answer the question; An information processing method characterized by being executed by a computer. [Item 6] storing a plurality of video data; obtaining a question from a user; providing a prompt to a large-scale language model, the prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question, to determine the video data to be provided from among the video data to answer the question; A program for causing a computer to execute the following. [Explanation of symbols]

[0056] 1 User terminal 2 Management Server

Claims

1. a video data storage unit for storing a plurality of video data; a question acquisition unit for acquiring a question from a user; a playback video determination unit that provides a prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question to a large-scale language model, and determines the video data to be provided in response to the question from among the video data; An information processing system comprising:

2. 2. The information processing system according to claim 1, the playback video determination unit includes in the prompt an instruction for creating the video data and a playback order to be provided in response to the question, selects the video data to answer the question from among the video data, and determines the playback order of the selected video data; An information processing system comprising:

3. 3. The information processing system according to claim 2, the playback video determination unit further provides the prompt, including an instruction to explain a reason for selecting the video data, to the large-scale language model to generate the reason; An information processing system comprising:

4. 3. The information processing system according to claim 2, a playback processing unit that plays back the video data in the playback order; An information processing system comprising:

5. storing a plurality of video data; obtaining a question from a user; providing a prompt to a large-scale language model, the prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question, to determine the video data to be provided from among the video data to answer the question; An information processing method characterized by being executed by a computer.

6. storing a plurality of video data; obtaining a question from a user; providing a prompt to a large-scale language model, the prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question, to determine the video data to be provided from among the video data to answer the question; A program for causing a computer to execute the following.

Images