Content selection device, and program
The content selection device uses a natural language model to generate summary sentences and explanatory texts, addressing the challenge of selecting and managing event-related content by enhancing content selection and management efficiency.
Patent Information
- Application Number
- JP2023076334
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-02
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing natural language models lack an effective system for selecting and summarizing content related to specific events during a specified period, limiting their utilization in applications such as content selection and management.
A content selection device and program utilizing a natural language model generated by machine learning to generate summary sentences and explanatory texts, enabling the selection of content based on event-related sentences and images, with image recognition capabilities for identifying specific objects.
Enables efficient selection and management of content related to specific events by generating summary sentences and explanatory texts, allowing for targeted content acquisition based on event relevance and object identification.
Smart Images

Figure 2024160863000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a content selection device and a program capable of selecting content filmed during a specified period from sentences related to events during the specified period by using a natural language model. [Background technology]
[0002] As an information processing system that processes natural language using a learning model generated by machine learning or other methods, for example, a data generation method using a language model is known, as disclosed in Patent Document 1. The data generation method of Patent Document 1 uses original data to construct a prompt that serves as an input sentence for the language model, and inputs the constructed prompt into the language model to generate natural language. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2023-018624 A Summary of the Invention [Problem to be solved by the invention]
[0004] Recently, technologies related to natural language models, including the conventional technologies mentioned above, have made great strides. For example, natural language models generated by learning large amounts of text data are called large-scale language models, and are expected to be used for sentence generation, summarization, image captioning, etc.
[0005] On the other hand, natural language models, including large-scale language models, are a technology that has been rapidly developing in recent years, and the fields in which they can be effectively used are being explored. In this situation, there is a demand for information processing systems that can effectively utilize natural language models.
[0006] The present invention has been made in consideration of the above-mentioned problems, and has an object to provide a content selection device and program that can select content that was filmed during a specified period from sentences related to events during the specified period using a natural language model. [Means for solving the problem]
[0007] In order to solve such problems, the content selection device of the invention of claim 1 is a content selection device that selects content, which is still images and / or videos taken during a specified period, based on sentences related to events during the specified period, and is characterized by comprising: a summary generation unit that generates a summary that summarizes one or more of the sentences using a natural language model generated by machine learning; an explanation generation unit that generates, for each of the one or more pieces of content, an explanation that explains the content of the content using a natural language model generated by machine learning; and a content selection unit that selects the explanation similar to the content of the summary and acquires the content corresponding to the selected explanation.
[0008] The invention of claim 2 is characterized in that, in the invention of claim 1, the content selection device, the summary generation unit generates the summary of the text related to the subject for which the content is selected, and the explanation generation unit generates the explanation for each of the contents in which the subject is photographed.
[0009] The invention of claim 3 is characterized in that, in the invention of claim 2, the object is a person.
[0010] The invention of claim 4 is characterized in that, in the invention of claim 1, the summary generation unit extracts the sentence related to the subject for selecting the content from the one or more sentences, and generates the summary of the extracted sentence.
[0011] The invention of claim 5 is the invention of claim 1, wherein the description generation unit extracts the content in which a subject for selecting the content is photographed from the one or more of the content by image recognition, and generates the description for each of the extracted content. It is characterized by:
[0012] The invention of claim 6 is characterized in that, in the invention of claim 1, the description generation unit extracts the description relating to the subject for selecting the content from the description generated for each of the contents, and the content selection unit selects the description similar to the content of the summary from the extracted description.
[0013] The invention of claim 7 is characterized in that, in the invention of claim 1, the content selection unit selects the description based on the cosine similarity between a summary vector indicating a vector of the summary and a description vector indicating a vector of the description.
[0014] The invention of claim 8 is a computer-readable program for causing a computer to function as the content selection device according to any one of claims 1 to 7. Effect of the Invention
[0015] According to the configuration of the present invention, a content selection device generates a summary that summarizes one or more sentences related to an event in a specified period using a natural language model, generates an explanation for each of one or more contents using the natural language model, selects an explanation similar to the content of the summary, and acquires the content corresponding to the selected explanation. With this configuration, it is possible to select content for a specified period from sentences related to events in the specified period using the natural language model. [Brief description of the drawings]
[0016] [Figure 1]1 is a diagram conceptually showing a configuration of a content selection system and a content selection device according to a first embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram showing an overview of content selection in the first embodiment of the present invention. [Diagram 3] FIG. 3 is a flow diagram showing a flow of content selection in the first embodiment of the present invention. [Figure 4] FIG. 11 is a diagram showing an overview of content selection in the second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0017] [First embodiment of the invention] Hereinafter, each embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a diagram conceptually showing the configuration of a content selection system and a content selection device according to a first embodiment of the present invention.
[0018] [System Usage] 1, for example, content selection system 1A is used by user 201 to create and record content using user terminal 2, and by user 202 to acquire the recorded content using user terminal 2. As will be described later, content selection system 1A of this embodiment 1 has a configuration for automatically selecting content that user 202 is likely to want to acquire from the content created and recorded by user 201, and having user 202 receive it.
[0019] The content selection system 1A of this embodiment 1 is used, for example, in child welfare facilities such as nurseries, and educational institutions such as kindergartens, elementary and junior high schools, to sell content to users 202, and to transfer content between users 201 and 202. This transfer of content may be a sale involving a fee, or a free transfer. For example, when the content selection system 1A is used in a nursery school, it is assumed that the user 201 is a nursery school teacher working at the nursery school, or a manager or administrator of the nursery school, and the user 202 is a parent who leaves their child at the nursery school. However, the users 201 and 202 may be people other than those mentioned above.
[0020] The content selection system 1A may be used by any legal entity, association, or group that sells or transfers content using the user terminal 2. For example, the system may be used by a company, a national or local government, a youth baseball or soccer team, or a community of sports, art, or performing arts enthusiasts. Furthermore, the content selection system 1A may be lent to the above-mentioned legal entity, association, or group for a fee or free of charge.
[0021] [content] The contents used in the content selection system 1A of this embodiment 1 are, for example, various data such as digital photos, digital videos, text documents, etc. That is, the contents in this embodiment 1 are various data that the user 202 using the user terminal 2 can obtain via the network 3, and that the users 201 and 202 using the user terminal 2 can exchange via the network 3.
[0022] For example, when the content selection system 1A of the first embodiment is used in a nursery school, the content is, for example, still images and / or videos. For example, this content is an image of a child taken by a nursery teacher or the like as the user 201 with a camera or the like of the user terminal 2. This content is mainly a photograph or video of a specific child who is the child of a specific parent or user 202, but it may also be a photograph or video of a child or landscape other than the child of the user 202. The photograph or video may be accompanied by image attribute information such as the shooting location and shooting date and time, or text or pictures written on the image by the nursery teacher or the like who took the photograph or video.
[0023] The content of this embodiment 1 can be acquired or exchanged using the user terminal 2, and may be data other than photographs and videos, such as text data or audio data of sentences that make up various documents, so long as it is suitable for the purpose or nature of the corporation, association, or group that uses the content selection system 1A and the purpose of the users 201, 202.
[0024] [Text] The sentences used in the content selection system 1A of the first embodiment are one or more sentences showing any semantic content, and / or one or more figures, symbols, etc. showing any semantic content. As will be described later, in the content selection system 1A of the first embodiment, the sentences are used to generate summaries used in content selection.
[0025] For example, when the content selection system 1A is used in a nursery school, the text is a communication text written by a nursery school teacher (user 201) or a parent of a child (user 202) in a communication notebook in which the nursery school teacher and the child's parent communicate with each other. Also, text in a document such as a diary or report in the nursery school created by the nursery school teacher (user 201) or text in a private childcare record document created by the parent (user 202) is also applicable. These texts are highly relevant to the children at the nursery school and are useful for identifying photos and videos of the children. Note that texts created by people other than the users 201 and 202 (for example, news articles or personal posts on SNS) may be used as the text. The text may be written in any language such as Japanese, English, or French, and may include figures, pictures, etc. When the text is a character, the text is mainly composed of text data, but handwritten characters on paper or imaged characters may also be used. Furthermore, audio data, video data, etc. may be used to create the text.
[0026] [System Configuration] Fig. 1 shows a system configuration diagram and a functional block diagram of the content selection system and the content selection device of this embodiment 1. The content selection system 1A of this embodiment 1 shown in Fig. 1 includes the content selection device 1 of this embodiment 1 and a user terminal 2, which are connected to each other via a network 3 so as to be able to communicate with each other.
[0027] [Content selection device] The content selection device 1 of the first embodiment shown in Fig. 1 is configured using a server computer. The content selection device 1 selects content, which is still images and / or videos taken during a specified period, based on text related to events during the specified period. As shown in Fig. 1, the content selection device 1 includes a summary generation unit 11, an explanation generation unit 12, a content selection unit 13, and a content selection database 14.
[0028] The summary generation unit 11 generates a summary that summarizes one or more sentences related to an event in a specified period. The summary generation unit 11 of the first embodiment includes a natural language model 11a generated by machine learning, and generates a summary of the sentences using the natural language model 11a. In the first embodiment, the sentences are recorded in a sentence database 4, which will be described later.
[0029] The explanatory text generation unit 12 generates explanatory text that explains the content of one or more pieces of content related to an event in a specified period. The explanatory text generation unit 12 in the first embodiment includes a natural language model 12a generated by machine learning, and generates a summary text using the natural language model 12a. In the first embodiment, the content is recorded in a content database 5, which will be described later.
[0030] The content selection unit 13 selects an explanatory sentence similar to the summary generated by the summary generation unit 11 from each of the explanatory sentences generated by the explanatory sentence generation unit 12, and acquires the content corresponding to the selected explanatory sentence.
[0031] The content selection unit 13 of embodiment 1 includes a summary sentence vector generation unit 13a that generates a summary sentence vector indicating the vector of the summary sentence generated by the summary sentence generation unit 11 described above, an explanation sentence vector generation unit 13b that generates an explanation sentence vector indicating each vector of the explanation sentence generated by the explanation sentence generation unit 12 described above, a similarity generation unit 13c that generates a similarity between each of the summary sentence vectors and the explanation sentence vectors, and a content acquisition unit 13d that selects a predetermined number of explanation sentences based on the generated similarities and acquires content corresponding to the selected explanation sentences.
[0032] The content selection database 14 is a database that manages information necessary for the functions of the content selection device 1. In the first embodiment, the content selection database 14 is configured as an RDBMS (Relational Database Management System) provided in the content selection device 1. The content selection database 14 records information that identifies the names of the children and the attributes of the children (for example, information such as the children's dates of birth, the classes they belong to, the names of their childcare workers, and the names of their parents).
[0033] It should be noted that any method may be selected to manage the information required for the functions of the content selection device 1. For example, instead of an RDBMS, the content selection database 14 may be a file arranged in a predetermined format in a predetermined area of a storage device provided in the content selection device 1. In addition, the text database 4 and content database 5 described later are used in the first embodiment, and the information required for the functions of the content selection device 1 may be managed using these databases.
[0034] As described above, in the first embodiment, the summary generation unit 11 includes the natural language model 11a, and generates a summary using the natural language model 11a. The natural language model 11a is a trained model generated by machine learning that generates a summary by summarizing one or more sentences from a target for which a summary is to be generated, that is, one or more sentences related to an event in a specified period. The natural language model 11a may be a well-known natural language model capable of generating a summary from one or more sentences, or may be a so-called large-scale language model generated by machine learning using a large amount of training data.
[0035] As the natural language model 11a in the first embodiment, it is possible to use a trained model having a very large number of parameters.
[0036] Specifically, it is possible to apply any one of the trained models illustrated in the following (Model Example 1) to (Model Example 5) to the natural language model 11a. (Model Example 1) BERT (Bidirectional Encoder Representations from Transformers): A model that takes into account the entire input document, learns the context of words, and obtains a vector representation of the document. (Model Example 2) GPT (Generative Pre-trained Transformer): A model that can learn the context before and after words in a document and obtain a vector representation of the document. (Model Example 3) XLNet (eXtreme Multi-task Learning with a Language Model): A model that can obtain a vector representation of a document by taking into account the positional information of words and learning the context. (Model Example 4) RoBERTa (Robustly Optimized BERT Pretraining Approach): A model trained on a larger dataset by improving the preprocessing of BERT training data. (Model Example 5) T5 (Text-to-Text Transfer Transformer): A model specialized in solving text-to-text conversion tasks and also capable of obtaining a vector representation of a document.
[0037] Any language model other than the above (Model Example 1) to (Model Example 5) may be used as the natural language model 11a. In addition, the summary sentence generation unit 11 may use a natural language model 11a that does not fall under a so-called large-scale language model.
[0038] In the first embodiment, the summary generation unit 11 is configured to include the natural language model 11a, but the natural language model 11a may be configured by a computer or a group of computers different from the content selection device 1. For example, when a function of processing natural language by a natural language model is provided in an information processing system configured by a computer different from the content selection device 1, the function of the information processing system may be the natural language model 11a. Also, for example, the natural language model 11a may be an interface with the information processing system, that is, a function of requesting the information processing system to generate a summary and receiving the summary that is the result of the processing from the information processing system.
[0039] In the first embodiment, the explanatory text generation unit 12 includes a natural language model 12a. The natural language model 12a is a trained model generated by machine learning that generates explanatory text indicating the details of content, based on the content being an image and / or a video, and the natural language model 12a may be configured according to the specific configuration of the content.
[0040] Furthermore, the natural language models 11a and 12a used in the first embodiment may be configured in part or in whole by a common natural language model, and for example, one natural language model may be used as the natural language model 11a used by the summary generation unit 11 and the natural language model 12a used by the explanatory generation unit 12. Any of the models exemplified in (Model Example 1) to (Model Example 5) above may be used as the natural language model 12a used by the explanatory generation unit 12. Also, like the above-mentioned natural language model 11a, any language model may be applied to the natural language model 12a.
[0041] In addition, in the first embodiment, as described above, vectors are generated for the summary and the description to determine the similarity. However, if the same vector can be generated from the same content regardless of the difference in language, the summary and the description may be in different languages, for example, the natural language model 11a of the summary generation unit 11 generates a summary from a sentence in Japanese, while the natural language model 12a of the description generation unit 12 generates a description of the content in English (see the explanation of step S22 described later). Alternatively, if the summary and the description are in different languages, the description may be translated into the language of the summary before determining the similarity.
[0042] In the first embodiment, the content selection device 1 is configured using a well-known server computer. A program that causes the server computer to function as the content selection device 1 is stored in advance in a hard disk drive (HDD), solid state drive (SSD), or other storage device of the server computer, and the program is loaded into the memory of the server computer and executed by a central processing unit (CPU), causing the server computer to function as the content selection device 1. Note that the hardware configuration of the content selection device 1 may be changed as desired, and for example, the content selection device 1 may be configured using two or more computers.
[0043] The text database 4 is a database that manages the above-mentioned texts. In the first embodiment, the text database 4 is an RDBMS that manages data related to texts such as journals, reports, contact books, and the like. The text database 4 may be a database dedicated to the content selection device 1, or a part or all of another information processing system may be connected to the content selection device 1 via the above-mentioned network 3 so as to be able to communicate with the content selection device 1.
[0044] The content database 5 is a database that manages the above-mentioned contents. The content database 5 in the first embodiment is configured as an RDBMS like the above-mentioned text database 4, and may be a database dedicated to the content selection device 1, or may be a database that is connected to the content selection device 1 via the above-mentioned network 3 to be able to communicate with a part or whole of another information processing system.
[0045] [User terminals and networks] Users 201, 202 who use the content selection device 1 use a user terminal 2 communicatively connected to the content selection device 1 via a network 3. The user terminal 2 is a well-known personal computer, smartphone, tablet computer, or other computer, and has a function for transmitting and receiving text and content to and from the content selection device 1. The user terminal 2 desirably has a camera function for taking content such as photos and videos of the children at the nursery school, but this function is not necessary.
[0046] 1 illustrates only one user terminal 2 for the sake of simplicity. However, when the content selection system 1A is actually implemented, there may be a plurality of user terminals 2. Specifically, each user 201 and each user 202 may use one or more computer terminals or mobile information terminals that they own or manage as the user terminal 2.
[0047] The network 3 is a computer network that communicatively connects the content selection device 1 and the user terminal 2. The network 3 may be a well-known computer network, for example, a wide area network such as the Internet, a LAN (Local Area Network), or a VPN (Virtual Private Network) constructed on a wide area network. The network 3 may be either a wired network or a wireless network, or may be configured by combining a wired network and a wireless network.
[0048] [Contents of processing] Next, a description will be given of the flow of a content selection process in embodiment 1. Fig. 2 is a diagram showing an overview of content selection in embodiment 1, and Fig. 3 is a flow diagram showing the flow of content selection in embodiment 1.
[0049] As described above, the content selection device 1 in the first embodiment is a device that selects content that was shot during a specified period based on text related to events during the specified period. As shown in Fig. 2 and Fig. 3, the content selection device 1 selects content through three steps: generating a summary text D12 (see step S1), generating an explanatory text D22 (see step S2), and selecting content D21 (see step S3).
[0050] The designated period here is a period used primarily for content selection.
[0051] For example, consider a case where the content selection system 1A is used in a nursery school. Then, consider a case where photos and videos of the daily lives of children at the nursery school and the status of various events during a specific period, for example, fiscal year 2023 (April 1, 2023 to March 31, 2024), are provided to the user 202 as content. In this case, fiscal year 2023, which is the target period for content selection, is set as the designated period. This designated period may be set automatically by the content selection unit 13, or may be set manually by the user 201 or the like using the user terminal 2.
[0052] In step S1, the content selection device 1 generates a summary D12 from a sentence D11 related to an event in a specified period by the summary generation unit 11. In the content selection device 1 in embodiment 1, the summary generation unit 11 acquires one or more sentences D11 related to an event in a specified period (see step S11), and then generates a summary D12 of the acquired sentence D11 (see step S12).
[0053] The document D11 is a document about an event during a specified period, and is a document written in a natural language about the event during the specified period. The document D11 may be, for example, a diary, a report, a contact book, or any other document.
[0054] In step S11, the summary generation unit 11 acquires sentences D11 related to events in the specified period. In the first embodiment, the sentences D11 are managed in the above-mentioned sentence database 4, and in step S11, the sentence database 4 is searched using the start and end dates of the specified period as conditions, and one or more sentences D11 that match the conditions, i.e., that relate to events in the specified period, are acquired.
[0055] In step S11, the method of acquiring the sentences D11 in the summary sentence generation unit 11 may be selected arbitrarily. For example, in step S11, one or more sentences D11 related to an event in a specified period may be transmitted from the user terminal 2 to the content selection device 1 by a user operation, without using the sentence database 4.
[0056] In this case, the summary generation unit 11 receives the transmitted document D11. In this case, the summary generation unit 11 may be configured to perform subsequent processing assuming that a period beginning with the date and time of the earliest event and ending with the date and time of the latest event is specified for one or more documents D11 received by the summary generation unit 11.
[0057] Also, it may be arbitrarily selected which information of the document D11 acquired by the summary generating unit 11 is used to determine that the document D11 is related to the specified period. For example, if the document D11 is a journal, it may be determined whether the date on which the journal was created belongs to the specified period. Also, for example, if the document D11 refers to a specific date and time, it may be determined whether the mentioned date and time belongs to the specified period.
[0058] Furthermore, the recording method and recording medium for the one or more documents D11 acquired by the summary generation unit 11 in step S11 may be selected arbitrarily. In the first embodiment, the document database 4 is used, but for example, a text file using a predetermined character code and a predetermined encoding method may be placed in a predetermined area of a storage device of the content selection device 1 or another computer.
[0059] The text D11 acquired by the summary generation unit 11 in step S11 may be, for example, a handwritten paper medium, an image of the paper medium, or sound or a video including sound. In this case, for example, the user may use the user terminal 2 to convert the text D11 on paper or in other formats into text format and transmit it to the content selection device 1.
[0060] Furthermore, a specific configuration of the content selection device 1, for example the summary generation unit 11, may include a character recognition unit that recognizes characters in an image and converts them into text format, or a voice recognition unit that converts voice or video including voice into text format through voice recognition.The summary generation unit 11 may be configured to acquire data in various formats, such as text, image, voice, video, and the like, as document D11 in step S11 through processing by the character recognition unit or voice recognition unit.
[0061] In addition, if the text D11 obtained in step S11 is written in a language other than the language supported by the natural language model 11a used to generate the summary in step S12, the text D11 may be translated in step S11 into a language that the natural language model 11a can handle.
[0062] When one or more sentences D11 relating to events in the specified period are acquired in step S11, the summary generation unit 11 of the content selection device 1 then generates a summary D12 summarizing the one or more sentences D11 (see step S12). As described above, the summary generation unit 11 in the first embodiment uses the natural language model 11a to generate a summary D12 indicating a summary of the one or more sentences D11 from the one or more sentences D11 acquired in step S11.
[0063] The summary sentence D12 generated in step S12 may be composed of, for example, a single sentence, or may be composed of two or more sentences. In the first embodiment, the summary sentence D12 is recorded in the content selection database 14 of the content selection device 1 in association with a designated period and information specifying one or more sentences D11 to be summarized by the summary sentence D12.
[0064] In the first embodiment, the process of step S1 is composed of step S11 of acquiring one or more documents D11 related to events in the specified period, and step S12 of summarizing the documents D11 acquired in step S11, but the method of generating the summary D12 based on the one or more documents D11 may be selected from any known method in addition to the above. For example, the document D11 may be acquired in step S11 without judging whether it belongs to the specified period, and the summary generation in step S12 may generate the summary D12 for events in the specified period from the acquired documents D11.
[0065] In step S2, for each of one or more contents D21, the explanation generator 12 of the content selection device 1 generates an explanation D22 indicating the details of the content. In the content selection device 1 in embodiment 1, the explanation generator 12 acquires one or more contents D21 (see step S21), and then generates an explanation D22 for each of the acquired contents D21 (see step S22).
[0066] The content D21 acquired in step S21 is a still image and / or a video, as described above. In the first embodiment, the content D21 is managed by the content database 5, as described above, and in step S21, the content D21 is acquired from the content database 5. The method of acquiring the content D21 in step S21 may be changed arbitrarily. For example, the user may operate the user terminal 2 to transmit the content D21 to the content selection device 1, and step S11 may be configured as a process of receiving the transmitted content D21.
[0067] The explanatory text generation unit 12 in the first embodiment generates explanatory text D22 for each of the contents D21 acquired in step S21 by using the natural language model 12a (see step S22). In the first embodiment, the explanatory text D22 generated in step S22 is data expressing the contents of the contents D21 by a single sentence. However, the configuration of the explanatory text D22 may be changed arbitrarily, and for example, the explanatory text D22 may be configured by two or more sentences.
[0068] In step S22, the explanatory text generation unit 12 may also arbitrarily select the language of the explanatory text D22, the recording method, etc. For example, in the case where an explanatory text vector described later can generate an identical or very similar vector according to the content of the explanatory text D22 regardless of the language, the explanatory text generation unit 12 may configure the explanatory text D22 in a language different from that of the summary text D12.
[0069] Here, examples of configurations in which the languages are different include converting an English summary sentence D12 into a Japanese explanatory sentence D22, converting a Japanese summary sentence S12 in kanji into an explanatory sentence D22 in hiragana or katakana, etc. In addition, when a difference occurs in the vector depending on the language, the explanatory sentence generation unit 12 may be configured to generate explanatory sentence D22 in a language different from that of the summary sentence D12, and then translate the explanatory sentence D22 into the same language as the summary sentence D12.
[0070] In embodiment 1, the generated explanatory text D22 is linked to information identifying the content D21 corresponding to the explanatory text D22, i.e., the content D21 from which the explanatory text D22 was generated and the content D21 whose contents are explained by the explanatory text D22, and is recorded in the content selection database 14 of the content selection device 1.
[0071] In the content selection in the first embodiment, the summary generation process in summary generation step S1 is the previous process and the explanatory text generation process in step S2 is the next process, but the order of these processes may be changed as desired. That is, the summary generation process in step S1 may be executed after the explanatory text generation process in step S2. Moreover, the summary generation process in step S1 and the explanatory text generation process in step S2 may be executed in parallel.
[0072] In addition, in the content selection in the first embodiment, the process of generating an explanatory text in step S2 may be configured to be automatically executed, for example, daily, weekly, or at other predetermined intervals. In this case, in step S21, the content D21 for which an explanatory text D22 has not been generated may be acquired from the content database 5, and the explanatory text D22 generated in step S22 may be linked to the content D21 and recorded in the content selection database 14.
[0073] After generating the summary D12 in step S1 and the explanation D22 in step S2, the content selection device 1 then selects content D21 by the content selection unit 13 (see step S3). Step S3 in the first embodiment includes generating a summary vector (see step S31), generating an explanation vector (see step S32), generating a similarity (see step S33), selecting an explanation (see step S34), and acquiring content (see step S35).
[0074] In step S31, the summary sentence vector generation unit 13a generates a summary sentence vector indicating the vector of the summary sentence D12 generated by the summary sentence generation unit 11 in the above-mentioned step S1. A known method of vectorizing a sentence may be used as a method for generating a vector of the summary sentence D12. For example, the number of words included in the summary sentence D12 or the frequency of occurrence may be vectorized as a feature. Alternatively, some or all of the words included in the summary sentence D12 may be mapped into a vector space to obtain a vector of the summary sentence D12 consisting of the vectorized words.
[0075] The summary sentence vector is recorded in the content selection database 14 of the content selection device 1 in association with information that identifies the summary sentence D12 from which the summary sentence vector was generated.
[0076] In step S32, the explanatory text vector generating unit 13b generates explanatory text vectors indicating the vectors of each explanatory text D22 generated by the explanatory text generating unit 12 in the above-mentioned step S2. The method for generating the vectors of the explanatory text D22 may be the same as the method for vectorizing the summary text D12 in step S31, or a well-known vectorization method capable of generating a vector capable of determining the similarity with the summary text vector. The explanatory text vector is recorded in the content selection database 14 of the content selection device 1 in association with information identifying the explanatory text D22 from which the explanatory text vector was generated.
[0077] After the summary sentence vector indicating the vector of the summary sentence D12 and the explanation sentence vector indicating the vector of the explanation sentence D22 are generated, the similarity between the summary sentence vector and the explanation sentence vector is generated (see step S33).
[0078] Any known method may be used as a method for generating the similarity, but in the first embodiment, the cosine value of the angle between the summary sentence vector and the description sentence vector is calculated as the cosine similarity. In the first embodiment, the cosine similarity is used as the similarity, and the inner product of the summary sentence vector and the description sentence vector is divided by the norm, i.e., the magnitude of the summary sentence vector and the description sentence vector, to generate the similarity as a value normalized within the range of -1 to 1. The generated similarity is linked to information specifying the summary sentence vector and the description sentence vector that are the source of the similarity, and recorded in the content selection database 14 of the content selection device 1.
[0079] In the first embodiment, the summary sentence vector and the explanation sentence vector can be obtained based on the models and methods exemplified in the following (method example 1) to (method example 4). (Method Example 1) Bag-of-words (BoW) model: Counts the words in a document and converts them into vectors as features. (Example Method 2) TF-IDF: An improved version of the BoW model, this determines the importance of a word by taking into account not only its frequency of occurrence, but also its frequency of occurrence across the entire document set. (Example Method 3) Word2Vec: A method that uses a neural network to map words into a vector space, obtaining a vector representation that captures the meaning of the word. (Method Example 4) Doc2Vec: Extends Word2Vec and converts the entire document into a vector representation.
[0080] After generating the similarity with the summary sentence vector for each of the description sentence vectors, the content selection device 1 then selects a predetermined number of description sentences D22 that are similar to the summary sentence D12 based on the similarity (see step S34), and acquires content D21 that corresponds to the selected description sentence D22, i.e., the content selection device 1 is the source of the description sentence D22 and whose contents are explained by the description sentence D22 (see step S34).
[0081] The method for selecting the explanatory text D22 in step S34 may be selected arbitrarily, but in embodiment 1, as described above, a cosine similarity normalized to the range of -1 to 1 is generated as the similarity, and a predetermined number of explanatory texts D22 are selected in order of cosine similarity closest to 1, i.e., in order of highest similarity.
[0082] In the first embodiment, as described above, the explanatory text D22 is recorded in the content selection database 14 in association with information identifying the content D21 corresponding to the explanatory text D22, and the information associated with the selected explanatory text D22 is used to obtain the content D21 that corresponds to the explanatory text D22, i.e., the source of the explanatory text D22 and whose contents are explained by the explanatory text D22.
[0083] This completes the description of embodiment 1. In embodiment 1, the summary generation unit 11 uses the natural language model 11a to generate a summary D12 that summarizes one or more sentences D11, the explanation generation unit 12 uses the natural language model 12a to generate an explanation D22 for each of one or more contents D21, and the content selection unit 13 selects an explanation D22 similar to the summary D12 and acquires the content D21 corresponding to the selected explanation D22.
[0084] In the first embodiment, this configuration allows content to be selected by utilizing a natural language model. The content selection device 1 in the first embodiment can be used, for example, to select photos by month or by term when creating a class album in a school or other organization.
[0085] [Embodiment 2 of the invention] Hereinafter, a second embodiment of the present invention will be described. In the second embodiment, in addition to the content selection in the first embodiment, content is selected using image recognition of people and other objects.
[0086] [Basic configuration] The content selection system of this embodiment 2 has the same basic configuration as the content selection system 1A shown in Fig. 1. However, in embodiment 2, the content selection device 1 constituting the content selection system 1A has a summary generation unit 11 and an explanation generation unit 12 each having an image recognition function. The content selection device 1 of embodiment 2 can use the result of image recognition when generating the summary D12 and the explanation D22.
[0087] For example, consider a case where the content selection system 1A of this embodiment 2 is used in a nursery school and selects content centered on an image of a specific object, for example, a child. Here, the summary generation unit 11 and the description generation unit 12 of the content selection device 1 of embodiment 2 have a function to recognize the face of a child (which may be a specific part of the face or a physical feature other than the face) that appears in a photograph or video. The summary generation unit 11 and the description generation unit 12 also have a function to identify the name of a child whose face has been recognized. Information that identifies the name and attributes of a child from the recognized face of the child (such as the child's date of birth, the class to which the child belongs, the name of the child's teacher, the name of the parent, etc.) is recorded in the content selection database 14.
[0088] The summary generation unit 11 and the description generation unit 12 can identify the name of a child from the face of the child that appears in a photograph or video, and generate a summary D12 and a description D22 using information such as the child's name and attributes.
[0089] [Processing Procedure] The processing procedure of the content selection system 1A of the second embodiment will be described using the above-mentioned example in which the content selection system 1A is used in a nursery school.
[0090] Fig. 4 is a diagram showing an overview of content selection in embodiment 2. As shown in Fig. 4, in embodiment 2, one or more documents D11 and contents D21 related to events in a specified period are narrowed down by a target D3.
[0091] The object D3 is, for example, the face of each child in a nursery school. The summary generation unit 11 and the explanation generation unit 12 recognize the face of each child by an image recognition function and set it as the object D3. When the summary generation unit 11 and the explanation generation unit 12 recognize the face image of the child set as the object D3, they use the recognition result to identify the name of the child.
[0092] As shown in Fig. 4, the summary generation unit 11 can extract, from all of the text D11, text D11 that includes the name of the child who is the target D3 (name A in Fig. 4) and a communication book or diary related to the child who is the target D3, and generate a summary D12 from the extracted text D11. As shown in Fig. 4, the explanation generation unit 12 can extract, from all of the content D21, content D21 that is a photo or video that includes the face or figure of the child who is the target D3, and generate an explanation D22 from the extracted content D21.
[0093] In this second embodiment, the target D3 used by the summary sentence generation unit 11 and the explanation sentence generation unit 12 may be other than the faces of the children, for example, a person other than the children, such as a specific nursery teacher or a parent, a specific building, a specific place, a specific event, etc. For example, in the first embodiment described above, it was explained that the content selection device 1 can be used to select photos to be used in a school or other class album, but in the second embodiment, it is also possible to select photos in a specific class album that include students, teachers, or other people, and also to select photos that include a school trip, a school festival, a sports festival, or other events, or places, buildings, etc. visited at the events.
[0094] Any method may be selected as a method for specifying the target D3, and for example, when the target D3 is a person, the target D3 may be specified using a name and / or a facial photograph, or when information on the person is managed in a database, the target D3 may be specified using an ID or other information that identifies the person in the database. These specifications may be made by automatic processing of the summary sentence generation unit 11 or the explanation sentence generation unit 12, or may be made by manual operation using the user terminal 2 of the users 201, 202.
[0095] In embodiment 2, when generating a summary sentence D12, first, one or more sentences D11 related to a target D3 are extracted from one or more sentences D11 related to events in a specified period, and then a summary sentence D12 is generated from the extracted one or more sentences D11.
[0096] In the second embodiment, the method of extracting the text D11 may be selected from among known methods. For example, the text D11 managed in the text database 4 may be searched for using the target D3 as a condition in addition to the specified period. For example, after obtaining the text D11 based on the specified period as in the first embodiment, extraction based on the target D3 may be performed. In this case, extraction may be performed based on whether the text D11 contains a word indicating the target D3. For example, a vector of a word indicating the target D3 may be generated, and the text D11 may be extracted if the similarity to the vector of the text D11 is equal to or higher than a predetermined value.
[0097] In the second embodiment, the method of generating summary sentences D12 from sentences D11 extracted using the target D3 is the same as that in the first embodiment.
[0098] In addition, in the second embodiment, when generating the explanatory text D22 for the content D21, first, one or more contents D21 in which the target D3 is photographed by image recognition are extracted from one or more contents D21, and then an explanatory text D22 is generated for each of the extracted one or more contents D21.
[0099] A specific method of image recognition in the second embodiment may be selected arbitrarily from known image processing techniques. In the second embodiment, a method of generating explanatory text D22 from extracted content D21 is the same as in the first embodiment. In addition, a method of extraction in generating explanatory text D22 may be selected arbitrarily.
[0100] In the second embodiment, extraction is performed by image recognition as described above, but for example, the process up to generation of the explanatory sentence D22 may be performed in the same manner as in the first embodiment, and explanatory sentences D22 relating to the target D3 may be extracted from the generated explanatory sentences D22, and subsequent processing may be performed for each of the extracted explanatory sentences D22.
[0101] For example, if there is no image of the target D3 and it can only be specified by its name, title, or other text information, the device can be configured to generate an explanatory text D22 and then extract the explanatory text D22 relating to the target D3, thereby narrowing down the search by the target D3.
[0102] In the second embodiment, after generating the summary sentence D12 and the explanation sentence D22, the content D21 is selected by the same process as in the first embodiment. That is, after generating a summary sentence vector indicating the vector of the summary sentence D12 and an explanation sentence vector indicating the vector of the explanation sentence D22, the similarity between the summary sentence vector and the explanation sentence vector is generated as a cosine similarity, and a predetermined number of explanation sentences D22 are selected in order of the cosine similarity closest to 1, that is, in order of the similarity between the summary sentence D12 and the explanation sentence D22.
[0103] This concludes the description of the second embodiment of the present invention. In the second embodiment, the content D21 can be selected not only by a specified period, but also by a person, a place, an event, or other target D3.
[0104] The embodiments of the present invention are not limited to the first and second embodiments. For example, in the second embodiment, the summary text D12 and the explanatory text D22 are both narrowed down by the target D3, but when generating the summary text vector, if the reference to the target D3 in the summary text D12 is sufficiently reflected in the value of the generated vector, it is presumed that the explanatory text D22 and the content D21, which are highly similar to the summary text D12, are also highly likely to be related to the target D3. In this case, extraction by the target D3 is performed only by generating the summary text D12, and the generation of the explanatory text D22 and the subsequent selection process may be configured in the same manner as in the first embodiment, and the content D21 may be selected as the content related to the target D3 based on its similarity to the summary text D12.
[0105] Other specific configurations are not limited to those of the present embodiment, and various modifications are possible without departing from the spirit of the present invention. [Explanation of symbols]
[0106] 1 Content selection device 2. User terminal 3. Network 4. Text Database 5 Content Database 11 Summary sentence generator 11a Natural Language Models 12 Description Generation Unit 12a Natural Language Models 13 Content Selection Department 13a Summary vector generation unit 13b Description vector generation section 13c Similarity generator 13d Content Acquisition Department 14 Content Selection Database
Claims
1. A content selection device that selects content, which is a still image and / or a video captured during a specified period, based on a sentence related to an event during the specified period, comprising: a summary generation unit that generates a summary by summarizing one or more of the sentences using a natural language model generated by machine learning; an explanatory text generation unit that generates explanatory text for each of the one or more pieces of content by using a natural language model generated by machine learning; a content selection unit that selects the explanatory text similar to the content of the summary text and acquires the content corresponding to the selected explanatory text; A content selection device comprising:
2. The content selection device includes: the summary generation unit generates the summary of the text related to a subject from which the content is selected; The description generation unit generates the description for each of the contents in which the object is photographed.
2. The content selection device according to claim 1.
3. 3. The content selection device according to claim 2, wherein the object is a person.
4. The summary generation unit extracts the sentence related to the subject for selecting the content from the one or more sentences, and generates the summary of the extracted sentence.
2. The content selection device according to claim 1.
5. The description generation unit extracts the content in which a subject for selecting the content is photographed from the one or more content by image recognition, and describes the description for each of the extracted content.
2. The content selection device according to claim 1.
6. the description generation unit extracts the description regarding a target for selecting the content from the description generated for each of the content; The content selection unit selects an explanatory sentence similar to the content of the summary sentence from the extracted explanatory sentences.
2. The content selection device according to claim 1.
7. The content selection unit selects the description based on a cosine similarity between a summary vector indicating a vector of the summary and a description vector indicating a vector of the description.
2. The content selection device according to claim 1.
8. A program for causing a computer to function as the content selection device according to any one of claims 1 to 7.
Citation Information
Patent Citations
Data generation method using language model, computer device, and computer program
JP2023018624A
Cited By
Class AB amplification device
CN121710826A
Information processing device, information processing method, and recording medium
WO2026160293A1
