System

Abstract

To provide a system that enables individual creators or local media who lack resources to generate a content according to their purposes.SOLUTION: A system according to the embodiment comprises an uploading unit, a selection unit, and a generation unit. The uploading unit uploads a topic. The selection unit selects a purpose based on the topic uploaded by the uploading unit. The generation unit generates a content according to the purpose selected by the selection unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, individual creators and local media had difficulty developing multimedia content due to a lack of resources.

[0005] The system according to the embodiment aims to enable individual creators and local media lacking in resources to generate content suited to their intended purpose. [Means for solving the problem]

[0006] The system according to the embodiment includes an upload unit, a selection unit, and a generation unit. The upload unit uploads content. The selection unit selects a use based on the content uploaded by the upload unit. The generation unit generates content according to the use selected by the selection unit. Effect of the Invention

[0007] The system according to the embodiment enables individual creators and local media that lack resources to generate content suited to specific purposes. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The content generation system according to the embodiment of the present invention is a system for media such as individual creators and local media that have stories but lack resources, where the users can focus on reporting, upload the stories (text, audio, video, etc.) onto a messenger app, and generate content according to the purpose through chat and use selection. In this system, a user uploads the stories (text, audio, video, etc.) that they have collected onto a messenger app and selects the purpose through chat, and a generation AI generates content. For example, the generation AI transcribes blogs from audio data, and then performs automatic translation and dubbing to enable global expansion. The generation AI can also create stamps and novels from illustrations. This mechanism allows individual creators and local media to focus on reporting while making up for the lack of resources and efficiently generate various content. As a result, the content generation system allows individual creators and local media to focus on reporting while making up for the lack of resources and efficiently generate various content.

[0029] The content generation system according to the embodiment includes an upload unit, a selection unit, and a generation unit. The upload unit uploads material (text, audio, video, etc.) that a user has collected on a messenger app. For example, audio data of an interview collected or video data that has been shot can be uploaded. The selection unit selects a use based on the material uploaded by the upload unit. For example, a use such as creating a blog transcript from audio data or performing a Vtuber broadcast can be selected. The generation unit generates content according to the use selected by the selection unit. For example, the generation AI transcribes a blog from audio data, and further performs automatic translation and dubbing to enable global expansion. The generation AI can also create stamps and novels from illustrations. For example, the generation AI receives audio data as input and generates a blog transcript. Next, the generation AI automatically translates the generated transcript and makes it compatible with multiple languages. Furthermore, the generation AI generates dubbed audio based on the translated text to enable global expansion. This allows the content generation system to efficiently perform operations from uploading material to selecting a use and generating content.

[0030] The uploading unit uploads the material (text, audio, video, etc.) that the user has collected on the messenger app. For example, the user can upload audio data of interviews collected and video data that the user has filmed. Specifically, the user can upload data directly from a smartphone or computer through the messenger app. The uploaded data is stored on a cloud server and used for subsequent processing. In the case of audio data, the user records interviews or meetings and uploads the audio files. In the case of video data, the user uploads footage filmed at events or on-site. In the case of text data, the user can upload interview notes and drafts of articles. The uploading unit centrally manages these data and performs appropriate processing according to the type and format of the data. For example, the audio data is automatically sent to a voice recognition system, and the text data is sent to a natural language processing system. Furthermore, the uploading unit adds meta information (for example, upload date and time, user ID, type of data, etc.) to the data to ensure smooth subsequent processing. In this way, the uploading unit efficiently collects the material that the user has collected and plays a role in smoothly proceeding with the processing of the entire system.

[0031] The selection unit selects a use based on the material uploaded by the upload unit. For example, the use can be selected to create a blog transcription from the audio data or to broadcast a Vtuber. Specifically, the selection unit analyzes the type and content of the uploaded data and suggests the most suitable use. For example, in the case of audio data, the selection unit analyzes the content using voice recognition technology, and suggests transcription if it is a record of an interview or a meeting. In the case of video data, the selection unit analyzes the content using video analysis technology, and suggests creating a highlight video or broadcasting a Vtuber if it is a record of an event. In the case of text data, the selection unit analyzes the content using natural language processing technology, and suggests blog posting or creating a news article if it is a draft of an article. The selection unit can also learn the user's past selection history and preferences to make more personalized suggestions. For example, if a specific user frequently transcribes blogs, the selection unit will give priority to proposing transcription to that user. In addition, the selection unit provides an interface for the user to select the proposed use, making it intuitive and easy to operate. As a result, the selection unit selects the most suitable use based on the uploaded material, and supports content generation according to the user's needs.

[0032] The generation unit generates content according to the purpose selected by the selection unit. For example, the generation AI transcribes blogs from voice data, and then performs automatic translation and dubbing to enable global expansion. The generation AI can also create stamps and novels from illustrations. Specifically, the generation AI receives voice data as input and converts it into text using voice recognition technology. This text is then adjusted for grammar and expressions using natural language processing technology to make it easy to read. The generation AI then sends the generated transcription to an automatic translation system to translate it into multiple languages. The translated text can then be dubbed using voice synthesis technology to provide it as video or audio content. In the case of illustrations, the generation AI analyzes the input illustrations and uses them as illustrations for stamps and novels. In the case of stamps, the generation AI converts the illustrations into the appropriate size and format so that they can be used in messenger apps. In the case of novels, the generation AI generates a story based on the illustrations and creates content that combines text and illustrations. The generation unit automatically performs these processes, allowing users to generate high-quality content without any effort. In addition, the generator stores the generated content on a cloud server so that the user can access it at any time, thereby enabling the generator to efficiently generate a variety of content according to the selected purpose and meet the needs of the user.

[0033] The voice data transcribing unit may include a unit that transcribes voice data. The voice data may be in, for example, an MP3 format, a WAV format, an AAC format, or the like, but is not limited to such examples. The unit that transcribes voice data may convert the voice data into text data using, for example, a generation AI. For example, the generation AI may receive voice data as an input and generate a verbatim transcription. The generation AI may also extract the main points of the voice data and generate a summary transcription. Furthermore, the generation AI may adjust the format based on the content of the voice data and generate a visually easy-to-understand transcription. This allows the voice data to be transcribed to generate text content. Some or all of the above-mentioned processing in the unit that transcribes voice data may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the unit that transcribes voice data may input the voice data to the generation AI and cause the generation AI to generate text data.

[0034] The content may be provided with an automatic translation unit that performs automatic translation. The automatic translation includes, but is not limited to, technologies such as machine translation and neural network translation. The automatic translation unit translates text data into multiple languages, for example, using a generation AI. For example, the generation AI receives text data as input and generates a word-for-word translation. The generation AI can also understand the context of the text data and generate a free translation. Furthermore, the generation AI can adjust the format of the translated text and generate a translation that is visually easy to understand. This allows the content to be made multilingual by automatic translation. Some or all of the above-mentioned processing in the automatic translation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the automatic translation unit can input text data to the generation AI and have the generation AI generate a translation.

[0035] The system may include an illustration creation unit that creates stamps from illustrations. The illustrations include, for example, hand-drawn illustrations and digital illustrations, but are not limited to such examples. The illustration creation unit converts illustration data into stamp data, for example, using a generation AI. For example, the generation AI receives illustration data as an input and generates a stamp design. The generation AI can also adjust the size and color of the stamp based on the content of the illustration data. Furthermore, the generation AI can also emphasize the visual elements of the stamp and generate a stamp with high visibility. This makes it possible to generate visual content by creating stamps from illustrations. Some or all of the above-mentioned processes in the illustration creation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the illustration creation unit may input illustration data to the generation AI and cause the generation AI to generate stamp data.

[0036] The novel creation unit may include a novel creation unit that creates a novel from illustrations. Novels include, but are not limited to, short stories, full-length novels, and light novels. The novel creation unit converts illustration data into novel data, for example, using a generation AI. For example, the generation AI receives illustration data as an input and generates a story of a novel. The generation AI can also describe characters and settings in detail based on the content of the illustration data. Furthermore, the generation AI can adjust the style and perspective of the novel to generate a novel that is attractive to readers. In this way, a narrative content can be generated by creating a novel from illustrations. Some or all of the above-mentioned processes in the novel creation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the novel creation unit can input illustration data to the generation AI and cause the generation AI to generate novel data.

[0037] The Vtuber distribution unit may include a unit for performing Vtuber distribution. Vtuber distribution includes, for example, live streaming and recorded distribution, but is not limited to such examples. The unit for performing Vtuber distribution controls the Vtuber character using a generation AI, for example, and performs distribution. For example, the generation AI generates the actions and expressions of the Vtuber character in real time and reflects them in the distribution. The generation AI can also generate the lines and performance of the Vtuber based on the distribution content. Furthermore, the generation AI can dynamically change the behavior of the Vtuber according to the comments and reactions of the viewers. This makes it possible to generate live streaming content by performing Vtuber distribution. Part or all of the above-mentioned processing in the unit for performing Vtuber distribution may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the unit for performing Vtuber distribution can cause the generation AI to control the Vtuber character.

[0038] The upload unit can analyze the user's past upload history and select the optimal upload method. For example, the upload unit can automatically display the types of content that the user frequently uploaded in the past as candidates. The upload unit can also preferentially suggest the upload method (audio, text, etc.) that the user used in the past. Furthermore, the upload unit can predict and suggest the upload method to be used in a specific time period from the user's past upload history. In this way, the optimal upload method can be provided by analyzing the past upload history. A part or all of the above-mentioned processing in the upload unit may be performed, for example, using AI, or may be performed without using AI. For example, the upload unit can input the user's past upload history data into AI and cause the AI ​​to select the optimal upload method.

[0039] The upload unit can apply different upload protocols depending on the type of material at the time of uploading. For example, in the case of text data, the upload unit automatically applies a character limit and format. In addition, in the case of audio data, the upload unit can also apply an optimal upload protocol depending on the sound quality and file format. Furthermore, in the case of video data, the upload unit can also apply an optimal upload protocol depending on the resolution and file size. This allows efficient uploading by applying an optimal upload protocol depending on the type of material. Part or all of the above-mentioned processing in the upload unit may be performed using, for example, AI, or may be performed without using AI. For example, the upload unit can input material type data to AI and cause AI to apply the optimal upload protocol.

[0040] The uploading unit can prioritize uploading highly relevant material in consideration of the geographical location information of the user at the time of uploading. For example, when the user is in a specific area, the uploading unit prioritizes uploading material related to the area. In addition, when the user is traveling, the uploading unit can also prioritize uploading material related to the travel destination. Furthermore, when the user is at an event venue, the uploading unit can also prioritize uploading material related to the event. In this way, by considering the geographical location information of the user, highly relevant material can be uploaded preferentially. Part or all of the above-mentioned processing in the uploading unit may be performed, for example, using AI, or may be performed without using AI. For example, the uploading unit can input the geographical location information data of the user to AI and cause the AI ​​to select highly relevant material.

[0041] The uploading unit can analyze the user's social media activity and upload related material at the time of uploading. For example, the uploading unit preferentially uploads material related to topics that the user is talking about on social media. The uploading unit can also upload material related to the posted content of an account that the user follows. Furthermore, the uploading unit can also upload material related to topics of groups and communities in which the user participates. This makes it possible to efficiently upload related material by analyzing the user's social media activity. A part or all of the above-mentioned processing in the uploading unit may be performed using, for example, AI, or may be performed without using AI. For example, the uploading unit can input the user's social media activity data into AI and cause the AI ​​to select related material.

[0042] The selection unit can adjust the priority of the uses based on the importance of the material at the time of selection. For example, the selection unit preferentially displays important material to make it easier to select. The selection unit can also postpone less important material and display it lower among the options. Furthermore, the selection unit can dynamically change the display order of the options according to the importance of the material. This allows important material to be preferentially selected by adjusting the priority of the uses based on the importance of the material. A part or all of the above-mentioned processing in the selection unit may be performed, for example, using AI or may be performed without using AI. For example, the selection unit can input the importance data of the material to AI and cause AI to adjust the priority of the uses.

[0043] The selection unit can apply different selection algorithms depending on the category of the material when making a selection. For example, in the case of a text material, the selection unit applies a keyword matching algorithm. In addition, in the case of an audio material, the selection unit can also apply a voice recognition algorithm. Furthermore, in the case of a video material, the selection unit can also apply a video analysis algorithm. This allows for efficient selection by applying a selection algorithm according to the material category. Some or all of the above-mentioned processing in the selection unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection unit can input material category data to AI and cause the AI ​​to apply the selection algorithm.

[0044] At the time of selection, the selection unit can determine the priority of the uses based on the submission time of the material. For example, the selection unit preferentially displays the most recently submitted material. The selection unit can also postpone the use of material that has been submitted earlier. Furthermore, the selection unit can dynamically change the display order of the options according to the submission time. In this way, the priority of the uses can be determined based on the submission time of the material, so that the latest material can be preferentially selected. A part or all of the above-mentioned processing in the selection unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection unit can input the submission time data of the material to AI and have the AI ​​execute the determination of the priority of the uses.

[0045] The selection unit can adjust the order of uses based on the relevance of the material at the time of selection. The selection unit, for example, preferentially displays highly relevant material. The selection unit can also postpone less relevant material. Furthermore, the selection unit can dynamically change the display order of options according to the relevance of the material. In this way, by adjusting the order of uses based on the relevance of the material, it is possible to preferentially select highly relevant material. A part or all of the above-mentioned processing in the selection unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection unit can input the relevance data of the material to AI and cause the AI ​​to adjust the order of uses.

[0046] The generating unit can adjust the level of detail of the generation based on the importance of the material at the time of generation. For example, the generating unit generates important material in detail to increase the amount of information. The generating unit can also generate material with low importance by simplifying it. Furthermore, the generating unit can dynamically change the level of detail of the generation according to the importance of the material. As a result, important material can be generated in detail by adjusting the level of detail of the generation based on the importance of the material. A part or all of the above-mentioned processing in the generating unit may be performed, for example, using AI, or may be performed without using AI. For example, the generating unit can input the importance data of the material to AI and cause AI to adjust the level of detail of the generation.

[0047] The generation unit can apply different generation algorithms depending on the category of the material at the time of generation. For example, in the case of a text material, the generation unit applies a natural language generation algorithm. In addition, in the case of an audio material, the generation unit can also apply a voice synthesis algorithm. Furthermore, in the case of a video material, the generation unit can also apply a video generation algorithm. This enables efficient generation by applying a generation algorithm according to the material category. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the generation unit can input material category data to AI and cause the AI ​​to apply the generation algorithm.

[0048] The generation unit can determine the generation priority based on the submission time of the material at the time of generation. For example, the generation unit generates the most recently submitted material with priority. The generation unit can also postpone the material that was submitted earlier. Furthermore, the generation unit can dynamically change the generation priority according to the submission time. In this way, the generation priority can be determined based on the submission time of the material, so that the latest material can be generated with priority. A part or all of the above-mentioned processing in the generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the generation unit can input the submission time data of the material to AI and have the AI ​​execute the determination of the generation priority.

[0049] The generation unit can adjust the order of generation based on the relevance of the material at the time of generation. For example, the generation unit generates material with high relevance with priority. The generation unit can also postpone material with low relevance. Furthermore, the generation unit can dynamically change the order of generation according to the relevance of the material. In this way, by adjusting the order of generation based on the relevance of the material, material with high relevance can be generated with priority. A part or all of the above-mentioned processing in the generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the generation unit can input the relevance data of the material to AI and cause AI to adjust the order of generation.

[0050] The unit that transcribes the voice data can adjust the level of detail of the transcription based on the importance of the voice during transcription. For example, the unit that transcribes the voice data transcribes important voice in detail to increase the amount of information. The unit that transcribes the voice data can also transcribe voice with low importance by simplifying it. Furthermore, the unit that transcribes the voice data can dynamically change the level of detail of the transcription according to the importance of the voice. As a result, important voice can be transcribed in detail by adjusting the level of detail of the transcription based on the importance of the voice. A part or all of the above-mentioned processing in the unit that transcribes the voice data may be performed using, for example, AI, or may be performed without using AI. For example, the unit that transcribes the voice data can input the importance data of the voice into AI and cause AI to adjust the level of detail of the transcription.

[0051] The unit that transcribes the voice data can determine the priority of transcription based on the submission time of the voice when transcribing. For example, the unit that transcribes the voice data prioritizes transcription of the most recently submitted voice. In addition, the unit that transcribes the voice data can postpone the voice that was submitted earlier. Furthermore, the unit that transcribes the voice data can dynamically change the priority of transcription according to the submission time. In this way, by determining the priority of transcription based on the submission time of the voice, the latest voice can be preferentially transcribed. A part or all of the above-mentioned processing in the unit that transcribes the voice data may be performed, for example, using AI or may be performed without using AI. For example, the unit that transcribes the voice data can input the submission time data of the voice into AI and cause AI to execute the determination of the priority of transcription.

[0052] The automatic translation unit can adjust the level of detail of the translation based on the importance of the material during translation. For example, the automatic translation unit translates important material in detail to increase the amount of information. The automatic translation unit can also simplify and translate material with low importance. Furthermore, the automatic translation unit can dynamically change the level of detail of the translation according to the importance of the material. As a result, important material can be translated in detail by adjusting the level of detail of the translation based on the importance of the material. Some or all of the above-mentioned processing in the automatic translation unit may be performed, for example, using AI, or may be performed without using AI. For example, the automatic translation unit can input the importance data of the material to AI and have the AI ​​adjust the level of detail of the translation.

[0053] The automatic translation unit can determine the priority of translation based on the submission time of the material during translation. For example, the automatic translation unit gives priority to translating the most recently submitted material. The automatic translation unit can also postpone material that has been submitted a long time ago. Furthermore, the automatic translation unit can dynamically change the priority of translation according to the submission time. In this way, by determining the priority of translation based on the submission time of the material, the latest material can be translated with priority. Some or all of the above-mentioned processing in the automatic translation unit may be performed, for example, using AI, or may be performed without using AI. For example, the automatic translation unit can input the submission time data of the material to AI and have the AI ​​execute the determination of the translation priority.

[0054] The illustration creation unit can adjust the level of detail of the stamp based on the importance of the illustration when creating the stamp. For example, the creation unit stamps important illustrations in detail to increase the amount of information. The illustration creation unit can also simplify illustrations with low importance and create stamps. Furthermore, the illustration creation unit can dynamically change the level of detail of the stamp according to the importance of the illustration. As a result, important illustrations can be stamped in detail by adjusting the level of detail of the stamp based on the importance of the illustration. A part or all of the above-mentioned processing in the illustration creation unit may be performed, for example, using AI or may be performed without using AI. For example, the illustration creation unit can input the importance data of the illustration to AI and cause AI to adjust the level of detail of the stamp.

[0055] When creating a stamp, the illustration creation unit can determine the priority of the stamp based on the submission time of the illustration. For example, the illustration creation unit can prioritize the most recently submitted illustration as a stamp. The illustration creation unit can also postpone illustrations submitted earlier. Furthermore, the illustration creation unit can dynamically change the priority of stamping according to the submission time. In this way, by determining the priority of stamps based on the submission time of the illustration, the latest illustrations can be prioritized as stamps. A part or all of the above-mentioned processing in the illustration creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the illustration creation unit can input the submission time data of the illustration to AI and have the AI ​​execute the determination of the priority of stamps.

[0056] The novel creation unit can adjust the level of detail of the novel based on the importance of the illustrations when creating the novel. For example, the novel creation unit can novelize important illustrations in detail to increase the amount of information. The novel creation unit can also simplify and novelize illustrations with low importance. Furthermore, the novel creation unit can dynamically change the level of detail of the novel according to the importance of the illustrations. As a result, important illustrations can be novelized in detail by adjusting the level of detail of the novel based on the importance of the illustrations. A part or all of the above-mentioned processing in the novel creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the novel creation unit can input importance data of the illustrations to AI and cause the AI ​​to adjust the level of detail of the novel.

[0057] The novel creation unit can determine the priority of novels based on the submission time of illustrations when creating a novel. For example, the novel creation unit prioritizes novelization of recently submitted illustrations. The novel creation unit can also postpone illustrations submitted earlier. Furthermore, the novel creation unit can dynamically change the priority of novelization according to the submission time. In this way, by determining the priority of novels based on the submission time of illustrations, the latest illustrations can be prioritized for novelization. A part or all of the above-mentioned processing in the novel creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the novel creation unit can input illustration submission time data into AI and have the AI ​​execute the determination of the priority of novels.

[0058] The unit performing Vtuber distribution can adjust the level of detail of the distribution based on the importance of the material during distribution. For example, the unit performing Vtuber distribution can distribute important material in detail to increase the amount of information. In addition, the unit performing Vtuber distribution can also simplify and distribute material with low importance. Furthermore, the unit performing Vtuber distribution can dynamically change the level of detail of the distribution according to the importance of the material. As a result, important material can be distributed in detail by adjusting the level of detail of the distribution based on the importance of the material. Part or all of the above-mentioned processing in the unit performing Vtuber distribution may be performed using, for example, AI, or may be performed without using AI. For example, the unit performing Vtuber distribution can input the importance data of the material into AI and have AI adjust the level of detail of the distribution.

[0059] The Vtuber distribution unit can determine the priority of distribution based on the submission time of the material at the time of distribution. For example, the Vtuber distribution unit prioritizes the distribution of recently submitted material. In addition, the Vtuber distribution unit can postpone material submitted earlier. Furthermore, the Vtuber distribution unit can dynamically change the priority of distribution according to the submission time. In this way, by determining the priority of distribution based on the submission time of the material, the latest material can be preferentially distributed. Part or all of the above-mentioned processing in the Vtuber distribution unit may be performed using, for example, AI, or may be performed without using AI. For example, the Vtuber distribution unit can input the submission time data of the material into AI and have AI execute the determination of the priority of distribution.

[0060] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0061] The content generation system may further include a feedback unit. The feedback unit may collect user feedback on the generated content and reflect it in the next content generation. For example, when a user provides a comment or rating on a generated blog post, the generation AI may learn the feedback and provide more accurate content the next time it is generated. The feedback unit may also collect user feedback in real time and immediately reflect it in the generation AI. Furthermore, the feedback unit may analyze the user feedback and provide data for improving the algorithm of the generation AI. This allows the content generation system to provide content that is more suited to the needs of the user.

[0062] The automatic translation unit can use a specialized dictionary to further improve the accuracy of the translation. For example, when translating a text in the medical field, an accurate translation can be performed by referring to a medical dictionary. Also, when translating a text in the legal field, a legal dictionary can be used to provide a specialized translation. Furthermore, when translating a text in a technical field, a technical dictionary can be used to achieve a highly accurate translation. This makes it possible to provide an accurate translation specialized for a specialized field.

[0063] The illustration creation unit can further customize the design of the stamp based on the user's preferences. For example, if the user likes a particular color, a stamp based on that color can be generated. Also, if the user likes a particular character style, a stamp that matches that style can be created. Furthermore, if the user likes a particular theme, a stamp that matches that theme can be provided. In this way, stamps can be generated according to the user's preferences.

[0064] Vtuber streamers can also analyze viewer reactions in real time and dynamically adjust the content of their stream. For example, if viewers are sending many likes and comments, they can make the stream more exciting. Also, if viewers are bored, they can change the content of the stream to pique their interest. Furthermore, if viewers are interested in a particular topic, they can add content related to that topic. This allows for flexible streaming based on viewer reactions.

[0065] The upload unit can further analyze the user's past upload history and suggest the optimal upload time. For example, if the user has uploaded a lot of content in a particular time period in the past, that time period can be suggested. Also, if the user has uploaded a lot of content on a particular day of the week, that day of the week can be suggested. Furthermore, if the user has uploaded content related to a particular event or season, that timing can be suggested. In this way, the optimal upload time can be provided based on the user's past upload history.

[0066] The upload unit can further take into account the user's network conditions when applying different upload protocols depending on the type of content. For example, if the user is in a slow network environment, a lightweight protocol can be applied to quickly upload. If the user is in a high-speed network environment, a high-quality protocol can be applied to upload detailed data. Furthermore, if the user is in an unstable network environment, a protocol that supports intermittent uploads can be applied. This makes it possible to provide the optimal upload protocol according to the user's network conditions.

[0067] The process flow of the first embodiment will be briefly described below.

[0068] Step 1: The uploading unit uploads the material (text, audio, video, etc.) that the user has collected onto the messenger app. For example, audio data of an interview or video data that has been filmed can be uploaded. Step 2: The selection unit selects a use based on the content uploaded by the upload unit. For example, the selection unit can select a use such as creating a blog transcript from the audio data or streaming as a VTuber. Step 3: The generation unit generates content according to the purpose selected by the selection unit. For example, the generation AI transcribes blogs from audio data, and then performs automatic translation and dubbing to enable global expansion. The generation AI can also create stamps and novels from illustrations. For example, the generation AI receives audio data as input and generates a blog transcript. The generation AI then automatically translates the generated transcript to make it compatible with multiple languages. The generation AI then generates dubbed audio based on the translated text to enable global expansion.

[0069] (Example 2) The content generation system according to the embodiment of the present invention is a system for media such as individual creators and local media that have stories but lack resources, where the users can focus on reporting, upload the stories (text, audio, video, etc.) onto a messenger app, and generate content according to the purpose through chat and use selection. In this system, a user uploads the stories (text, audio, video, etc.) that they have collected onto a messenger app and selects the purpose through chat, and a generation AI generates content. For example, the generation AI transcribes blogs from audio data, and then performs automatic translation and dubbing to enable global expansion. The generation AI can also create stamps and novels from illustrations. This mechanism allows individual creators and local media to focus on reporting while making up for the lack of resources and efficiently generate various content. As a result, the content generation system allows individual creators and local media to focus on reporting while making up for the lack of resources and efficiently generate various content.

[0070] The content generation system according to the embodiment includes an upload unit, a selection unit, and a generation unit. The upload unit uploads material (text, audio, video, etc.) that a user has collected on a messenger app. For example, audio data of an interview collected or video data that has been shot can be uploaded. The selection unit selects a use based on the material uploaded by the upload unit. For example, a use such as creating a blog transcript from audio data or performing a Vtuber broadcast can be selected. The generation unit generates content according to the use selected by the selection unit. For example, the generation AI transcribes a blog from audio data, and further performs automatic translation and dubbing to enable global expansion. The generation AI can also create stamps and novels from illustrations. For example, the generation AI receives audio data as input and generates a blog transcript. Next, the generation AI automatically translates the generated transcript and makes it compatible with multiple languages. Furthermore, the generation AI generates dubbed audio based on the translated text to enable global expansion. This allows the content generation system to efficiently perform operations from uploading material to selecting a use and generating content.

[0071] The uploading unit uploads the material (text, audio, video, etc.) that the user has collected on the messenger app. For example, the user can upload audio data of interviews collected and video data that the user has filmed. Specifically, the user can upload data directly from a smartphone or computer through the messenger app. The uploaded data is stored on a cloud server and used for subsequent processing. In the case of audio data, the user records interviews or meetings and uploads the audio files. In the case of video data, the user uploads footage filmed at events or on-site. In the case of text data, the user can upload interview notes and drafts of articles. The uploading unit centrally manages these data and performs appropriate processing according to the type and format of the data. For example, the audio data is automatically sent to a voice recognition system, and the text data is sent to a natural language processing system. Furthermore, the uploading unit adds meta information (for example, upload date and time, user ID, type of data, etc.) to the data to ensure smooth subsequent processing. In this way, the uploading unit efficiently collects the material that the user has collected and plays a role in smoothly proceeding with the processing of the entire system.

[0072] The selection unit selects a use based on the material uploaded by the upload unit. For example, the use can be selected to create a blog transcription from the audio data or to broadcast a Vtuber. Specifically, the selection unit analyzes the type and content of the uploaded data and suggests the most suitable use. For example, in the case of audio data, the selection unit analyzes the content using voice recognition technology, and suggests transcription if it is a record of an interview or a meeting. In the case of video data, the selection unit analyzes the content using video analysis technology, and suggests creating a highlight video or broadcasting a Vtuber if it is a record of an event. In the case of text data, the selection unit analyzes the content using natural language processing technology, and suggests blog posting or creating a news article if it is a draft of an article. The selection unit can also learn the user's past selection history and preferences to make more personalized suggestions. For example, if a specific user frequently transcribes blogs, the selection unit will give priority to proposing transcription to that user. In addition, the selection unit provides an interface for the user to select the proposed use, making it intuitive and easy to operate. As a result, the selection unit selects the most suitable use based on the uploaded material, and supports content generation according to the user's needs.

[0073] The generation unit generates content according to the purpose selected by the selection unit. For example, the generation AI transcribes blogs from voice data, and then performs automatic translation and dubbing to enable global expansion. The generation AI can also create stamps and novels from illustrations. Specifically, the generation AI receives voice data as input and converts it into text using voice recognition technology. This text is then adjusted for grammar and expressions using natural language processing technology to make it easy to read. The generation AI then sends the generated transcription to an automatic translation system to translate it into multiple languages. The translated text can then be dubbed using voice synthesis technology to provide it as video or audio content. In the case of illustrations, the generation AI analyzes the input illustrations and uses them as illustrations for stamps and novels. In the case of stamps, the generation AI converts the illustrations into the appropriate size and format so that they can be used in messenger apps. In the case of novels, the generation AI generates a story based on the illustrations and creates content that combines text and illustrations. The generation unit automatically performs these processes, allowing users to generate high-quality content without any effort. In addition, the generator stores the generated content on a cloud server so that the user can access it at any time, thereby enabling the generator to efficiently generate a variety of content according to the selected purpose and meet the needs of the user.

[0074] The voice data transcribing unit may include a unit that transcribes voice data. The voice data may be in, for example, an MP3 format, a WAV format, an AAC format, or the like, but is not limited to such examples. The unit that transcribes voice data may convert the voice data into text data using, for example, a generation AI. For example, the generation AI may receive voice data as an input and generate a verbatim transcription. The generation AI may also extract the main points of the voice data and generate a summary transcription. Furthermore, the generation AI may adjust the format based on the content of the voice data and generate a visually easy-to-understand transcription. This allows the voice data to be transcribed to generate text content. Some or all of the above-mentioned processing in the unit that transcribes voice data may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the unit that transcribes voice data may input the voice data to the generation AI and cause the generation AI to generate text data.

[0075] The content may be provided with an automatic translation unit that performs automatic translation. The automatic translation includes, but is not limited to, technologies such as machine translation and neural network translation. The automatic translation unit translates text data into multiple languages, for example, using a generation AI. For example, the generation AI receives text data as input and generates a word-for-word translation. The generation AI can also understand the context of the text data and generate a free translation. Furthermore, the generation AI can adjust the format of the translated text and generate a translation that is visually easy to understand. This allows the content to be made multilingual by automatic translation. Some or all of the above-mentioned processing in the automatic translation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the automatic translation unit can input text data to the generation AI and have the generation AI generate a translation.

[0076] The system may include an illustration creation unit that creates stamps from illustrations. The illustrations include, for example, hand-drawn illustrations and digital illustrations, but are not limited to such examples. The illustration creation unit converts illustration data into stamp data, for example, using a generation AI. For example, the generation AI receives illustration data as an input and generates a stamp design. The generation AI can also adjust the size and color of the stamp based on the content of the illustration data. Furthermore, the generation AI can also emphasize the visual elements of the stamp and generate a stamp with high visibility. This makes it possible to generate visual content by creating stamps from illustrations. Some or all of the above-mentioned processes in the illustration creation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the illustration creation unit may input illustration data to the generation AI and cause the generation AI to generate stamp data.

[0077] The novel creation unit may include a novel creation unit that creates a novel from illustrations. Novels include, but are not limited to, short stories, full-length novels, and light novels. The novel creation unit converts illustration data into novel data, for example, using a generation AI. For example, the generation AI receives illustration data as an input and generates a story of a novel. The generation AI can also describe characters and settings in detail based on the content of the illustration data. Furthermore, the generation AI can adjust the style and perspective of the novel to generate a novel that is attractive to readers. In this way, a narrative content can be generated by creating a novel from illustrations. Some or all of the above-mentioned processes in the novel creation unit may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the novel creation unit can input illustration data to the generation AI and cause the generation AI to generate novel data.

[0078] The Vtuber distribution unit may include a unit for performing Vtuber distribution. Vtuber distribution includes, for example, live streaming and recorded distribution, but is not limited to such examples. The unit for performing Vtuber distribution controls the Vtuber character using a generation AI, for example, and performs distribution. For example, the generation AI generates the actions and expressions of the Vtuber character in real time and reflects them in the distribution. The generation AI can also generate the lines and performance of the Vtuber based on the distribution content. Furthermore, the generation AI can dynamically change the behavior of the Vtuber according to the comments and reactions of the viewers. This makes it possible to generate live streaming content by performing Vtuber distribution. Part or all of the above-mentioned processing in the unit for performing Vtuber distribution may be performed, for example, using the generation AI, or may be performed without using the generation AI. For example, the unit for performing Vtuber distribution can cause the generation AI to control the Vtuber character.

[0079] The upload unit can estimate the user's emotion and adjust the upload timing of the content based on the estimated user's emotion. For example, when the user is stressed, the upload unit provides a simple mode for quickly uploading. In addition, when the user is relaxed, the upload unit can also provide detailed upload options and suggest a customizable upload method. Furthermore, when the user is in a hurry, the upload unit can prioritize voice input and quickly upload content. This allows the content to be uploaded at a more appropriate time by adjusting the upload timing according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the upload unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the upload unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0080] The upload unit can analyze the user's past upload history and select the optimal upload method. For example, the upload unit can automatically display the types of content that the user frequently uploaded in the past as candidates. The upload unit can also preferentially suggest the upload method (audio, text, etc.) that the user used in the past. Furthermore, the upload unit can predict and suggest the upload method to be used in a specific time period from the user's past upload history. In this way, the optimal upload method can be provided by analyzing the past upload history. A part or all of the above-mentioned processing in the upload unit may be performed, for example, using AI, or may be performed without using AI. For example, the upload unit can input the user's past upload history data into AI and cause the AI ​​to select the optimal upload method.

[0081] The upload unit can apply different upload protocols depending on the type of material at the time of uploading. For example, in the case of text data, the upload unit automatically applies a character limit and format. In addition, in the case of audio data, the upload unit can also apply an optimal upload protocol depending on the sound quality and file format. Furthermore, in the case of video data, the upload unit can also apply an optimal upload protocol depending on the resolution and file size. This allows efficient uploading by applying an optimal upload protocol depending on the type of material. Part or all of the above-mentioned processing in the upload unit may be performed using, for example, AI, or may be performed without using AI. For example, the upload unit can input material type data to AI and cause AI to apply the optimal upload protocol.

[0082] The upload unit can estimate the user's emotion and determine the priority of the material to be uploaded based on the estimated user's emotion. For example, when the user is excited, the upload unit preferentially uploads important material. Also, when the user is relaxed, the upload unit can upload the material in order. Furthermore, when the user is in a hurry, the upload unit can prioritize the material that can be uploaded most easily. In this way, by determining the priority of the material according to the user's emotion, important material can be preferentially uploaded. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the upload unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the upload unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0083] The uploading unit can prioritize uploading highly relevant material in consideration of the geographical location information of the user at the time of uploading. For example, when the user is in a specific area, the uploading unit prioritizes uploading material related to the area. In addition, when the user is traveling, the uploading unit can also prioritize uploading material related to the travel destination. Furthermore, when the user is at an event venue, the uploading unit can also prioritize uploading material related to the event. In this way, by considering the geographical location information of the user, highly relevant material can be uploaded preferentially. Part or all of the above-mentioned processing in the uploading unit may be performed, for example, using AI, or may be performed without using AI. For example, the uploading unit can input the geographical location information data of the user to AI and cause the AI ​​to select highly relevant material.

[0084] The uploading unit can analyze the user's social media activity and upload related material at the time of uploading. For example, the uploading unit preferentially uploads material related to topics that the user is talking about on social media. The uploading unit can also upload material related to the posted content of an account that the user follows. Furthermore, the uploading unit can also upload material related to topics of groups and communities in which the user participates. This makes it possible to efficiently upload related material by analyzing the user's social media activity. A part or all of the above-mentioned processing in the uploading unit may be performed using, for example, AI, or may be performed without using AI. For example, the uploading unit can input the user's social media activity data into AI and cause the AI ​​to select related material.

[0085] The selection unit can estimate the user's emotion and adjust the display method of the use selection based on the estimated user's emotion. For example, when the user is nervous, the selection unit provides a simple and highly visible display method. Furthermore, when the user is relaxed, the selection unit can also provide a display method including detailed information. Furthermore, when the user is in a hurry, the selection unit can also provide a display method that focuses on the main points. This makes it possible to provide a more appropriate display method by adjusting the display method of the use selection according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the selection unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the selection unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0086] The selection unit can adjust the priority of the uses based on the importance of the material at the time of selection. For example, the selection unit preferentially displays important material to make it easier to select. The selection unit can also postpone less important material and display it lower among the options. Furthermore, the selection unit can dynamically change the display order of the options according to the importance of the material. This allows important material to be preferentially selected by adjusting the priority of the uses based on the importance of the material. A part or all of the above-mentioned processing in the selection unit may be performed, for example, using AI or may be performed without using AI. For example, the selection unit can input the importance data of the material to AI and cause AI to adjust the priority of the uses.

[0087] The selection unit can apply different selection algorithms depending on the category of the material when making a selection. For example, in the case of a text material, the selection unit applies a keyword matching algorithm. In addition, in the case of an audio material, the selection unit can also apply a voice recognition algorithm. Furthermore, in the case of a video material, the selection unit can also apply a video analysis algorithm. This allows for efficient selection by applying a selection algorithm according to the material category. Some or all of the above-mentioned processing in the selection unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection unit can input material category data to AI and cause the AI ​​to apply the selection algorithm.

[0088] The selection unit can estimate the user's emotion and adjust the length of the usage selection based on the estimated user's emotion. For example, when the user is in a hurry, the selection unit provides short and to the point options. In addition, when the user is relaxed, the selection unit can also provide longer options including detailed explanations. In addition, when the user is excited, the selection unit can also provide visually stimulating options. This allows more appropriate options to be provided by adjusting the length of the usage selection according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the selection unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the selection unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0089] At the time of selection, the selection unit can determine the priority of the uses based on the submission time of the material. For example, the selection unit preferentially displays the most recently submitted material. The selection unit can also postpone the use of material that has been submitted earlier. Furthermore, the selection unit can dynamically change the display order of the options according to the submission time. In this way, the priority of the uses can be determined based on the submission time of the material, so that the latest material can be preferentially selected. A part or all of the above-mentioned processing in the selection unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection unit can input the submission time data of the material to AI and have the AI ​​execute the determination of the priority of the uses.

[0090] The selection unit can adjust the order of uses based on the relevance of the material at the time of selection. The selection unit, for example, preferentially displays highly relevant material. The selection unit can also postpone less relevant material. Furthermore, the selection unit can dynamically change the display order of options according to the relevance of the material. In this way, by adjusting the order of uses based on the relevance of the material, it is possible to preferentially select highly relevant material. A part or all of the above-mentioned processing in the selection unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection unit can input the relevance data of the material to AI and cause the AI ​​to adjust the order of uses.

[0091] The generation unit can estimate the user's emotion and adjust the expression method of the generated content based on the estimated user's emotion. For example, when the user is relaxed, the generation unit generates content that progresses at a leisurely pace. Also, when the user is in a hurry, the generation unit can generate content that emphasizes the shortest route. Furthermore, when the user is excited, the generation unit can generate content that adds a visually stimulating effect. In this way, more appropriate content can be generated by adjusting the expression method of the content according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the generation unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the generation unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0092] The generating unit can adjust the level of detail of the generation based on the importance of the material at the time of generation. For example, the generating unit generates important material in detail to increase the amount of information. The generating unit can also generate material with low importance by simplifying it. Furthermore, the generating unit can dynamically change the level of detail of the generation according to the importance of the material. As a result, important material can be generated in detail by adjusting the level of detail of the generation based on the importance of the material. A part or all of the above-mentioned processing in the generating unit may be performed, for example, using AI, or may be performed without using AI. For example, the generating unit can input the importance data of the material to AI and cause AI to adjust the level of detail of the generation.

[0093] The generation unit can apply different generation algorithms depending on the category of the material at the time of generation. For example, in the case of a text material, the generation unit applies a natural language generation algorithm. In addition, in the case of an audio material, the generation unit can also apply a voice synthesis algorithm. Furthermore, in the case of a video material, the generation unit can also apply a video generation algorithm. This enables efficient generation by applying a generation algorithm according to the material category. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the generation unit can input material category data to AI and cause the AI ​​to apply the generation algorithm.

[0094] The generation unit can estimate the user's emotion and adjust the length of the content to be generated based on the estimated user's emotion. For example, when the user is in a hurry, the generation unit generates short, to-the-point content. In addition, when the user is relaxed, the generation unit can also generate longer content including detailed explanations. Furthermore, when the user is excited, the generation unit can also generate content with visually stimulating effects. This allows more appropriate content to be generated by adjusting the length of the content according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the generation unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the generation unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0095] The generation unit can determine the generation priority based on the submission time of the material at the time of generation. For example, the generation unit generates the most recently submitted material with priority. The generation unit can also postpone the material that was submitted earlier. Furthermore, the generation unit can dynamically change the generation priority according to the submission time. In this way, the generation priority can be determined based on the submission time of the material, so that the latest material can be generated with priority. A part or all of the above-mentioned processing in the generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the generation unit can input the submission time data of the material to AI and have the AI ​​execute the determination of the generation priority.

[0096] The generation unit can adjust the order of generation based on the relevance of the material at the time of generation. For example, the generation unit generates material with high relevance with priority. The generation unit can also postpone material with low relevance. Furthermore, the generation unit can dynamically change the order of generation according to the relevance of the material. In this way, by adjusting the order of generation based on the relevance of the material, material with high relevance can be generated with priority. A part or all of the above-mentioned processing in the generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the generation unit can input the relevance data of the material to AI and cause AI to adjust the order of generation.

[0097] The unit that transcribes the voice data can estimate the user's emotion and adjust the expression method of the transcription based on the estimated emotion of the user. For example, when the user is relaxed, the unit that transcribes the voice data generates a transcription that proceeds at a leisurely pace. In addition, when the user is in a hurry, the unit that transcribes the voice data can also generate a transcription that emphasizes the shortest route. Furthermore, when the user is excited, the unit that transcribes the voice data can also generate a transcription that adds a visually stimulating effect. This allows a more appropriate transcription to be generated by adjusting the expression method of the transcription according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the unit that transcribes the voice data may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, a unit that transcribes voice data can input a user's emotional data into a generation AI and have the generation AI estimate the emotions.

[0098] The unit that transcribes the voice data can adjust the level of detail of the transcription based on the importance of the voice during transcription. For example, the unit that transcribes the voice data transcribes important voice in detail to increase the amount of information. The unit that transcribes the voice data can also transcribe voice with low importance by simplifying it. Furthermore, the unit that transcribes the voice data can dynamically change the level of detail of the transcription according to the importance of the voice. As a result, important voice can be transcribed in detail by adjusting the level of detail of the transcription based on the importance of the voice. A part or all of the above-mentioned processing in the unit that transcribes the voice data may be performed using, for example, AI, or may be performed without using AI. For example, the unit that transcribes the voice data can input the importance data of the voice into AI and cause AI to adjust the level of detail of the transcription.

[0099] The unit that transcribes the voice data can estimate the user's emotion and adjust the length of the transcription based on the estimated user's emotion. For example, when the user is in a hurry, the unit that transcribes the voice data can generate a short transcription that is to the point. Also, when the user is relaxed, the unit that transcribes the voice data can generate a longer transcription that includes detailed explanations. Furthermore, when the user is excited, the unit that transcribes the voice data can generate a transcription that includes visually stimulating effects. This allows a more appropriate transcription to be generated by adjusting the length of the transcription according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the unit that transcribes the voice data may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, a unit that transcribes voice data can input a user's emotional data into a generation AI and have the generation AI estimate the emotions.

[0100] The unit that transcribes the voice data can determine the priority of transcription based on the submission time of the voice when transcribing. For example, the unit that transcribes the voice data prioritizes transcription of the most recently submitted voice. In addition, the unit that transcribes the voice data can postpone the voice that was submitted earlier. Furthermore, the unit that transcribes the voice data can dynamically change the priority of transcription according to the submission time. In this way, by determining the priority of transcription based on the submission time of the voice, the latest voice can be preferentially transcribed. A part or all of the above-mentioned processing in the unit that transcribes the voice data may be performed, for example, using AI or may be performed without using AI. For example, the unit that transcribes the voice data can input the submission time data of the voice into AI and cause AI to execute the determination of the priority of transcription.

[0101] The automatic translation unit can estimate the user's emotion and adjust the translation expression method based on the estimated user's emotion. For example, when the user is relaxed, the automatic translation unit generates a translation that proceeds at a leisurely pace. In addition, when the user is in a hurry, the automatic translation unit can also generate a translation that emphasizes the shortest route. Furthermore, when the user is excited, the automatic translation unit can also generate a translation that adds a visually stimulating effect. This allows a more appropriate translation to be generated by adjusting the translation expression method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the automatic translation unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the automatic translation unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0102] The automatic translation unit can adjust the level of detail of the translation based on the importance of the material during translation. For example, the automatic translation unit translates important material in detail to increase the amount of information. The automatic translation unit can also simplify and translate material with low importance. Furthermore, the automatic translation unit can dynamically change the level of detail of the translation according to the importance of the material. As a result, important material can be translated in detail by adjusting the level of detail of the translation based on the importance of the material. Some or all of the above-mentioned processing in the automatic translation unit may be performed, for example, using AI, or may be performed without using AI. For example, the automatic translation unit can input the importance data of the material to AI and have the AI ​​adjust the level of detail of the translation.

[0103] The automatic translation unit can estimate the user's emotion and adjust the length of the translation based on the estimated user's emotion. For example, when the user is in a hurry, the automatic translation unit generates a short translation that is to the point. In addition, when the user is relaxed, the automatic translation unit can also generate a longer translation that includes detailed explanations. Furthermore, when the user is excited, the automatic translation unit can also generate a translation that adds visually stimulating effects. This allows a more appropriate translation to be generated by adjusting the length of the translation according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the automatic translation unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the automatic translation unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0104] The automatic translation unit can determine the priority of translation based on the submission time of the material during translation. For example, the automatic translation unit gives priority to translating the most recently submitted material. The automatic translation unit can also postpone material that has been submitted a long time ago. Furthermore, the automatic translation unit can dynamically change the priority of translation according to the submission time. In this way, by determining the priority of translation based on the submission time of the material, the latest material can be translated with priority. Some or all of the above-mentioned processing in the automatic translation unit may be performed, for example, using AI, or may be performed without using AI. For example, the automatic translation unit can input the submission time data of the material to AI and have the AI ​​execute the determination of the translation priority.

[0105] The illustration creation unit can estimate the user's emotion and adjust the stamp design based on the estimated user's emotion. For example, when the user is relaxed, the illustration creation unit generates a stamp with a relaxed design. Also, when the user is in a hurry, the illustration creation unit can generate a stamp with a simple and highly visible design. Furthermore, when the user is excited, the illustration creation unit can generate a stamp with a visually stimulating design. This allows a more appropriate stamp to be generated by adjusting the stamp design according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the illustration creation unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the illustration creation unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0106] The illustration creation unit can adjust the level of detail of the stamp based on the importance of the illustration when creating the stamp. For example, the illustration creation unit stamps an important illustration in detail to increase the amount of information. The illustration creation unit can also simplify an illustration with low importance and create a stamp. Furthermore, the illustration creation unit can dynamically change the level of detail of the stamp according to the importance of the illustration. As a result, important illustrations can be stamped in detail by adjusting the level of detail of the stamp based on the importance of the illustration. A part or all of the above-mentioned processing in the illustration creation unit may be performed, for example, using AI or may be performed without using AI. For example, the illustration creation unit can input the importance data of the illustration to AI and cause AI to adjust the level of detail of the stamp.

[0107] The illustration creation unit can estimate the user's emotion and adjust the size of the stamp based on the estimated user's emotion. For example, when the user is relaxed, the illustration creation unit generates a larger stamp. Also, when the user is in a hurry, the illustration creation unit can generate a smaller stamp. Furthermore, when the user is excited, the illustration creation unit can generate a stamp of a visually stimulating size. This allows a more appropriate stamp to be generated by adjusting the size of the stamp according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the illustration creation unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the illustration creation unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0108] When creating a stamp, the illustration creation unit can determine the priority of the stamp based on the submission time of the illustration. For example, the illustration creation unit can prioritize the illustrations submitted recently as stamps. The illustration creation unit can also postpone illustrations submitted earlier. Furthermore, the illustration creation unit can dynamically change the priority of stamping according to the submission time. In this way, by determining the priority of stamps based on the submission time of the illustration, the latest illustrations can be prioritized as stamps. A part or all of the above-mentioned processing in the illustration creation unit may be performed using, for example, AI, or may be performed without using AI. For example, the illustration creation unit can input the submission time data of the illustrations to AI and have the AI ​​execute the determination of the priority of stamps.

[0109] The novel creation unit can estimate the user's emotions and adjust the way the novel is expressed based on the estimated user's emotions. For example, when the user is relaxed, the novel creation unit generates a novel that progresses at a leisurely pace. Also, when the user is in a hurry, the novel creation unit can generate a novel that emphasizes the shortest route. Furthermore, when the user is excited, the novel creation unit can generate a novel that adds a visually stimulating effect. This allows a more appropriate novel to be generated by adjusting the way the novel is expressed according to the user's emotions. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the novel creation unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the novel creation unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0110] The novel creation unit can adjust the level of detail of the novel based on the importance of the illustrations when creating the novel. For example, the novel creation unit can novelize important illustrations in detail to increase the amount of information. The novel creation unit can also simplify and novelize illustrations with low importance. Furthermore, the novel creation unit can dynamically change the level of detail of the novel according to the importance of the illustrations. As a result, important illustrations can be novelized in detail by adjusting the level of detail of the novel based on the importance of the illustrations. A part or all of the above-mentioned processing in the novel creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the novel creation unit can input importance data of the illustrations to AI and cause the AI ​​to adjust the level of detail of the novel.

[0111] The novel creation unit can estimate the user's emotions and adjust the length of the novel based on the estimated user's emotions. For example, when the user is in a hurry, the novel creation unit generates a short novel that is to the point. In addition, when the user is relaxed, the novel creation unit can also generate a longer novel that includes detailed explanations. Furthermore, when the user is excited, the novel creation unit can also generate a novel with visually stimulating effects. This allows a more appropriate novel to be generated by adjusting the length of the novel according to the user's emotions. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the novel creation unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the novel creation unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0112] The novel creation unit can determine the priority of novels based on the submission time of illustrations when creating a novel. For example, the novel creation unit prioritizes novelization of recently submitted illustrations. The novel creation unit can also postpone illustrations submitted earlier. Furthermore, the novel creation unit can dynamically change the priority of novelization according to the submission time. In this way, by determining the priority of novels based on the submission time of illustrations, the latest illustrations can be prioritized for novelization. A part or all of the above-mentioned processing in the novel creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the novel creation unit can input illustration submission time data into AI and have the AI ​​execute the determination of the priority of novels.

[0113] The Vtuber delivery unit can estimate the user's emotions and adjust the delivery expression method based on the estimated user's emotions. For example, when the user is relaxed, the Vtuber delivery unit can deliver at a leisurely pace. In addition, when the user is in a hurry, the Vtuber delivery unit can also deliver with an emphasis on the shortest route. Furthermore, when the user is excited, the Vtuber delivery unit can also deliver with a visually stimulating effect added. This allows for more appropriate delivery by adjusting the delivery expression method according to the user's emotions. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the Vtuber delivery unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the Vtuber delivery unit can input the user's emotion data into the generation AI and cause the generation AI to execute emotion estimation.

[0114] The unit performing Vtuber distribution can adjust the level of detail of the distribution based on the importance of the material during distribution. For example, the unit performing Vtuber distribution can distribute important material in detail to increase the amount of information. In addition, the unit performing Vtuber distribution can also simplify and distribute material with low importance. Furthermore, the unit performing Vtuber distribution can dynamically change the level of detail of the distribution according to the importance of the material. As a result, important material can be distributed in detail by adjusting the level of detail of the distribution based on the importance of the material. Part or all of the above-mentioned processing in the unit performing Vtuber distribution may be performed using, for example, AI, or may be performed without using AI. For example, the unit performing Vtuber distribution can input the importance data of the material into AI and have AI adjust the level of detail of the distribution.

[0115] The Vtuber delivery unit can estimate the user's emotions and adjust the length of the delivery based on the estimated user's emotions. For example, when the user is in a hurry, the Vtuber delivery unit can deliver a short, to-the-point delivery. In addition, when the user is relaxed, the Vtuber delivery unit can also deliver a longer delivery including detailed explanations. Furthermore, when the user is excited, the Vtuber delivery unit can also deliver a delivery with visually stimulating effects. This allows for more appropriate delivery by adjusting the length of the delivery according to the user's emotions. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the Vtuber delivery unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the Vtuber delivery unit can input the user's emotion data into the generation AI and cause the generation AI to execute emotion estimation.

[0116] The Vtuber distribution unit can determine the priority of distribution based on the submission time of the material at the time of distribution. For example, the Vtuber distribution unit prioritizes the distribution of recently submitted material. In addition, the Vtuber distribution unit can postpone material submitted earlier. Furthermore, the Vtuber distribution unit can dynamically change the priority of distribution according to the submission time. In this way, by determining the priority of distribution based on the submission time of the material, the latest material can be preferentially distributed. Part or all of the above-mentioned processing in the Vtuber distribution unit may be performed using, for example, AI, or may be performed without using AI. For example, the Vtuber distribution unit can input the submission time data of the material into AI and have AI execute the determination of the priority of distribution.

[0117] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0118] The content generation system may further include a feedback unit. The feedback unit may collect user feedback on the generated content and reflect it in the next content generation. For example, when a user provides a comment or rating on a generated blog post, the generation AI may learn the feedback and provide more accurate content the next time it is generated. The feedback unit may also collect user feedback in real time and immediately reflect it in the generation AI. Furthermore, the feedback unit may analyze the user feedback and provide data for improving the algorithm of the generation AI. This allows the content generation system to provide content that is more suited to the needs of the user.

[0119] The unit that transcribes the voice data can further analyze the emotion of the voice and adjust the tone of the transcription based on the emotion. For example, if the voice data expresses joy, the transcription can be made brighter in tone. If the voice data expresses sadness, the transcription can be made calmer in tone. If the voice data expresses anger, the transcription can be made more emphatic in tone. This allows for the generation of a transcription that reflects the emotion of the voice data.

[0120] The automatic translation unit can use a specialized dictionary to further improve the accuracy of the translation. For example, when translating a text in the medical field, a medical dictionary can be referenced to provide an accurate translation. When translating a text in the legal field, a legal dictionary can be used to provide a specialized translation. When translating a text in the technical field, a technical dictionary can be used to provide a highly accurate translation. This makes it possible to provide accurate translations specialized for specialized fields.

[0121] The illustration creation unit can further customize the design of the stamp based on the user's preferences. For example, if the user likes a particular color, a stamp based on that color can be generated. Also, if the user likes a particular character style, a stamp that matches that style can be created. Furthermore, if the user likes a particular theme, a stamp that matches that theme can be provided. In this way, stamps can be generated according to the user's preferences.

[0122] The novel creation unit can further estimate the user's emotions and select a genre of the novel based on the estimated emotions. For example, if the user is relaxed, a fantasy or romance genre can be selected. If the user is excited, an action or thriller genre can be selected. If the user is sad, a drama or a moving story can be selected. In this way, a novel of a genre according to the user's emotions can be generated.

[0123] Vtuber streamers can also analyze viewer reactions in real time and dynamically adjust the content of their stream. For example, if viewers are sending many likes and comments, they can make the stream more exciting. Also, if viewers are bored, they can change the content of the stream to pique their interest. Furthermore, if viewers are interested in a particular topic, they can add content related to that topic. This allows for flexible streaming based on viewer reactions.

[0124] The uploading unit can further estimate the user's emotions and automatically select the format of the material to be uploaded based on the estimated emotions. For example, if the user is stressed, a simple text format can be selected. If the user is relaxed, a detailed video format can be selected. If the user is in a hurry, an audio format can be selected. This allows the material to be uploaded in the optimal format according to the user's emotions.

[0125] The upload unit can further analyze the user's past upload history and suggest the optimal upload time. For example, if the user has uploaded a lot of content in a particular time period in the past, that time period can be suggested. Also, if the user has uploaded a lot of content on a particular day of the week, that day of the week can be suggested. Furthermore, if the user has uploaded content related to a particular event or season, that timing can be suggested. In this way, the optimal upload time can be provided based on the user's past upload history.

[0126] The upload unit can further take into account the user's network conditions when applying different upload protocols depending on the type of content. For example, if the user is in a slow network environment, a lightweight protocol can be applied to quickly upload. If the user is in a high-speed network environment, a high-quality protocol can be applied to upload detailed data. Furthermore, if the user is in an unstable network environment, a protocol that supports intermittent uploads can be applied. This makes it possible to provide the optimal upload protocol according to the user's network conditions.

[0127] The uploading unit can further estimate the user's emotions and automatically edit the content of the material to be uploaded based on the estimated emotions. For example, if the user is excited, editing can be performed to emphasize important points. If the user is relaxed, editing can be performed to include detailed explanations. Furthermore, if the user is in a hurry, editing can be performed to include a concise summary. In this way, material can be uploaded with optimal content according to the user's emotions.

[0128] The process flow of the second embodiment will be briefly described below.

[0129] Step 1: The uploading unit uploads the material (text, audio, video, etc.) that the user has collected onto the messenger app. For example, audio data of an interview or video data that has been filmed can be uploaded. Step 2: The selection unit selects a use based on the content uploaded by the upload unit. For example, the selection unit can select a use such as creating a blog transcript from the audio data or streaming as a VTuber. Step 3: The generation unit generates content according to the purpose selected by the selection unit. For example, the generation AI transcribes blogs from audio data, and then performs automatic translation and dubbing to enable global expansion. The generation AI can also create stamps and novels from illustrations. For example, the generation AI receives audio data as input and generates a blog transcript. The generation AI then automatically translates the generated transcript to make it compatible with multiple languages. The generation AI then generates dubbed audio based on the translated text to enable global expansion.

[0130] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0131] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0132] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0133] Each of the multiple elements including the above-mentioned upload unit, selection unit, and generation unit is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the upload unit is realized by the control unit 46A of the smart device 14, and uploads the material that the user has collected on a messenger app. The selection unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and selects a use based on the uploaded material. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates content according to the selected use. The generation unit may be realized, for example, by the control unit 46A of the smart device 14. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0134] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0135] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0136] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0137] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0138] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0139] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0140] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0141] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0142] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0143] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0144] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0145] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0146] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0147] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0148] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0149] Each of the multiple elements including the above-mentioned upload unit, selection unit, and generation unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the upload unit is realized by the control unit 46A of the smart glasses 214, and uploads the material that the user has collected on a messenger app. The selection unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and selects a use based on the uploaded material. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates content according to the selected use. The generation unit may be realized, for example, by the control unit 46A of the smart glasses 214. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0150] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0151] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0152] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0153] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0154] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0155] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0156] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0157] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0158] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0159] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0160] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0161] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0162] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0163] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0164] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0165] Each of the multiple elements including the above-mentioned upload unit, selection unit, and generation unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the upload unit is realized by the control unit 46A of the headset type terminal 314, and uploads the material that the user has collected on a messenger app. The selection unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and selects a use based on the uploaded material. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates content according to the selected use. The generation unit may be realized, for example, by the control unit 46A of the headset type terminal 314. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0166] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0167] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0168] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0169] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0170] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0171] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0172] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0173] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0174] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0175] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0176] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0177] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0178] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0179] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0180] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0181] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0182] Each of the multiple elements including the above-mentioned upload unit, selection unit, and generation unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the upload unit is realized by the control unit 46A of the robot 414, and uploads the material that the user has collected on a messenger app. The selection unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and selects a use based on the uploaded material. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates content according to the selected use. The generation unit may be realized, for example, by the control unit 46A of the robot 414. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0183] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0184] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0185] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0186] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0187] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0188] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0189] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0190] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0191] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0192] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0193] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0194] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0195] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0196] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0197] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0198] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0199] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0200] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0201] (Appendix 1) An upload section for uploading content; A selection unit that selects a use based on the content uploaded by the upload unit; a generation unit that generates content according to the purpose selected by the selection unit. A system characterized by: (Appendix 2) Equipped with a section for transcribing voice data 2. The system of claim 1, (Appendix 3) Equipped with an automatic translation section for automatic translation 2. The system of claim 1, (Appendix 4) Equipped with an illustration creation section that creates stamps from illustrations 2. The system of claim 1, (Appendix 5) Includes a novel creation section that creates novels from illustrations 2. The system of claim 1, (Appendix 6) Equipped with a department for Vtuber distribution 2. The system of claim 1, (Appendix 7) The upload unit is Includes a specific method for estimating user emotions and adjusting the timing of uploading content based on the estimated user emotions. 2. The system of claim 1, (Appendix 8) The upload unit is Analyze the user's past upload history and select the appropriate upload method 2. The system of claim 1, (Appendix 9) The upload unit is When uploading, apply the appropriate upload protocol depending on the type of content. 2. The system of claim 1, (Appendix 10) The upload unit is It includes a specific method for estimating user sentiment and prioritizing content to be uploaded based on the estimated user sentiment. 2. The system of claim 1, (Appendix 11) The upload unit is When uploading, it takes into account the user's geographic location information to prioritize uploading content that is more relevant to them. 2. The system of claim 1, (Appendix 12) The upload unit is When uploading, analyze the user's social media activity and upload relevant content. 2. The system of claim 1, (Appendix 13) The selection unit is A specific method for estimating a user's emotion and adjusting the display method of the usage selection based on the estimated user's emotion is included. 2. The system of claim 1, (Appendix 14) The selection unit is When selecting, adjust the priority of the usage based on the importance of the topic 2. The system of claim 1, (Appendix 15) The selection unit is When selecting, apply different selection algorithms depending on the category of the story 2. The system of claim 1, (Appendix 16) The selection unit is A specific method for estimating a user's emotion and adjusting the length of the usage selection based on the estimated user's emotion is included. 2. The system of claim 1, (Appendix 17) The selection unit is During selection, prioritize applications based on when the ideas were submitted 2. The system of claim 1, (Appendix 18) The selection unit is When selecting, adjust the order of uses based on the relevance of the topic 2. The system of claim 1, (Appendix 19) The generation unit is A specific method for estimating a user's emotion and adjusting the expression method of the generated content based on the estimated user's emotion is included. 2. The system of claim 1, (Appendix 20) The generation unit is At generation time, adjust the level of detail generated based on the importance of the material 2. The system of claim 1, (Appendix 21) The generation unit is During generation, different generation algorithms are applied depending on the category of the material. 2. The system of claim 1, (Appendix 22) The generation unit is A specific method for estimating a user's emotion and adjusting the length of the generated content based on the estimated user's emotion is included. 2. The system of claim 1, (Appendix 23) The generation unit is During generation, prioritize generation based on when the content was submitted 2. The system of claim 1, (Appendix 24) The generation unit is At generation time, adjust the order of generation based on the relevance of the content 2. The system of claim 1, (Appendix 25) The unit for transcribing the voice data includes: Includes a specific method for estimating a user's emotions and adjusting the expression of the transcription based on the estimated user's emotions. 3. The system of claim 2, (Appendix 26) The unit for transcribing the voice data includes: When transcribing, adjust transcription detail based on the importance of the audio 3. The system of claim 2, (Appendix 27) The unit for transcribing the voice data includes: Includes specific methods for estimating user sentiment and adjusting the length of the transcription based on the estimated user sentiment. 3. The system of claim 2, (Appendix 28) The unit for transcribing the voice data includes: When transcribing, prioritize transcriptions based on when the audio was submitted 3. The system of claim 2, (Appendix 29) The automatic translation unit for automatic translation includes: A specific method for estimating a user's emotion and adjusting the translation expression based on the estimated user's emotion is included. 4. The system of claim 3. (Appendix 30) The automatic translation unit for automatic translation includes: During translation, adjust the level of detail in the translation based on the importance of the content 4. The system of claim 3. (Appendix 31) The automatic translation unit for automatic translation includes: Specific methods for estimating user sentiment and adjusting translation length based on the estimated user sentiment are included. 4. The system of claim 3. (Appendix 32) The automatic translation unit for automatic translation includes: During translation, prioritize translations based on when content was submitted 4. The system of claim 3. (Appendix 33) An illustration creation unit that creates a stamp from the illustration, A specific method for estimating a user's emotion and adjusting the stamp design based on the estimated user's emotion is included. 5. The system of claim 4. (Appendix 34) An illustration creation unit that creates a stamp from the illustration, When creating a stamp, adjust the level of detail based on the importance of the illustration. 5. The system of claim 4. (Appendix 35) An illustration creation unit that creates a stamp from the illustration, A specific method for estimating a user's emotion and adjusting the size of the stamp based on the estimated user's emotion is included. 5. The system of claim 4. (Appendix 36) An illustration creation unit that creates a stamp from the illustration, When creating stamps, prioritize stamps based on when the illustrations were submitted 5. The system of claim 4. (Appendix 37) A novel creation section that creates a novel from the illustrations, A specific method for estimating a user's emotions and adjusting the way a novel is presented based on the estimated user's emotions is included. 6. The system of claim 5, (Appendix 38) A novel creation section that creates a novel from the illustrations, When creating a novel, adjust the level of detail in the novel based on the importance of the illustrations. 6. The system of claim 5, (Appendix 39) A novel creation section that creates a novel from the illustrations, A specific method for estimating a user's emotion and adjusting the length of a novel based on the estimated user's emotion is included. 6. The system of claim 5, (Appendix 40) A novel creation section that creates a novel from the illustrations, When creating a novel, prioritize the novel based on when illustrations are submitted. 6. The system of claim 5, (Appendix 41) The Vtuber distribution department is A specific method for estimating a user's emotion and adjusting the delivery expression method based on the estimated user's emotion is included. 7. The system of claim 6, (Appendix 42) The Vtuber distribution department is When streaming, adjust the level of detail in your stream based on the importance of the content 7. The system of claim 6, (Appendix 43) The Vtuber distribution department is A specific method for estimating a user's emotion and adjusting the length of delivery based on the estimated user's emotion is included. 7. The system of claim 6, (Appendix 44) The Vtuber distribution department is When streaming, prioritize streaming based on when content was submitted 7. The system of claim 6, [Explanation of symbols]

[0202] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. An upload section for uploading content; A selection unit that selects a use based on the content uploaded by the upload unit; a generation unit that generates content according to the purpose selected by the selection unit. A system characterized by:

2. Equipped with a section for transcribing voice data 2. The system of claim 1 .

3. Equipped with an automatic translation section for automatic translation 2. The system of claim 1 .

4. Equipped with an illustration creation section that creates stamps from illustrations 2. The system of claim 1 .

5. Includes a novel creation section that creates novels from illustrations 2. The system of claim 1 .

6. Equipped with a department for Vtuber distribution 2. The system of claim 1 .

7. The upload unit is Includes a specific method for estimating user emotions and adjusting the timing of uploading content based on the estimated user emotions.

2. The system of claim 1 .

8. The upload unit is Analyze the user's past upload history and select the appropriate upload method 2. The system of claim 1 .

9. The upload unit is When uploading, apply the appropriate upload protocol depending on the type of content.

2. The system of claim 1 .

10. The upload unit is It includes a specific method for estimating user sentiment and prioritizing content to be uploaded based on the estimated user sentiment.

2. The system of claim 1 .

Images