System

Abstract

A system is provided.SOLUTION: This system is provided with a means for receiving a plurality of contents transmitted by a user, a means for analyzing the received contents, and for extracting a keyword or a tag, a means for automatically generating an article in a news format based on the analyzed data, a means for selecting the media format of the generated article, and a means for generating the article based on the selected media format, and for distributing it to a plurality of receivers.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Therefore, the patent specification should state the "problem that the invention aims to solve" and the "means for solving the problem."

[0005] Traditionally, social networking sites and blogs have been the main methods for sharing personal experiences and thoughts with others, but these methods tend to leave information buried, limiting their ability to stimulate sharing and communication within specific communities. Furthermore, there were limited ways to compile information in specific formats, such as news or magazine style, making it difficult for users to share information in the way they intended. Given this background, there was a demand for a method to automatically compile users' experiences and thoughts into articles and easily share them with others in a specific format. [Means for solving the problem]

[0006] The present invention provides a means for receiving multiple pieces of content sent by a user, analyzing the received content to extract keywords and tags, and automatically generating news-style articles based on the analyzed data. It also provides a system that allows a user to select the media format of the generated article and generates an article based on the selected media format and distributes it to multiple recipients, thereby enabling the user to easily share their experiences and thoughts with others in a specific format. Furthermore, when generating articles, the system includes a means for performing image recognition and text analysis on the received content, automatically generating captions, and composing the entire article, thereby improving the quality of information and enabling information sharing in line with the user's intentions.

[0007] Ok, now I will follow your instructions and create definition sentences for important words.

[0008] A "User" is an individual or organization that uses the System to transmit Content.

[0009] "Content" refers to digital data such as photos, videos, and text.

[0010] The "server" is the central part of the system that analyzes content received from users, stores it in a database, and generates and distributes articles.

[0011] "Analysis" is the process of recognizing the content of received content and extracting important information.

[0012] "Keywords" are highly important words extracted as a result of content analysis.

[0013] A "tag" is identification information that is assigned to classify and organize content.

[0014] "Generative AI" is artificial intelligence that analyzes received data and automatically generates news-style articles.

[0015] "News format" refers to a format in which the generated article has the appearance of a typical news article.

[0016] "Media format" refers to the format in which articles are provided that can be selected by the user, such as news format, magazine style, or album style.

[0017] "Distribution" is the process of sending the generated article to recipients via messaging services such as LINE.

[0018] A "caption" is a short description added to a photo or video. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] Understood. Now, let's write the "Form for carrying out the invention."

[0041] The present invention provides a system that allows users to share their experiences and thoughts with others. This system receives and analyzes content sent by users, and generates and distributes articles based on that content, thereby stimulating communication.

[0042] System Overview

[0043] The system consists of the following main components:

[0044] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[0045] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[0046] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data.

[0047] User operation procedure

[0048] Users can use a dedicated LINE account to send content, such as photos, videos, and comments from a sports day, using the LINE chat function.

[0049] Server Processing

[0050] The server receives content sent from LINE accounts. After receiving the data, the server analyzes it and extracts important keywords and tags. It uses image recognition technology to understand the content of photos and videos, and natural language processing tools to analyze text data. It then invokes a generation AI based on the analysis results to automatically generate news-style articles.

[0051] Processing of generated AI

[0052] The generative AI automatically generates articles based on the received data and analysis results. First, it generates appropriate captions from images and videos, and then it generates the sentences that make up the overall story. In this process, it takes into account keywords and tags to create articles that reflect the user's intentions.

[0053] Selecting a Media Format

[0054] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[0055] Article generation and distribution

[0056] The server generates the final article based on the media format selected by the user. The generated article is sent to the user as a final preview. After the user confirms it, they click the distribution button, and the server distributes the article via LINE based on the specified recipient list.

[0057] Specific examples

[0058] Example 1: Sharing your child's growth

[0059] 1. The user (parent) opens a dedicated LINE account and sends photos, videos, and impressions of the sports day.

[0060] 2. The server receives this data, checks the format and size of the images and videos, and analyzes the text.

[0061] 3. The AI ​​generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[0062] 4. The user selects the magazine-style media format, and the server sends the magazine-style article as the final preview.

[0063] 5. After user confirms, distribute the article to relatives and friends.

[0064] Example 2: Amateur sports promotion

[0065] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[0066] 2. The server receives and analyzes this data.

[0067] 3. Based on the content of the video, the generative AI generates a caption such as "The next generation star player's amazing play," and assembles the whole video into an article.

[0068] 4. The user selects a news-like media format, and the server sends the news-like article as the final preview.

[0069] 5. After the user has confirmed it, the article is distributed to fans and supporters.

[0070] Through the above-described embodiments, the present invention provides specific means for effectively sharing users' experiences and thoughts and for activating communication.

[0071] The processing flow will be explained below.

[0072] Okay, now I'll explain the process step by step as you instructed.

[0073] Step 1:

[0074] The user opens a dedicated LINE account, where they are authenticated to use the system.

[0075] Step 2:

[0076] A user sends content (photos, videos, text) using the LINE chat function. At this time, the user checks the content to be sent and clicks the send button at the appropriate time.

[0077] Step 3:

[0078] The server receives the content from the LINE account and temporarily stores the received data in a buffer.

[0079] Step 4:

[0080] The server analyzes the received content, using image recognition technology to extract important elements in the case of images, and text analysis tools to extract keywords and tags from the text.

[0081] Step 5:

[0082] The server stores the parsed data in a database, including the user ID, timestamp, and extracted keywords and tags.

[0083] Step 6:

[0084] The server calls the AI ​​generator, which uses the saved data as input to automatically generate news-style articles. The AI ​​analyzes the content of photos and videos and generates appropriate captions.

[0085] Step 7:

[0086] The generative AI generates captions and stories and determines the overall structure. For example, it creates a caption such as "The moving moment when we came in second place at the sports day" and reflects it in the article.

[0087] Step 8:

[0088] The server sends a preview link of the generated article to the user, who clicks the preview link to check the content of the article.

[0089] Step 9:

[0090] The user selects the media format of the article on the preview screen. Options include magazine, news, album, etc.

[0091] Step 10:

[0092] The server generates the final article based on the media format selected by the user, and the final article is stored on the server.

[0093] Step 11:

[0094] The server sends the final preview link back to the user, who then checks the final preview and verifies that there are no problems with the content or format of the article.

[0095] Step 12:

[0096] The user clicks the distribution button, which causes the server to distribute the article to recipients via LINE based on the distribution list.

[0097] Through each of the above steps, this system completes the process of effectively generating and sharing user experiences and thoughts as articles.

[0098] Example 1

[0099] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] Conventional content sharing systems have limited the means by which users can effectively share their experiences and thoughts with others, often requiring them to manually create articles. This places a heavy burden on users and limits the quality and quantity of information they want to share. It has also been difficult to generate articles compatible with multiple media formats and to effectively distribute them to recipients. To solve these issues, a system that can automatically analyze user experiences and share them effectively is needed.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0102] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, means for performing text analysis using a natural language processing tool, means for analyzing photos and videos using image recognition technology, means for automatically generating articles using an artificial intelligence model, and means for sending the results of the data analysis to the artificial intelligence as prompt sentences. This allows the user to check high-quality articles automatically generated based on the sent content in various media formats and effectively distribute them to multiple recipients.

[0103] "User terminal" refers to an electronic device used by a user to transmit content, including a smartphone, tablet, personal computer, etc.

[0104] "Server" refers to the central system that receives, analyzes, and stores content sent by users, and automatically generates and distributes articles.

[0105] A "generative AI model" refers to an artificial intelligence model that automatically generates articles based on received data and analysis results, and uses natural language generation technology.

[0106] A "prompt sentence" refers to a sentence that is input into a generative AI model based on the results of data analysis, and includes instructions for generating appropriate captions and stories.

[0107] The "means for receiving" refers to a mechanism by which the server receives multiple contents sent by the user.

[0108] "Means for analyzing" refers to the processes and techniques used to analyze received content and extract significant keywords and tags.

[0109] "Means for automatically generating news-style articles" refers to systems and technologies for automatically creating news-style text based on analyzed data.

[0110] "Means for selecting media format" refers to a mechanism for choosing from multiple formats for the appearance and layout of the generated article.

[0111] "Means for distribution to multiple recipients" refers to a mechanism by which the server transmits the generated article to multiple designated recipients.

[0112] "Natural language processing tools" refers to software and technology for analyzing text data and extracting important keywords and tags.

[0113] "Image recognition technology" refers to technology that analyzes the content of photos and videos and recognizes objects and scenes.

[0114] "Means for automatically generating articles using an artificial intelligence model" refers to a mechanism that uses a generative AI model to automatically create articles based on received and analyzed data.

[0115] "Means for transmitting data analysis results to artificial intelligence as prompt sentences" refers to the processes and techniques for generating the analyzed data as prompt sentences and passing them to an artificial intelligence model.

[0116] The "means for final confirmation" refers to a mechanism by which a user can confirm, edit, or approve the generated article.

[0117] The "means for sending a final preview to a user" refers to a mechanism for sending a preview of the generated article to a user for confirmation.

[0118] The present invention provides a system that allows users to share their experiences and thoughts with others. This system receives and analyzes content sent by users, and generates and distributes articles based on that content, thereby stimulating communication.

[0119] System Overview

[0120] The system consists of the following main components:

[0121] User terminal: A device for transmitting content (such as a smartphone, tablet, or personal computer).

[0122] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[0123] Generative AI model: An artificial intelligence model that automatically generates news-style articles based on the data it receives.

[0124] User operations

[0125] Users send content using dedicated messaging applications. For example, users send photos, videos, and impressions of a sports day using the chat function of the messaging application.

[0126] Server Processing

[0127] The server receives the content (photos, videos, text) sent by the user using the messaging API. After receiving the content, the server performs the following actions:

[0128] 1. Data Analysis: The server analyzes the data sent and extracts important keywords and tags. Specifically, it uses the following techniques:

[0129] Image analysis: The content of the photos and videos sent is analyzed using image recognition technology. Specifically, general image recognition technology is used.

[0130] Text analysis: Use natural language processing tools to analyze the text submitted by the user, such as spaCy or a natural language processing API.

[0131] 2. Calling a generative AI model: Based on the analysis results, a generative AI model (such as a common generative AI model like GPT-4) is called to automatically generate a news-style article.

[0132] Prompt creation: The server compiles the results of image analysis and text analysis into a prompt. Examples of specific prompts are as follows:

[0133] Photo caption: "The touching moment when I came second at the sports day."

[0134] Testimonial: "The children did a great job in beautiful weather. We were all particularly impressed by the high jump."

[0135] Media Format Selection

[0136] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[0137] Article generation and distribution

[0138] The server generates the final article based on the media format selected by the user, and sends the article to the user as a final preview. After the user gives their final confirmation, they click the distribute button, and the server sends the article to multiple specified recipients.

[0139] Specific examples

[0140] Example 1: Sharing your child's growth

[0141] 1. The user opens a dedicated messaging application and sends photos, videos, and impressions of the sports day.

[0142] 2. The server receives this data using a messaging API.

[0143] 3. The server uses image recognition technology to analyze the content of the photo and obtain the information that the photo "came in second place." At the same time, it uses natural language processing tools to analyze the comments and extract important keywords (such as "great weather" and "the kids worked hard").

[0144] 4. The server sends prompts based on the analysis results to the generative AI model.

[0145] 5. The generative AI model generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[0146] 6. The user clicks on the preview link sent by the server and selects the magazine-style media format.

[0147] 7. The server generates a magazine-style article and sends it to the user as a final preview.

[0148] 8. The user checks the preview and clicks the share button, and the server distributes the article to the specified relatives and friends.

[0149] In this way, the present invention provides a concrete means for effectively sharing users' experiences and thoughts and stimulating communication.

[0150] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0151] Step 1: Submit your content

[0152] Users use a dedicated messaging application to send content such as photos, videos, and text. Specifically, users select a photo or video on the messaging application and enter their thoughts and descriptions in text. This content is then sent to the server via the messaging API.

[0153] Input: Photos, videos, text

[0154] Output: Data sent from the user to the server

[0155] Step 2: Receiving content

[0156] The server receives the content sent by the user using the messaging API, and stores it in a database for further analysis.

[0157] Input: User-submitted content (photos, videos, text)

[0158] Output: Content received and stored in a database

[0159] Step 3: Prepare for data analysis

[0160] The server prepares the received content for analysis by placing the data in a queue for passing to the analysis block.

[0161] Input: Content stored in the database

[0162] Output: Content queued for analysis

[0163] Step 4: Data analysis

[0164] The server analyzes the transmitted content in the following way:

[0165] 1. Image analysis: The server uses image recognition technology to analyze the content of the photos and videos you send. For example, it uses common image recognition technology to recognize objects and scenes in the images and gather information for caption generation.

[0166] Input: Image or video in the content

[0167] Output: Information about objects and scenes in the image

[0168] 2. Text Analysis: The server uses natural language processing tools to analyze the text submitted by the user, such as "spaCy" or "a type of natural language processing API," to extract important keywords and tags.

[0169] Input: The text submitted by the user

[0170] Output: Important keywords, tags, and sentiment analysis results

[0171] Step 5: Invoke the generative AI model

[0172] Based on the analysis results, the server calls a generative AI model (such as a common generative AI model like "GPT-4") to automatically generate news-style articles.

[0173] Input: Image analysis results, text analysis results

[0174] Output: Prompt sentence to be passed to the generative AI model

[0175] Specific behavior: The server generates a prompt based on the parsed result:

[0176] Photo caption: "The touching moment when I came second at the sports day."

[0177] Testimonial: "The children did a great job in beautiful weather. We were all particularly impressed by the high jump."

[0178] Step 6: Auto-generating articles

[0179] The generative AI model generates news-style articles based on prompts sent from the server, creates appropriate captions based on the input prompts, and builds the entire story.

[0180] Input: prompt statement

[0181] Output: Auto-generated article

[0182] What it does: A generative AI model generates news articles, including photo captions and the overall story.

[0183] Step 7: Select the media format

[0184] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[0185] Input: Preview link, generated article

[0186] Output: Selected media format

[0187] What happens: The user clicks on the preview link you sent them and selects the media format they want.

[0188] Step 8: Final article generation and confirmation

[0189] The server generates the final article based on the media format selected by the user, and sends the generated article to the user as a final preview.

[0190] Input: Selected media format, generated article

[0191] Output: Final preview article

[0192] What happens: The server formats the article using the selected media format and sends it to the user as a final preview.

[0193] Step 9: Article Distribution

[0194] After the user checks the final preview, they click the distribute button, and the server sends the article to multiple specified recipients.

[0195] Input: Final preview article, recipient list

[0196] Output: Delivery completion notification to the recipient

[0197] What happens: The user clicks the distribute button, and the server distributes the article to the specified recipient list.

[0198] (Application example 1)

[0199] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0200] In systems that allow users to effectively share their experiences and thoughts with others, there are problems such as the complexity of content analysis and article generation, and the inability of users to edit or preview articles, which increases the possibility of articles being distributed that do not match the user's intentions.

[0201] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0202] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, and means for previewing and editing the generated articles. This allows users to edit articles in a way that suits their own intentions and distribute them after final confirmation.

[0203] "User" means an individual or organization that uses the system to submit content to share their experiences and thoughts with others.

[0204] "Content" is a general term for digital data such as photos, videos, and text sent by users.

[0205] "Means for receiving" refers to the method or technology by which a central system such as a server receives the transmitted content.

[0206] "Means for analysis" refers to methods and technologies for understanding the content of received content and extracting important keywords and tags.

[0207] A "keyword" is a word or phrase that indicates important information about the content and characterizes the content.

[0208] A "tag" is a label that indicates the attributes or category of content, and is information that makes it easier for users to organize and search for content.

[0209] A "news-style article" is a document that describes a user's experience in the form of news, including automatically generated content.

[0210] "Automatic generation means" refers to the software and algorithms used to mechanically generate articles based on the analysis results.

[0211] The "media format" refers to the format or style for displaying the generated article, and includes news style, magazine style, album style, and the like.

[0212] A "production method" is a method or technique for producing an article in a selected media format.

[0213] A "delivery means" is a method or technique for sending the final generated article to the intended recipient.

[0214] A "previewing means" is a method or technique for displaying the generated article for final user confirmation.

[0215] "Editing means" refers to methods or techniques that allow users to make changes to generated articles.

[0216] A specific system architecture and its operation for implementing the present invention will now be described.

[0217] System Overview

[0218] The system of the present invention consists of the following main components:

[0219] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[0220] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[0221] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data.

[0222] Communication infrastructure: The Internet that connects user terminals and servers.

[0223] User terminal operation procedure

[0224] Users use a dedicated mobile application to send content, such as photos, videos, and impressions of sports days, using the app's in-app features.

[0225] Server Processing

[0226] The server receives the content sent from the user terminal. The received content is processed as follows:

[0227] 1. Data reception: The server receives the photos, videos, and text data sent by the user.

[0228] 2. Data Analysis: Analyzes the received data and extracts important keywords and tags. Using image recognition technology and natural language processing tools, the content of photos and videos is analyzed in detail.

[0229] 3. Generative AI call: Based on the analysis results, a generative AI model is called to automatically generate news-style articles. Libraries such as Hugging Face Transformers are used.

[0230] Article Generation Process

[0231] The generative AI automatically generates articles based on the received data and analysis results. Specifically, it generates appropriate captions from images and videos, and then generates the sentences that make up the overall story.

[0232] Select and preview media formats

[0233] The generated article is sent as a preview from the server to the user's device. The user can select from multiple media formats, such as news, magazine, and album, and edit the article as needed.

[0234] delivery

[0235] After the user has given their final confirmation, the article is distributed to multiple recipients via LINE or other social media platforms.

[0236] Specific examples

[0237] Example 1: Sharing your child's growth

[0238] 1. The user (parent) opens the dedicated app and sends photos, videos, and impressions of the sports day.

[0239] 2. The server receives this data, checks the format and size of the images and videos, and analyzes the text.

[0240] 3. The AI ​​generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[0241] 4. The user selects the magazine-style media format, and the server sends the magazine-style article as the final preview.

[0242] 5. After user confirms, distribute the article to relatives and friends.

[0243] Example 2: Amateur sports promotion

[0244] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[0245] 2. The server receives and analyzes this data.

[0246] 3. Based on the content of the video, the generative AI generates a caption such as "The next generation star player's amazing play," and assembles the whole video into an article.

[0247] 4. The user selects a news-like media format, and the server sends the news-like article as the final preview.

[0248] 5. After the user has confirmed it, the article is distributed to fans and supporters.

[0249] Prompt Sentence Examples

[0250] "Upload five photos from the sports day and enter the following comment: 'Today was a great sports day. It was impressive to see the kids running around with such energy.'"

[0251] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0252] Step 1:

[0253] The user launches a dedicated mobile application and sends content such as photos, videos, and text.

[0254] Input: Photos, videos, and text data from users.

[0255] Output: The content sent to the server.

[0256] Specific actions: The user uses the app to enter photos and videos from the sports day, as well as their thoughts, and then taps the send button.

[0257] Step 2:

[0258] The server receives the content sent by the user.

[0259] Input: Digital content sent from a user device.

[0260] Output: Data stored in the server's storage.

[0261] Specific operation: The server receives a request from the user terminal and stores the received content in temporary storage.

[0262] Step 3:

[0263] The server analyzes the received content and extracts keywords and tags.

[0264] Input: Photos, videos, and text data stored on the server.

[0265] Output: A list of extracted keywords and tags.

[0266] Specific operation: The server analyzes the content using image recognition technology (e.g., OpenCV) and natural language processing tools (e.g., NLTK) and extracts important keywords and tags.

[0267] Step 4:

[0268] The server calls a generative AI model based on the analysis results and automatically generates news-style articles.

[0269] Input: Analysis results (list of keywords and tags), generative AI model.

[0270] Output: The generated news-style article.

[0271] How it works: The server uses the Hugging Face Transformers library to input the analysis results into a generative AI model and provide a prompt for article generation. It uses a prompt such as, "Today was a great sports day. It was impressive to see the children running around with such energy."

[0272] Step 5:

[0273] The server transmits the generated article to the user terminal as a preview.

[0274] Input: A generated news-style article.

[0275] Output: Article preview displayed on the user's device.

[0276] Specific operation: The server converts the article into HTML format and sends it to the user's device. The user can then preview the article through the app.

[0277] Step 6:

[0278] The user reviews the article and makes edits as necessary.

[0279] Input: The article preview shown to the user.

[0280] Output: The edited article.

[0281] What it does: Users can check the preview, correct typos, or enter additional information. Once they're done editing, they tap the save button.

[0282] Step 7:

[0283] The server delivers the final checked article to multiple recipients.

[0284] Input: The last article edited and reviewed by the user.

[0285] Output: The final article distributed to multiple recipients.

[0286] Specific operation: The server distributes the articles confirmed by the user via LINE or other SNS based on the specified recipient list.

[0287] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0288] Understood. Now, I will describe the "Form for carrying out the invention" based on the "Claims for the invention combining emotion engines."

[0289] This invention provides a system that generates more accurate and emotionally rich articles by combining analysis of content sent by users with an emotion engine that recognizes the user's emotions. This system allows users to effectively share their experiences and thoughts with others, stimulating communication.

[0290] System configuration

[0291] The system consists of the following main components:

[0292] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[0293] Server: The central system that receives, analyzes, stores, generates, and distributes content.

[0294] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data and sentiment data.

[0295] Sentiment Engine: A tool that analyzes and extracts sentiment information from user-submitted content.

[0296] User operation procedure

[0297] Users can send photos, videos, and text from a dedicated LINE account. For example, they can use the LINE chat function to send photos, videos, and comments about a sports day.

[0298] Server Processing

[0299] The server receives the content sent by the user, stores it in a buffer, and then analyzes it using image recognition technology and text analysis tools to extract important keywords and tags.

[0300] Emotion engine processing

[0301] The emotion engine recognizes emotions contained in the received content. For example, it analyzes the user's emotions such as joy, surprise, and sadness from the transmitted text and voice data. This generates emotion tags and assigns them to the content. These emotion tags are taken into account in the article generation process.

[0302] Processing of generated AI

[0303] The generative AI automatically generates articles based on the analyzed data and emotion tags. It creates captions for images and videos and constructs stories that reflect the emotion tags. For example, when generating a caption to express "the touching moment of coming second in a sports day," it reflects the user's emotion tag for joy.

[0304] Selecting a Media Format

[0305] The user can click on the preview link sent by the server to view the generated article, where they can choose from multiple media formats such as magazine, news, and album.

[0306] Article generation and distribution

[0307] The server generates the final article based on the media format selected by the user. This final article is saved on the server and sent back to the user as a preview link. The user can check the final preview and click the share button to distribute the article via LINE to the specified recipient list.

[0308] Specific examples

[0309] Example 1: Sharing your child's growth

[0310] 1. The user (parent) opens a dedicated LINE account and sends photos, videos, and impressions of the sports day.

[0311] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[0312] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[0313] 4. The generative AI generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[0314] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[0315] 6. After user confirms, distribute the article to relatives and friends.

[0316] Example 2: Amateur sports promotion

[0317] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[0318] 2. The server receives and analyzes this data.

[0319] 3. The emotion engine recognizes the user's emotions of "surprise" and "pride" from their post-match impressions and assigns emotion tags.

[0320] 4. The generative AI generates captions such as "The next generation star player's amazing play" and composes articles that reflect the sentiment tags.

[0321] 5. The user selects a news-like media format and the server sends the news-like article as the final preview.

[0322] 6. After the user has confirmed it, the article is distributed to fans and supporters.

[0323] In this way, the present invention provides a system that combines emotion engines to enable users to share their experiences and thoughts in a richer way and achieve more effective communication.

[0324] The processing flow will be explained below.

[0325] Understood. Now, we will explain each processing step based on the "Patent Claims for Inventions Combining Emotion Engines" in detail in the following format.

[0326] Step 1:

[0327] Users use a dedicated LINE account to send content such as photos, videos, and text. For example, a user can upload photos and videos from their child's sports day, or write their thoughts after the event, and click the send button.

[0328] Step 2:

[0329] The server receives the content sent by the user, and the received content is temporarily stored in a buffer.

[0330] Step 3:

[0331] The server analyzes the received content, specifically using image recognition technology to analyze the content of photos and videos, and using text analysis tools to extract keywords and tags from the text data.

[0332] Step 4:

[0333] The emotion engine recognizes the user's emotions from the received and analyzed text and voice data. For example, it recognizes the emotion of "joy" from the positive nuances in the text and generates an emotion tag.

[0334] Step 5:

[0335] The server stores the analyzed data and emotion tags in a database. The stored data includes user IDs, timestamps, keywords, tags, and emotion tags.

[0336] Step 6:

[0337] The server calls the generation AI, which automatically generates news-style articles using content data and emotion tags as input. The generation AI generates captions from images and videos and composes articles that reflect the emotion tags.

[0338] Step 7:

[0339] The AI ​​then creates an entire story based on the generated captions and emotion tags. For example, it incorporates an emotionally rich caption such as "The touching moment when we came in second place at the sports day."

[0340] Step 8:

[0341] The server sends the user a preview link of the generated article, which the user clicks to view the article.

[0342] Step 9:

[0343] The user selects the media format of the article on the preview screen, such as magazine style, news style, or album style.

[0344] Step 10:

[0345] The server generates the final article based on the media format selected by the user, and the final article is stored on the server.

[0346] Step 11:

[0347] The server sends a final preview link back to the user, who then checks the final preview to ensure the article is correct in terms of content and format.

[0348] Step 12:

[0349] The user clicks the distribution button, which causes the server to distribute the article to recipients via LINE based on the distribution list.

[0350] As a result, a system that combines an emotion engine can share users' experiences and thoughts more accurately and emotionally.

[0351] Example 2

[0352] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0353] Conventional content generation systems are limited to analyzing the content submitted by users and do not adequately consider emotional expression. As a result, the generated articles do not effectively reflect users' emotions and thoughts, resulting in insufficient content sharing and communication. To solve this problem, a system that analyzes emotions and reflects them in article generation is needed.

[0354] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0355] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for recognizing emotions from the received content and generating emotion tags, means for automatically generating news-style articles based on the analyzed data and emotion tags, means for selecting a media format for the generated article, and means for generating articles based on the selected media format and distributing them to multiple recipients. This enables article generation that reflects the user's emotions, enabling more effective content sharing and communication.

[0356] "User" means an individual or organization that utilizes the system to submit content, review generated articles, and distribute them.

[0357] "Content" refers to information such as photos, videos, and text that users submit to the system.

[0358] The "receiving means" is a mechanism by which the server receives the content sent by the user and temporarily stores it.

[0359] "Analysis means" refers to a system that analyzes received content using image recognition technology and text analysis tools to extract important keywords and tags.

[0360] The "emotion engine" is a mechanism that recognizes emotions within received content and generates emotion tags based on those emotions.

[0361] An "emotion tag" is an identifier that indicates the emotional information analyzed by the emotion engine and is used to generate articles.

[0362] "Generative AI" is artificial intelligence that automatically generates news-style articles based on analyzed data and emotion tags.

[0363] "Media format" refers to the layout and style in which an article is displayed.

[0364] The "distribution means" is a mechanism for transmitting the generated article to multiple recipients.

[0365] The "final confirmation means" is a mechanism by which a user can finally confirm the content of a generated article.

[0366] Okay, so based on the information we have so far, we'll create a detailed description.

[0367] This invention provides a system that generates more accurate and emotionally rich articles by combining analysis of content sent by users with an emotion engine that recognizes the user's emotions. This system allows users to effectively share their experiences and thoughts with others, stimulating communication.

[0368] System configuration

[0369] The system consists of the following main components:

[0370] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[0371] Server: The central system that receives, analyzes, stores, generates, and distributes content.

[0372] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data and sentiment data.

[0373] Sentiment Engine: A tool that analyzes and extracts sentiment information from user-submitted content.

[0374] User operation procedure

[0375] Users send photos, videos, and text from a dedicated messaging application. For example, when sending photos, videos, or impressions of a sports day, they use the chat function of the messaging application.

[0376] Server Processing

[0377] The server receives the content sent by the user. After receiving the content, it first temporarily stores it in a buffer. Then, the server analyzes the data using image recognition technology (e.g., a general image recognition API) and text analysis tools (e.g., a general text analysis API) to extract important keywords and tags.

[0378] Emotion engine processing

[0379] The emotion engine recognizes emotions contained in the received content. For example, it analyzes the user's emotions such as joy, surprise, and sadness from the transmitted text and voice data. This generates emotion tags and assigns them to the content. These emotion tags are taken into account in the article generation process.

[0380] Processing of generated AI

[0381] The generative AI automatically generates articles based on the analyzed data and emotion tags. It creates captions for images and videos and constructs stories that reflect the emotion tags. For example, when generating a caption to express "the touching moment of coming second in a sports day," it reflects the user's emotion tag of "joy."

[0382] Selecting a Media Format

[0383] The user can click on the preview link sent by the server to view the generated article, where they can choose from multiple media formats such as magazine, news, and album.

[0384] Article generation and distribution

[0385] The server generates the final article based on the media format selected by the user, saves the final article on the server, and sends it back to the user as a preview link. The user can view the final preview and click the distribute button, which distributes the article via a messaging application based on the specified recipient list.

[0386] Specific examples

[0387] Sharing your child's growth

[0388] 1. The user (parent) opens a dedicated messaging application and sends photos, videos, and impressions of the sports day.

[0389] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[0390] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[0391] 4. The generative AI generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[0392] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[0393] 6. After user confirms, distribute the article to relatives and friends.

[0394] Amateur sports promotion

[0395] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[0396] 2. The server receives and analyzes this data.

[0397] 3. The emotion engine recognizes the user's emotions of "surprise" and "pride" from their post-match impressions and assigns emotion tags.

[0398] 4. The generative AI generates captions such as "The next generation star player's amazing play" and composes articles that reflect the sentiment tags.

[0399] 5. The user selects a news-like media format and the server sends the news-like article as the final preview.

[0400] 6. After the user has confirmed it, the article is distributed to fans and supporters.

[0401] This system enables users to effectively share their experiences and thoughts, enabling richer communication.

[0402] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0403] I understand. Now, I will explain the program processing flow of this system by dividing it into specific steps.

[0404] Program processing flow

[0405] Step 1: User submits content

[0406] A user sends content such as photos, videos, and text using a dedicated messaging application.

[0407] Input: Photos, videos, and text data sent by users.

[0408] Output: The content data sent to the server.

[0409] Specific operation: The user opens a message sending application on a smartphone or PC, enters "Photos from the sports day," "Video from the sports day," and "I was so happy to come in second place at the sports day," and presses the send button.

[0410] Step 2: The server receives and stores the content

[0411] The server receives the content sent by the user and temporarily stores it in a buffer.

[0412] Input: Content data submitted by the user.

[0413] Output: The content data stored in the buffer.

[0414] Specific operation: The server separates the received photos, videos, and text into their respective data formats and temporarily stores them in buffer storage with a file name such as "userID_12345".

[0415] Step 3: The server parses the content

[0416] The server uses image recognition technology and text analysis tools to analyze the stored content and extract important keywords and tags.

[0417] Input: The content data stored in the buffer.

[0418] Output: Extracted keywords, tag list.

[0419] Specific operation: The server uses an image recognition API to extract keywords such as "sports day" and "podium" from the photo, and uses a text analysis API to extract keywords such as "second place" and "I was happy" from the text data.

[0420] Step 4: The emotion engine recognizes emotions and assigns tags

[0421] The emotion engine analyzes the emotions contained in the received content, generates emotion tags, and assigns them to the content.

[0422] Input: Extracted keywords, tag list.

[0423] Output: Emotion-tagged data.

[0424] Specific operation: The emotion engine uses a text analysis API to identify the user's emotion, such as "joy" or "surprise," generate tags based on that emotion, and assign the "joy" emotion tag to the text data.

[0425] Step 5: Generative AI generates articles

[0426] Generative AI automatically generates news-style articles based on the analyzed data and sentiment tags.

[0427] Input: Emotion-tagged data.

[0428] Output: The generated news-style article.

[0429] Specific operation: The generative AI (for example, GPT-3) generates captions such as "The touching moment when we came in second place at the sports day" and creates articles that reflect the emotion tag "joy."

[0430] Step 6: User selects media format

[0431] The user clicks on the preview link sent from the server to check the generated article and selects a media format such as magazine style, news style, or album style.

[0432] Input: A generated news-style article.

[0433] Output: Information for the selected media format.

[0434] What happens: The user clicks the preview link to see the generated article and selects the "magazine-style layout."

[0435] Step 7: The server generates the final article

[0436] The server generates the final article based on the media format selected by the user.

[0437] Input: Selected media format information, generated news format article.

[0438] Output: The final generated article.

[0439] What it does: The server uses a magazine-style layout template to position photos, captions, and text appropriately to generate the final article.

[0440] Step 8: Server sends final article

[0441] The server sends the final article to the user as a preview link.

[0442] Input: The final generated article.

[0443] Output: Preview link.

[0444] Specific operation: The server saves the generated final article in storage, generates a preview link, and sends it to the user.

[0445] Step 9: User publishes article

[0446] The user sees the final preview and distributes the article to multiple recipients.

[0447] Input: Preview link.

[0448] Output: A list of recipients to whom the article will be distributed.

[0449] Specific operation: The user checks the final preview and clicks the "Distribute" button to send the article to the specified recipients. The server then distributes the article via a messaging application such as LINE based on the recipient list.

[0450] Through the above processing steps, the system generates articles that more fully reflect the user's experiences and emotions, achieving effective communication.

[0451] (Application example 2)

[0452] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0453] In recent years, there has been a growing demand for emotionally rich communication with others via the Internet. However, conventional content generation systems have difficulty automatically generating articles that accurately reflect users' emotions, and as a result, users' experiences and thoughts cannot be fully shared. Furthermore, the generated articles tend to be flat and unattractive, limiting their impact on the recipient. There is a need for a system that can solve these issues and automatically generate and distribute compelling content that reflects users' emotions.

[0454] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, means for analyzing emotions in the content and assigning emotion tags, and means for generating articles that reflect the emotion tags. This makes it possible for users to share their experiences and thoughts with others in an emotionally rich way, providing an emotional impact on recipients.

[0455] "Content" is a general term for digital data such as photos, videos, and text sent by users.

[0456] An "emotion tag" is an information tag that is assigned based on the emotion analyzed from the content sent by the user.

[0457] "Generative AI" is artificial intelligence that automatically generates news-style articles and captions based on received data and emotional data.

[0458] The "emotion engine" is a tool that analyzes emotions within content submitted by users and extracts that information.

[0459] "Media format" refers to the display format and design of the generated article, and includes news format, magazine format, album format, and the like.

[0460] A "recipient list" is a set of designated recipients to whom a generated article is to be distributed.

[0461] "Keywords" are important words or phrases extracted by analyzing content.

[0462] A "tag" is classification information or identification information that is assigned to content.

[0463] "Stories" are emotionally rich, sequential articles and content created based on analyzed data and sentiment tags.

[0464] A "preview link" is a URL link provided to allow a user to see a preview of the generated article.

[0465] This invention is a system that analyzes content sent by users, recognizes their emotions, and automatically generates and distributes articles that reflect those emotions. Below, we will explain in detail the system and processing procedures for implementing this invention.

[0466] System configuration

[0467] User device:

[0468] Users use devices such as smartphones, tablets, and PCs to send content.

[0469] server:

[0470] This is the central system that receives, analyzes, stores, generates articles, and distributes content. The server uses the following software and libraries:

[0471] TextBlob: A library for analyzing text sentiment

[0472] Pillow (PIL): A library for image processing

[0473] Generative AI model: Artificial intelligence that generates articles based on content and sentiment data

[0474] Emotion Engine: A tool that analyzes emotions in user-submitted content

[0475] User operation procedure

[0476] Users can send photos, videos, and text from a dedicated smartphone app. For example, they can use the app's input interface to send photos, videos, and comments about a sports day.

[0477] Server Processing

[0478] The server receives the content sent by the user and first stores it in a temporary buffer. Next, it uses TextBlob to analyze the emotion of the received text data and generates emotion tags based on the results. It also uses Pillow to obtain basic information about the received image and video data.

[0479] Emotion engine processing

[0480] The emotion engine analyzes the user's emotions, such as joy, surprise, and sadness, from the received text and voice data, generates emotion tags, and assigns them to the content. These emotion tags are taken into account in the article generation process.

[0481] Processing of generated AI

[0482] The generative AI model automatically generates articles based on the analyzed data and sentiment tags, for example by adding captions to incoming photos and constructing stories that reflect the sentiment tags.

[0483] Preview and distribute content

[0484] The user can click the preview link sent by the server to check the generated article, where they can choose from multiple media formats such as news, magazine, album, etc. After final confirmation, the user clicks the distribution button, and the article is distributed to the specified recipient list.

[0485] Specific examples

[0486] Example 1: Sharing your child's progress

[0487] 1. The user (parent) opens a dedicated smartphone app and sends photos and videos of the sports day, along with their thoughts such as "Today was a fun sports day!"

[0488] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[0489] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[0490] 4. The generative AI model generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[0491] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[0492] 6. After user confirms, distribute the article to relatives and friends.

[0493] Prompt Sentence Examples

[0494] "We had a fun family sports day today! We took lots of photos and videos. Let us know what you think of it!"

[0495] In this way, this system allows users to share their experiences and thoughts with others in an emotionally rich way, realizing more effective communication.

[0496] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0497] Step 1:

[0498] The user launches the smartphone app, enters photos, videos, and text, and clicks the send button.

[0499] Input: photos, videos, text

[0500] Output: Transmitted data

[0501] Specific operation: The user enters, for example, photos and videos from the sports day and a comment such as "Today was a fun sports day!" into the smartphone app's input interface, and then presses the send button to send the data to the server.

[0502] Step 2:

[0503] The server receives the data sent by the user and temporarily stores it in a buffer.

[0504] Input: Send data

[0505] Output: Received data

[0506] Specific operation: The server detects the reception of transmitted data and temporarily stores it in buffer memory. Here, photos and videos are stored as binary data, and text is stored as character string data.

[0507] Step 3:

[0508] The server parses the incoming data and uses TextBlob to analyze the sentiment of the text and extract keywords and tags.

[0509] Input: Received data

[0510] Output: sentiment tags, keywords, tags

[0511] Specific operation: The server sends the saved string data to the TextBlob library for sentiment analysis. For example, from the text "Today was a fun sports day!", a "joy" tag is generated, and important keywords such as "sports day" and "fun" are extracted.

[0512] Step 4:

[0513] The server uses Pillow to obtain basic information about the received image and video data.

[0514] Input: Received data (images, videos)

[0515] Output: Basic information (image / video format, size, etc.)

[0516] What it does: The server opens the saved binary data using the Pillow library and retrieves basic information such as image format and size. It also analyzes video metadata.

[0517] Step 5:

[0518] The emotion engine analyzes emotions in the received data and assigns emotion tags.

[0519] Input: text data, image data, video data

[0520] Output: Emotion tag

[0521] Specific operation: The emotion engine in the server analyzes emotions from the entire received data. For example, it analyzes the content of a video and generates emotion tags such as "surprise" or "emotion" from specific scenes and assigns them to the data.

[0522] Step 6:

[0523] The generative AI model automatically generates articles based on the analyzed data and sentiment tags.

[0524] Input: emotion tags, keywords, basic information about images and videos

[0525] Output: Auto-generated article

[0526] Specific operation: The generative AI model on the server generates captions such as "The touching moment when we came in second place at the sports day" based on emotion tags and keywords, and creates an emotionally rich story-style article.

[0527] Step 7:

[0528] The server allows the user to select the media format of the generated article and then finalizes the article based on the selection.

[0529] Input: Auto-generated article

[0530] Output: Final generated article

[0531] Specific operation: The server sends the preview link to the user and lets them choose from media formats such as magazine format, news format, etc. Then it receives the selection results and constructs the final article.

[0532] Step 8:

[0533] The user checks the final generated article and clicks the distribution button.

[0534] Input: Final generated article

[0535] Output: Delivery instructions

[0536] Specific operation: The user clicks the preview link to check the final generated article, and then, if satisfied, clicks the publish button to instruct the article to be published.

[0537] Step 9:

[0538] The server distributes the final generated article based on a selected list of recipients.

[0539] Input: Delivery instructions, recipient list

[0540] Output: Delivery complete

[0541] Specific operation: The server receives distribution instructions and sends the article via email or social media based on the specified recipient list.

[0542] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0543] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0544] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0545] [Second embodiment]

[0546] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0547] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0548] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0549] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0550] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0551] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0552] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0553] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0554] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0555] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0556] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0557] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0558] Understood. Now, let's write the "Form for carrying out the invention."

[0559] The present invention provides a system that allows users to share their experiences and thoughts with others. This system receives and analyzes content sent by users, and generates and distributes articles based on that content, thereby stimulating communication.

[0560] System Overview

[0561] The system consists of the following main components:

[0562] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[0563] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[0564] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data.

[0565] User operation procedure

[0566] Users can use a dedicated LINE account to send content, such as photos, videos, and comments from a sports day, using the LINE chat function.

[0567] Server Processing

[0568] The server receives content sent from LINE accounts. After receiving the data, the server analyzes it and extracts important keywords and tags. It uses image recognition technology to understand the content of photos and videos, and natural language processing tools to analyze text data. It then invokes a generation AI based on the analysis results to automatically generate news-style articles.

[0569] Processing of generated AI

[0570] The generative AI automatically generates articles based on the received data and analysis results. First, it generates appropriate captions from images and videos, and then it generates the sentences that make up the overall story. In this process, it takes into account keywords and tags to create articles that reflect the user's intentions.

[0571] Selecting a Media Format

[0572] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[0573] Article generation and distribution

[0574] The server generates the final article based on the media format selected by the user. The generated article is sent to the user as a final preview. After the user confirms it, they click the distribution button, and the server distributes the article via LINE based on the specified recipient list.

[0575] Specific examples

[0576] Example 1: Sharing your child's growth

[0577] 1. The user (parent) opens a dedicated LINE account and sends photos, videos, and impressions of the sports day.

[0578] 2. The server receives this data, checks the format and size of the images and videos, and analyzes the text.

[0579] 3. The AI ​​generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[0580] 4. The user selects the magazine-style media format, and the server sends the magazine-style article as the final preview.

[0581] 5. After user confirms, distribute the article to relatives and friends.

[0582] Example 2: Amateur sports promotion

[0583] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[0584] 2. The server receives and analyzes this data.

[0585] 3. Based on the content of the video, the generative AI generates a caption such as "The next generation star player's amazing play," and assembles the whole video into an article.

[0586] 4. The user selects a news-like media format, and the server sends the news-like article as the final preview.

[0587] 5. After the user has confirmed it, the article is distributed to fans and supporters.

[0588] Through the above-described embodiments, the present invention provides specific means for effectively sharing users' experiences and thoughts and for activating communication.

[0589] The processing flow will be explained below.

[0590] Okay, now I'll explain the process step by step as you instructed.

[0591] Step 1:

[0592] The user opens a dedicated LINE account, where they are authenticated to use the system.

[0593] Step 2:

[0594] A user sends content (photos, videos, text) using the LINE chat function. At this time, the user checks the content to be sent and clicks the send button at the appropriate time.

[0595] Step 3:

[0596] The server receives the content from the LINE account and temporarily stores the received data in a buffer.

[0597] Step 4:

[0598] The server analyzes the received content, using image recognition technology to extract important elements in the case of images, and text analysis tools to extract keywords and tags from the text.

[0599] Step 5:

[0600] The server stores the parsed data in a database, including the user ID, timestamp, and extracted keywords and tags.

[0601] Step 6:

[0602] The server calls the AI ​​generator, which uses the saved data as input to automatically generate news-style articles. The AI ​​analyzes the content of photos and videos and generates appropriate captions.

[0603] Step 7:

[0604] The generative AI generates captions and stories and determines the overall structure. For example, it creates a caption such as "The moving moment when we came in second place at the sports day" and reflects it in the article.

[0605] Step 8:

[0606] The server sends a preview link of the generated article to the user, who clicks the preview link to check the content of the article.

[0607] Step 9:

[0608] The user selects the media format of the article on the preview screen. Options include magazine, news, album, etc.

[0609] Step 10:

[0610] The server generates the final article based on the media format selected by the user, and the final article is stored on the server.

[0611] Step 11:

[0612] The server sends the final preview link back to the user, who then checks the final preview and verifies that there are no problems with the content or format of the article.

[0613] Step 12:

[0614] The user clicks the distribution button, which causes the server to distribute the article to recipients via LINE based on the distribution list.

[0615] Through each of the above steps, this system completes the process of effectively generating and sharing user experiences and thoughts as articles.

[0616] Example 1

[0617] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0618] Conventional content sharing systems have limited the means by which users can effectively share their experiences and thoughts with others, often requiring them to manually create articles. This places a heavy burden on users and limits the quality and quantity of information they want to share. It has also been difficult to generate articles compatible with multiple media formats and to effectively distribute them to recipients. To solve these issues, a system that can automatically analyze user experiences and share them effectively is needed.

[0619] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0620] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, means for performing text analysis using a natural language processing tool, means for analyzing photos and videos using image recognition technology, means for automatically generating articles using an artificial intelligence model, and means for sending the results of the data analysis to the artificial intelligence as prompt sentences. This allows the user to check high-quality articles automatically generated based on the sent content in various media formats and effectively distribute them to multiple recipients.

[0621] "User terminal" refers to an electronic device used by a user to transmit content, including a smartphone, tablet, personal computer, etc.

[0622] "Server" refers to the central system that receives, analyzes, and stores content sent by users, and automatically generates and distributes articles.

[0623] A "generative AI model" refers to an artificial intelligence model that automatically generates articles based on received data and analysis results, and uses natural language generation technology.

[0624] A "prompt sentence" refers to a sentence that is input into a generative AI model based on the results of data analysis, and includes instructions for generating appropriate captions and stories.

[0625] The "means for receiving" refers to a mechanism by which the server receives multiple contents sent by the user.

[0626] "Means for analyzing" refers to the processes and techniques used to analyze received content and extract significant keywords and tags.

[0627] "Means for automatically generating news-style articles" refers to systems and technologies for automatically creating news-style text based on analyzed data.

[0628] "Means for selecting media format" refers to a mechanism for choosing from multiple formats for the appearance and layout of the generated article.

[0629] "Means for distribution to multiple recipients" refers to a mechanism by which the server transmits the generated article to multiple designated recipients.

[0630] "Natural language processing tools" refers to software and technology for analyzing text data and extracting important keywords and tags.

[0631] "Image recognition technology" refers to technology that analyzes the content of photos and videos and recognizes objects and scenes.

[0632] "Means for automatically generating articles using an artificial intelligence model" refers to a mechanism that uses a generative AI model to automatically create articles based on received and analyzed data.

[0633] "Means for transmitting data analysis results to artificial intelligence as prompt sentences" refers to the processes and techniques for generating the analyzed data as prompt sentences and passing them to an artificial intelligence model.

[0634] The "means for final confirmation" refers to a mechanism by which a user can confirm, edit, or approve the generated article.

[0635] The "means for sending a final preview to a user" refers to a mechanism for sending a preview of the generated article to a user for confirmation.

[0636] The present invention provides a system that allows users to share their experiences and thoughts with others. This system receives and analyzes content sent by users, and generates and distributes articles based on that content, thereby stimulating communication.

[0637] System Overview

[0638] The system consists of the following main components:

[0639] User terminal: A device for transmitting content (such as a smartphone, tablet, or personal computer).

[0640] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[0641] Generative AI model: An artificial intelligence model that automatically generates news-style articles based on the data it receives.

[0642] User operations

[0643] Users send content using dedicated messaging applications. For example, users send photos, videos, and impressions of a sports day using the chat function of the messaging application.

[0644] Server Processing

[0645] The server receives the content (photos, videos, text) sent by the user using the messaging API. After receiving the content, the server performs the following actions:

[0646] 1. Data Analysis: The server analyzes the data sent and extracts important keywords and tags. Specifically, it uses the following techniques:

[0647] Image analysis: The content of the photos and videos sent is analyzed using image recognition technology. Specifically, general image recognition technology is used.

[0648] Text analysis: Use natural language processing tools to analyze the text submitted by the user, such as spaCy or a natural language processing API.

[0649] 2. Calling a generative AI model: Based on the analysis results, a generative AI model (such as a common generative AI model like GPT-4) is called to automatically generate a news-style article.

[0650] Prompt creation: The server compiles the results of image analysis and text analysis into a prompt. Examples of specific prompts are as follows:

[0651] Photo caption: "The touching moment when I came second at the sports day."

[0652] Testimonial: "The children did a great job in beautiful weather. We were all particularly impressed by the high jump."

[0653] Media Format Selection

[0654] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[0655] Article generation and distribution

[0656] The server generates the final article based on the media format selected by the user, and sends the article to the user as a final preview. After the user gives their final confirmation, they click the distribute button, and the server sends the article to multiple specified recipients.

[0657] Specific examples

[0658] Example 1: Sharing your child's growth

[0659] 1. The user opens a dedicated messaging application and sends photos, videos, and impressions of the sports day.

[0660] 2. The server receives this data using a messaging API.

[0661] 3. The server uses image recognition technology to analyze the content of the photo and obtain the information that the photo "came in second place." At the same time, it uses natural language processing tools to analyze the comments and extract important keywords (such as "great weather" and "the kids worked hard").

[0662] 4. The server sends prompts based on the analysis results to the generative AI model.

[0663] 5. The generative AI model generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[0664] 6. The user clicks on the preview link sent by the server and selects the magazine-style media format.

[0665] 7. The server generates a magazine-style article and sends it to the user as a final preview.

[0666] 8. The user checks the preview and clicks the share button, and the server distributes the article to the specified relatives and friends.

[0667] In this way, the present invention provides a concrete means for effectively sharing users' experiences and thoughts and stimulating communication.

[0668] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0669] Step 1: Submit your content

[0670] Users use a dedicated messaging application to send content such as photos, videos, and text. Specifically, users select a photo or video on the messaging application and enter their thoughts and descriptions in text. This content is then sent to the server via the messaging API.

[0671] Input: Photos, videos, text

[0672] Output: Data sent from the user to the server

[0673] Step 2: Receiving content

[0674] The server receives the content sent by the user using the messaging API, and stores it in a database for further analysis.

[0675] Input: User-submitted content (photos, videos, text)

[0676] Output: Content received and stored in a database

[0677] Step 3: Prepare for data analysis

[0678] The server prepares the received content for analysis by placing the data in a queue for passing to the analysis block.

[0679] Input: Content stored in the database

[0680] Output: Content queued for analysis

[0681] Step 4: Data analysis

[0682] The server analyzes the transmitted content in the following way:

[0683] 1. Image analysis: The server uses image recognition technology to analyze the content of the photos and videos you send. For example, it uses common image recognition technology to recognize objects and scenes in the images and gather information for caption generation.

[0684] Input: Image or video in the content

[0685] Output: Information about objects and scenes in the image

[0686] 2. Text Analysis: The server uses natural language processing tools to analyze the text submitted by the user, such as "spaCy" or "a type of natural language processing API," to extract important keywords and tags.

[0687] Input: The text submitted by the user

[0688] Output: Important keywords, tags, and sentiment analysis results

[0689] Step 5: Invoke the generative AI model

[0690] Based on the analysis results, the server calls a generative AI model (such as a common generative AI model like "GPT-4") to automatically generate news-style articles.

[0691] Input: Image analysis results, text analysis results

[0692] Output: Prompt sentence to be passed to the generative AI model

[0693] Specific behavior: The server generates a prompt based on the parsed result:

[0694] Photo caption: "The touching moment when I came second at the sports day."

[0695] Testimonial: "The children did a great job in beautiful weather. We were all particularly impressed by the high jump."

[0696] Step 6: Auto-generating articles

[0697] The generative AI model generates news-style articles based on prompts sent from the server, creates appropriate captions based on the input prompts, and builds the entire story.

[0698] Input: prompt statement

[0699] Output: Auto-generated article

[0700] What it does: A generative AI model generates news articles, including photo captions and the overall story.

[0701] Step 7: Select the media format

[0702] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[0703] Input: Preview link, generated article

[0704] Output: Selected media format

[0705] What happens: The user clicks on the preview link you sent them and selects the media format they want.

[0706] Step 8: Final article generation and confirmation

[0707] The server generates the final article based on the media format selected by the user, and sends the generated article to the user as a final preview.

[0708] Input: Selected media format, generated article

[0709] Output: Final preview article

[0710] What happens: The server formats the article using the selected media format and sends it to the user as a final preview.

[0711] Step 9: Article Distribution

[0712] After the user checks the final preview, they click the distribute button, and the server sends the article to multiple specified recipients.

[0713] Input: Final preview article, recipient list

[0714] Output: Delivery completion notification to the recipient

[0715] What happens: The user clicks the distribute button, and the server distributes the article to the specified recipient list.

[0716] (Application example 1)

[0717] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0718] In systems that allow users to effectively share their experiences and thoughts with others, there are problems such as the complexity of content analysis and article generation, and the inability of users to edit or preview articles, which increases the possibility of articles being distributed that do not match the user's intentions.

[0719] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0720] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, and means for previewing and editing the generated articles. This allows users to edit articles in a way that suits their own intentions and distribute them after final confirmation.

[0721] "User" means an individual or organization that uses the system to submit content to share their experiences and thoughts with others.

[0722] "Content" is a general term for digital data such as photos, videos, and text sent by users.

[0723] "Means for receiving" refers to the method or technology by which a central system such as a server receives the transmitted content.

[0724] "Means for analysis" refers to methods and technologies for understanding the content of received content and extracting important keywords and tags.

[0725] A "keyword" is a word or phrase that indicates important information about the content and characterizes the content.

[0726] A "tag" is a label that indicates the attributes or category of content, and is information that makes it easier for users to organize and search for content.

[0727] A "news-style article" is a document that describes a user's experience in the form of news, including automatically generated content.

[0728] "Automatic generation means" refers to the software and algorithms used to mechanically generate articles based on the analysis results.

[0729] The "media format" refers to the format or style for displaying the generated article, and includes news style, magazine style, album style, and the like.

[0730] A "production method" is a method or technique for producing an article in a selected media format.

[0731] A "delivery means" is a method or technique for sending the final generated article to the intended recipient.

[0732] A "previewing means" is a method or technique for displaying the generated article for final user confirmation.

[0733] "Editing means" refers to methods or techniques that allow users to make changes to generated articles.

[0734] A specific system architecture and its operation for implementing the present invention will now be described.

[0735] System Overview

[0736] The system of the present invention consists of the following main components:

[0737] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[0738] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[0739] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data.

[0740] Communication infrastructure: The Internet that connects user terminals and servers.

[0741] User terminal operation procedure

[0742] Users use a dedicated mobile application to send content, such as photos, videos, and impressions of sports days, using the app's in-app features.

[0743] Server Processing

[0744] The server receives the content sent from the user terminal. The received content is processed as follows:

[0745] 1. Data reception: The server receives the photos, videos, and text data sent by the user.

[0746] 2. Data Analysis: Analyzes the received data and extracts important keywords and tags. Using image recognition technology and natural language processing tools, the content of photos and videos is analyzed in detail.

[0747] 3. Generative AI call: Based on the analysis results, a generative AI model is called to automatically generate news-style articles. Libraries such as Hugging Face Transformers are used.

[0748] Article Generation Process

[0749] The generative AI automatically generates articles based on the received data and analysis results. Specifically, it generates appropriate captions from images and videos, and then generates the sentences that make up the overall story.

[0750] Select and preview media formats

[0751] The generated article is sent as a preview from the server to the user's device. The user can select from multiple media formats, such as news, magazine, and album, and edit the article as needed.

[0752] delivery

[0753] After the user has given their final confirmation, the article is distributed to multiple recipients via LINE or other social media platforms.

[0754] Specific examples

[0755] Example 1: Sharing your child's growth

[0756] 1. The user (parent) opens the dedicated app and sends photos, videos, and impressions of the sports day.

[0757] 2. The server receives this data, checks the format and size of the images and videos, and analyzes the text.

[0758] 3. The AI ​​generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[0759] 4. The user selects the magazine-style media format, and the server sends the magazine-style article as the final preview.

[0760] 5. After user confirms, distribute the article to relatives and friends.

[0761] Example 2: Amateur sports promotion

[0762] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[0763] 2. The server receives and analyzes this data.

[0764] 3. Based on the content of the video, the generative AI generates a caption such as "The next generation star player's amazing play," and assembles the whole video into an article.

[0765] 4. The user selects a news-like media format, and the server sends the news-like article as the final preview.

[0766] 5. After the user has confirmed it, the article is distributed to fans and supporters.

[0767] Prompt Sentence Examples

[0768] "Upload five photos from the sports day and enter the following comment: 'Today was a great sports day. It was impressive to see the kids running around with such energy.'"

[0769] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0770] Step 1:

[0771] The user launches a dedicated mobile application and sends content such as photos, videos, and text.

[0772] Input: Photos, videos, and text data from users.

[0773] Output: The content sent to the server.

[0774] Specific actions: The user uses the app to enter photos and videos from the sports day, as well as their thoughts, and then taps the send button.

[0775] Step 2:

[0776] The server receives the content sent by the user.

[0777] Input: Digital content sent from a user device.

[0778] Output: Data stored in the server's storage.

[0779] Specific operation: The server receives a request from the user terminal and stores the received content in temporary storage.

[0780] Step 3:

[0781] The server analyzes the received content and extracts keywords and tags.

[0782] Input: Photos, videos, and text data stored on the server.

[0783] Output: A list of extracted keywords and tags.

[0784] Specific operation: The server analyzes the content using image recognition technology (e.g., OpenCV) and natural language processing tools (e.g., NLTK) and extracts important keywords and tags.

[0785] Step 4:

[0786] The server calls a generative AI model based on the analysis results and automatically generates news-style articles.

[0787] Input: Analysis results (list of keywords and tags), generative AI model.

[0788] Output: The generated news-style article.

[0789] How it works: The server uses the Hugging Face Transformers library to input the analysis results into a generative AI model and provide a prompt for article generation. It uses a prompt such as, "Today was a great sports day. It was impressive to see the children running around with such energy."

[0790] Step 5:

[0791] The server transmits the generated article to the user terminal as a preview.

[0792] Input: A generated news-style article.

[0793] Output: Article preview displayed on the user's device.

[0794] Specific operation: The server converts the article into HTML format and sends it to the user's device. The user can then preview the article through the app.

[0795] Step 6:

[0796] The user reviews the article and makes edits as necessary.

[0797] Input: The article preview shown to the user.

[0798] Output: The edited article.

[0799] What it does: Users can check the preview, correct typos, or enter additional information. Once they're done editing, they tap the save button.

[0800] Step 7:

[0801] The server delivers the final checked article to multiple recipients.

[0802] Input: The last article edited and reviewed by the user.

[0803] Output: The final article distributed to multiple recipients.

[0804] Specific operation: The server distributes the articles confirmed by the user via LINE or other SNS based on the specified recipient list.

[0805] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0806] Understood. Now, I will describe the "Form for carrying out the invention" based on the "Claims for the invention combining emotion engines."

[0807] This invention provides a system that generates more accurate and emotionally rich articles by combining analysis of content sent by users with an emotion engine that recognizes the user's emotions. This system allows users to effectively share their experiences and thoughts with others, stimulating communication.

[0808] System configuration

[0809] The system consists of the following main components:

[0810] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[0811] Server: The central system that receives, analyzes, stores, generates, and distributes content.

[0812] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data and sentiment data.

[0813] Sentiment Engine: A tool that analyzes and extracts sentiment information from user-submitted content.

[0814] User operation procedure

[0815] Users can send photos, videos, and text from a dedicated LINE account. For example, they can use the LINE chat function to send photos, videos, and comments about a sports day.

[0816] Server Processing

[0817] The server receives the content sent by the user, stores it in a buffer, and then analyzes it using image recognition technology and text analysis tools to extract important keywords and tags.

[0818] Emotion engine processing

[0819] The emotion engine recognizes emotions contained in the received content. For example, it analyzes the user's emotions such as joy, surprise, and sadness from the transmitted text and voice data. This generates emotion tags and assigns them to the content. These emotion tags are taken into account in the article generation process.

[0820] Processing of generated AI

[0821] The generative AI automatically generates articles based on the analyzed data and emotion tags. It creates captions for images and videos and constructs stories that reflect the emotion tags. For example, when generating a caption to express "the touching moment of coming second in a sports day," it reflects the user's emotion tag for joy.

[0822] Selecting a Media Format

[0823] The user can click on the preview link sent by the server to view the generated article, where they can choose from multiple media formats such as magazine, news, and album.

[0824] Article generation and distribution

[0825] The server generates the final article based on the media format selected by the user. This final article is saved on the server and sent back to the user as a preview link. The user can check the final preview and click the share button to distribute the article via LINE to the specified recipient list.

[0826] Specific examples

[0827] Example 1: Sharing your child's growth

[0828] 1. The user (parent) opens a dedicated LINE account and sends photos, videos, and impressions of the sports day.

[0829] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[0830] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[0831] 4. The generative AI generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[0832] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[0833] 6. After user confirms, distribute the article to relatives and friends.

[0834] Example 2: Amateur sports promotion

[0835] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[0836] 2. The server receives and analyzes this data.

[0837] 3. The emotion engine recognizes the user's emotions of "surprise" and "pride" from their post-match impressions and assigns emotion tags.

[0838] 4. The generative AI generates captions such as "The next generation star player's amazing play" and composes articles that reflect the sentiment tags.

[0839] 5. The user selects a news-like media format and the server sends the news-like article as the final preview.

[0840] 6. After the user has confirmed it, the article is distributed to fans and supporters.

[0841] In this way, the present invention provides a system that combines emotion engines to enable users to share their experiences and thoughts in a richer way and achieve more effective communication.

[0842] The processing flow will be explained below.

[0843] Understood. Now, we will explain each processing step based on the "Patent Claims for Inventions Combining Emotion Engines" in detail in the following format.

[0844] Step 1:

[0845] Users use a dedicated LINE account to send content such as photos, videos, and text. For example, a user can upload photos and videos from their child's sports day, or write their thoughts after the event, and click the send button.

[0846] Step 2:

[0847] The server receives the content sent by the user, and the received content is temporarily stored in a buffer.

[0848] Step 3:

[0849] The server analyzes the received content, specifically using image recognition technology to analyze the content of photos and videos, and using text analysis tools to extract keywords and tags from the text data.

[0850] Step 4:

[0851] The emotion engine recognizes the user's emotions from the received and analyzed text and voice data. For example, it recognizes the emotion of "joy" from the positive nuances in the text and generates an emotion tag.

[0852] Step 5:

[0853] The server stores the analyzed data and emotion tags in a database. The stored data includes user IDs, timestamps, keywords, tags, and emotion tags.

[0854] Step 6:

[0855] The server calls the generation AI, which automatically generates news-style articles using content data and emotion tags as input. The generation AI generates captions from images and videos and composes articles that reflect the emotion tags.

[0856] Step 7:

[0857] The AI ​​then creates an entire story based on the generated captions and emotion tags. For example, it incorporates an emotionally rich caption such as "The touching moment when we came in second place at the sports day."

[0858] Step 8:

[0859] The server sends the user a preview link of the generated article, which the user clicks to view the article.

[0860] Step 9:

[0861] The user selects the media format of the article on the preview screen, such as magazine style, news style, or album style.

[0862] Step 10:

[0863] The server generates the final article based on the media format selected by the user, and the final article is stored on the server.

[0864] Step 11:

[0865] The server sends a final preview link back to the user, who then checks the final preview to ensure the article is correct in terms of content and format.

[0866] Step 12:

[0867] The user clicks the distribution button, which causes the server to distribute the article to recipients via LINE based on the distribution list.

[0868] As a result, a system that combines an emotion engine can share users' experiences and thoughts more accurately and emotionally.

[0869] Example 2

[0870] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0871] Conventional content generation systems are limited to analyzing the content submitted by users and do not adequately consider emotional expression. As a result, the generated articles do not effectively reflect users' emotions and thoughts, resulting in insufficient content sharing and communication. To solve this problem, a system that analyzes emotions and reflects them in article generation is needed.

[0872] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0873] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for recognizing emotions from the received content and generating emotion tags, means for automatically generating news-style articles based on the analyzed data and emotion tags, means for selecting a media format for the generated article, and means for generating articles based on the selected media format and distributing them to multiple recipients. This enables article generation that reflects the user's emotions, enabling more effective content sharing and communication.

[0874] "User" means an individual or organization that utilizes the system to submit content, review generated articles, and distribute them.

[0875] "Content" refers to information such as photos, videos, and text that users submit to the system.

[0876] The "receiving means" is a mechanism by which the server receives the content sent by the user and temporarily stores it.

[0877] "Analysis means" refers to a system that analyzes received content using image recognition technology and text analysis tools to extract important keywords and tags.

[0878] The "emotion engine" is a mechanism that recognizes emotions within received content and generates emotion tags based on those emotions.

[0879] An "emotion tag" is an identifier that indicates the emotional information analyzed by the emotion engine and is used to generate articles.

[0880] "Generative AI" is artificial intelligence that automatically generates news-style articles based on analyzed data and emotion tags.

[0881] "Media format" refers to the layout and style in which an article is displayed.

[0882] The "distribution means" is a mechanism for transmitting the generated article to multiple recipients.

[0883] The "final confirmation means" is a mechanism by which a user can finally confirm the content of a generated article.

[0884] Okay, so based on the information we have so far, we'll create a detailed description.

[0885] This invention provides a system that generates more accurate and emotionally rich articles by combining analysis of content sent by users with an emotion engine that recognizes the user's emotions. This system allows users to effectively share their experiences and thoughts with others, stimulating communication.

[0886] System configuration

[0887] The system consists of the following main components:

[0888] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[0889] Server: The central system that receives, analyzes, stores, generates, and distributes content.

[0890] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data and sentiment data.

[0891] Sentiment Engine: A tool that analyzes and extracts sentiment information from user-submitted content.

[0892] User operation procedure

[0893] Users send photos, videos, and text from a dedicated messaging application. For example, when sending photos, videos, or impressions of a sports day, they use the chat function of the messaging application.

[0894] Server Processing

[0895] The server receives the content sent by the user. After receiving the content, it first temporarily stores it in a buffer. Then, the server analyzes the data using image recognition technology (e.g., a general image recognition API) and text analysis tools (e.g., a general text analysis API) to extract important keywords and tags.

[0896] Emotion engine processing

[0897] The emotion engine recognizes emotions contained in the received content. For example, it analyzes the user's emotions such as joy, surprise, and sadness from the transmitted text and voice data. This generates emotion tags and assigns them to the content. These emotion tags are taken into account in the article generation process.

[0898] Processing of generated AI

[0899] The generative AI automatically generates articles based on the analyzed data and emotion tags. It creates captions for images and videos and constructs stories that reflect the emotion tags. For example, when generating a caption to express "the touching moment of coming second in a sports day," it reflects the user's emotion tag of "joy."

[0900] Selecting a Media Format

[0901] The user can click on the preview link sent by the server to view the generated article, where they can choose from multiple media formats such as magazine, news, and album.

[0902] Article generation and distribution

[0903] The server generates the final article based on the media format selected by the user, saves the final article on the server, and sends it back to the user as a preview link. The user can view the final preview and click the distribute button, which distributes the article via a messaging application based on the specified recipient list.

[0904] Specific examples

[0905] Sharing your child's growth

[0906] 1. The user (parent) opens a dedicated messaging application and sends photos, videos, and impressions of the sports day.

[0907] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[0908] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[0909] 4. The generative AI generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[0910] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[0911] 6. After user confirms, distribute the article to relatives and friends.

[0912] Amateur sports promotion

[0913] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[0914] 2. The server receives and analyzes this data.

[0915] 3. The emotion engine recognizes the user's emotions of "surprise" and "pride" from their post-match impressions and assigns emotion tags.

[0916] 4. The generative AI generates captions such as "The next generation star player's amazing play" and composes articles that reflect the sentiment tags.

[0917] 5. The user selects a news-like media format and the server sends the news-like article as the final preview.

[0918] 6. After the user has confirmed it, the article is distributed to fans and supporters.

[0919] This system enables users to effectively share their experiences and thoughts, enabling richer communication.

[0920] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0921] I understand. Now, I will explain the program processing flow of this system by dividing it into specific steps.

[0922] Program processing flow

[0923] Step 1: User submits content

[0924] A user sends content such as photos, videos, and text using a dedicated messaging application.

[0925] Input: Photos, videos, and text data sent by users.

[0926] Output: The content data sent to the server.

[0927] Specific operation: The user opens a message sending application on a smartphone or PC, enters "Photos from the sports day," "Video from the sports day," and "I was so happy to come in second place at the sports day," and presses the send button.

[0928] Step 2: The server receives and stores the content

[0929] The server receives the content sent by the user and temporarily stores it in a buffer.

[0930] Input: Content data submitted by the user.

[0931] Output: The content data stored in the buffer.

[0932] Specific operation: The server separates the received photos, videos, and text into their respective data formats and temporarily stores them in buffer storage with a file name such as "userID_12345".

[0933] Step 3: The server parses the content

[0934] The server uses image recognition technology and text analysis tools to analyze the stored content and extract important keywords and tags.

[0935] Input: The content data stored in the buffer.

[0936] Output: Extracted keywords, tag list.

[0937] Specific operation: The server uses an image recognition API to extract keywords such as "sports day" and "podium" from the photo, and uses a text analysis API to extract keywords such as "second place" and "I was happy" from the text data.

[0938] Step 4: The emotion engine recognizes emotions and assigns tags

[0939] The emotion engine analyzes the emotions contained in the received content, generates emotion tags, and assigns them to the content.

[0940] Input: Extracted keywords, tag list.

[0941] Output: Emotion-tagged data.

[0942] Specific operation: The emotion engine uses a text analysis API to identify the user's emotion, such as "joy" or "surprise," generate tags based on that emotion, and assign the "joy" emotion tag to the text data.

[0943] Step 5: Generative AI generates articles

[0944] Generative AI automatically generates news-style articles based on the analyzed data and sentiment tags.

[0945] Input: Emotion-tagged data.

[0946] Output: The generated news-style article.

[0947] Specific operation: The generative AI (for example, GPT-3) generates captions such as "The touching moment when we came in second place at the sports day" and creates articles that reflect the emotion tag "joy."

[0948] Step 6: User selects media format

[0949] The user clicks on the preview link sent from the server to check the generated article and selects a media format such as magazine style, news style, or album style.

[0950] Input: A generated news-style article.

[0951] Output: Information for the selected media format.

[0952] What happens: The user clicks the preview link to see the generated article and selects the "magazine-style layout."

[0953] Step 7: The server generates the final article

[0954] The server generates the final article based on the media format selected by the user.

[0955] Input: Selected media format information, generated news format article.

[0956] Output: The final generated article.

[0957] What it does: The server uses a magazine-style layout template to position photos, captions, and text appropriately to generate the final article.

[0958] Step 8: Server sends final article

[0959] The server sends the final article to the user as a preview link.

[0960] Input: The final generated article.

[0961] Output: Preview link.

[0962] Specific operation: The server saves the generated final article in storage, generates a preview link, and sends it to the user.

[0963] Step 9: User publishes article

[0964] The user sees the final preview and distributes the article to multiple recipients.

[0965] Input: Preview link.

[0966] Output: A list of recipients to whom the article will be distributed.

[0967] Specific operation: The user checks the final preview and clicks the "Distribute" button to send the article to the specified recipients. The server then distributes the article via a messaging application such as LINE based on the recipient list.

[0968] Through the above processing steps, the system generates articles that more fully reflect the user's experiences and emotions, achieving effective communication.

[0969] (Application example 2)

[0970] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0971] In recent years, there has been a growing demand for emotionally rich communication with others via the Internet. However, conventional content generation systems have difficulty automatically generating articles that accurately reflect users' emotions, and as a result, users' experiences and thoughts cannot be fully shared. Furthermore, the generated articles tend to be flat and unattractive, limiting their impact on the recipient. There is a need for a system that can solve these issues and automatically generate and distribute compelling content that reflects users' emotions.

[0972] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, means for analyzing emotions in the content and assigning emotion tags, and means for generating articles that reflect the emotion tags. This makes it possible for users to share their experiences and thoughts with others in an emotionally rich way, providing an emotional impact on recipients.

[0973] "Content" is a general term for digital data such as photos, videos, and text sent by users.

[0974] An "emotion tag" is an information tag that is assigned based on the emotion analyzed from the content sent by the user.

[0975] "Generative AI" is artificial intelligence that automatically generates news-style articles and captions based on received data and emotional data.

[0976] The "emotion engine" is a tool that analyzes emotions within content submitted by users and extracts that information.

[0977] "Media format" refers to the display format and design of the generated article, and includes news format, magazine format, album format, and the like.

[0978] A "recipient list" is a set of designated recipients to whom a generated article is to be distributed.

[0979] "Keywords" are important words or phrases extracted by analyzing content.

[0980] A "tag" is classification information or identification information that is assigned to content.

[0981] "Stories" are emotionally rich, sequential articles and content created based on analyzed data and sentiment tags.

[0982] A "preview link" is a URL link provided to allow a user to see a preview of the generated article.

[0983] This invention is a system that analyzes content sent by users, recognizes their emotions, and automatically generates and distributes articles that reflect those emotions. Below, we will explain in detail the system and processing procedures for implementing this invention.

[0984] System configuration

[0985] User device:

[0986] Users use devices such as smartphones, tablets, and PCs to send content.

[0987] server:

[0988] This is the central system that receives, analyzes, stores, generates articles, and distributes content. The server uses the following software and libraries:

[0989] TextBlob: A library for analyzing text sentiment

[0990] Pillow (PIL): A library for image processing

[0991] Generative AI model: Artificial intelligence that generates articles based on content and sentiment data

[0992] Emotion Engine: A tool that analyzes emotions in user-submitted content

[0993] User operation procedure

[0994] Users can send photos, videos, and text from a dedicated smartphone app. For example, they can use the app's input interface to send photos, videos, and comments about a sports day.

[0995] Server Processing

[0996] The server receives the content sent by the user and first stores it in a temporary buffer. Next, it uses TextBlob to analyze the emotion of the received text data and generates emotion tags based on the results. It also uses Pillow to obtain basic information about the received image and video data.

[0997] Emotion engine processing

[0998] The emotion engine analyzes the user's emotions, such as joy, surprise, and sadness, from the received text and voice data, generates emotion tags, and assigns them to the content. These emotion tags are taken into account in the article generation process.

[0999] Processing of generated AI

[1000] The generative AI model automatically generates articles based on the analyzed data and sentiment tags, for example by adding captions to incoming photos and constructing stories that reflect the sentiment tags.

[1001] Preview and distribute content

[1002] The user can click the preview link sent by the server to check the generated article, where they can choose from multiple media formats such as news, magazine, album, etc. After final confirmation, the user clicks the distribution button, and the article is distributed to the specified recipient list.

[1003] Specific examples

[1004] Example 1: Sharing your child's progress

[1005] 1. The user (parent) opens a dedicated smartphone app and sends photos and videos of the sports day, along with their thoughts such as "Today was a fun sports day!"

[1006] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[1007] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[1008] 4. The generative AI model generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[1009] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[1010] 6. After user confirms, distribute the article to relatives and friends.

[1011] Prompt Sentence Examples

[1012] "We had a fun family sports day today! We took lots of photos and videos. Let us know what you think of it!"

[1013] In this way, this system allows users to share their experiences and thoughts with others in an emotionally rich way, realizing more effective communication.

[1014] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1015] Step 1:

[1016] The user launches the smartphone app, enters photos, videos, and text, and clicks the send button.

[1017] Input: photos, videos, text

[1018] Output: Transmitted data

[1019] Specific operation: The user enters, for example, photos and videos from the sports day and a comment such as "Today was a fun sports day!" into the smartphone app's input interface, and then presses the send button to send the data to the server.

[1020] Step 2:

[1021] The server receives the data sent by the user and temporarily stores it in a buffer.

[1022] Input: Send data

[1023] Output: Received data

[1024] Specific operation: The server detects the reception of transmitted data and temporarily stores it in buffer memory. Here, photos and videos are stored as binary data, and text is stored as character string data.

[1025] Step 3:

[1026] The server parses the incoming data and uses TextBlob to analyze the sentiment of the text and extract keywords and tags.

[1027] Input: Received data

[1028] Output: sentiment tags, keywords, tags

[1029] Specific operation: The server sends the saved string data to the TextBlob library for sentiment analysis. For example, from the text "Today was a fun sports day!", a "joy" tag is generated, and important keywords such as "sports day" and "fun" are extracted.

[1030] Step 4:

[1031] The server uses Pillow to obtain basic information about the received image and video data.

[1032] Input: Received data (images, videos)

[1033] Output: Basic information (image / video format, size, etc.)

[1034] What it does: The server opens the saved binary data using the Pillow library and retrieves basic information such as image format and size. It also analyzes video metadata.

[1035] Step 5:

[1036] The emotion engine analyzes emotions in the received data and assigns emotion tags.

[1037] Input: text data, image data, video data

[1038] Output: Emotion tag

[1039] Specific operation: The emotion engine in the server analyzes emotions from the entire received data. For example, it analyzes the content of a video and generates emotion tags such as "surprise" or "emotion" from specific scenes and assigns them to the data.

[1040] Step 6:

[1041] The generative AI model automatically generates articles based on the analyzed data and sentiment tags.

[1042] Input: emotion tags, keywords, basic information about images and videos

[1043] Output: Auto-generated article

[1044] Specific operation: The generative AI model on the server generates captions such as "The touching moment when we came in second place at the sports day" based on emotion tags and keywords, and creates an emotionally rich story-style article.

[1045] Step 7:

[1046] The server allows the user to select the media format of the generated article and then finalizes the article based on the selection.

[1047] Input: Auto-generated article

[1048] Output: Final generated article

[1049] Specific operation: The server sends the preview link to the user and lets them choose from media formats such as magazine format, news format, etc. Then it receives the selection results and constructs the final article.

[1050] Step 8:

[1051] The user checks the final generated article and clicks the distribution button.

[1052] Input: Final generated article

[1053] Output: Delivery instructions

[1054] Specific operation: The user clicks the preview link to check the final generated article, and then, if satisfied, clicks the publish button to instruct the article to be published.

[1055] Step 9:

[1056] The server distributes the final generated article based on a selected list of recipients.

[1057] Input: Delivery instructions, recipient list

[1058] Output: Delivery complete

[1059] Specific operation: The server receives distribution instructions and sends the article via email or social media based on the specified recipient list.

[1060] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1061] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1062] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1063] [Third embodiment]

[1064] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1065] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1066] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1067] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1068] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1069] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1070] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1071] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1072] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1073] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1074] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1075] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1076] Understood. Now, let's write the "Form for carrying out the invention."

[1077] The present invention provides a system that allows users to share their experiences and thoughts with others. This system receives and analyzes content sent by users, and generates and distributes articles based on that content, thereby stimulating communication.

[1078] System Overview

[1079] The system consists of the following main components:

[1080] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[1081] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[1082] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data.

[1083] User operation procedure

[1084] Users can use a dedicated LINE account to send content, such as photos, videos, and comments from a sports day, using the LINE chat function.

[1085] Server Processing

[1086] The server receives content sent from LINE accounts. After receiving the data, the server analyzes it and extracts important keywords and tags. It uses image recognition technology to understand the content of photos and videos, and natural language processing tools to analyze text data. It then invokes a generation AI based on the analysis results to automatically generate news-style articles.

[1087] Processing of generated AI

[1088] The generative AI automatically generates articles based on the received data and analysis results. First, it generates appropriate captions from images and videos, and then it generates the sentences that make up the overall story. In this process, it takes into account keywords and tags to create articles that reflect the user's intentions.

[1089] Selecting a Media Format

[1090] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[1091] Article generation and distribution

[1092] The server generates the final article based on the media format selected by the user. The generated article is sent to the user as a final preview. After the user confirms it, they click the distribution button, and the server distributes the article via LINE based on the specified recipient list.

[1093] Specific examples

[1094] Example 1: Sharing your child's growth

[1095] 1. The user (parent) opens a dedicated LINE account and sends photos, videos, and impressions of the sports day.

[1096] 2. The server receives this data, checks the format and size of the images and videos, and analyzes the text.

[1097] 3. The AI ​​generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[1098] 4. The user selects the magazine-style media format, and the server sends the magazine-style article as the final preview.

[1099] 5. After user confirms, distribute the article to relatives and friends.

[1100] Example 2: Amateur sports promotion

[1101] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[1102] 2. The server receives and analyzes this data.

[1103] 3. Based on the content of the video, the generative AI generates a caption such as "The next generation star player's amazing play," and assembles the whole video into an article.

[1104] 4. The user selects a news-like media format, and the server sends the news-like article as the final preview.

[1105] 5. After the user has confirmed it, the article is distributed to fans and supporters.

[1106] Through the above-described embodiments, the present invention provides specific means for effectively sharing users' experiences and thoughts and for activating communication.

[1107] The processing flow will be explained below.

[1108] Okay, now I'll explain the process step by step as you instructed.

[1109] Step 1:

[1110] The user opens a dedicated LINE account, where they are authenticated to use the system.

[1111] Step 2:

[1112] A user sends content (photos, videos, text) using the LINE chat function. At this time, the user checks the content to be sent and clicks the send button at the appropriate time.

[1113] Step 3:

[1114] The server receives the content from the LINE account and temporarily stores the received data in a buffer.

[1115] Step 4:

[1116] The server analyzes the received content, using image recognition technology to extract important elements in the case of images, and text analysis tools to extract keywords and tags from the text.

[1117] Step 5:

[1118] The server stores the parsed data in a database, including the user ID, timestamp, and extracted keywords and tags.

[1119] Step 6:

[1120] The server calls the AI ​​generator, which uses the saved data as input to automatically generate news-style articles. The AI ​​analyzes the content of photos and videos and generates appropriate captions.

[1121] Step 7:

[1122] The generative AI generates captions and stories and determines the overall structure. For example, it creates a caption such as "The moving moment when we came in second place at the sports day" and reflects it in the article.

[1123] Step 8:

[1124] The server sends a preview link of the generated article to the user, who clicks the preview link to check the content of the article.

[1125] Step 9:

[1126] The user selects the media format of the article on the preview screen. Options include magazine, news, album, etc.

[1127] Step 10:

[1128] The server generates the final article based on the media format selected by the user, and the final article is stored on the server.

[1129] Step 11:

[1130] The server sends the final preview link back to the user, who then checks the final preview and verifies that there are no problems with the content or format of the article.

[1131] Step 12:

[1132] The user clicks the distribution button, which causes the server to distribute the article to recipients via LINE based on the distribution list.

[1133] Through each of the above steps, this system completes the process of effectively generating and sharing user experiences and thoughts as articles.

[1134] Example 1

[1135] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1136] Conventional content sharing systems have limited the means by which users can effectively share their experiences and thoughts with others, often requiring them to manually create articles. This places a heavy burden on users and limits the quality and quantity of information they want to share. It has also been difficult to generate articles compatible with multiple media formats and to effectively distribute them to recipients. To solve these issues, a system that can automatically analyze user experiences and share them effectively is needed.

[1137] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1138] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, means for performing text analysis using a natural language processing tool, means for analyzing photos and videos using image recognition technology, means for automatically generating articles using an artificial intelligence model, and means for sending the results of the data analysis to the artificial intelligence as prompt sentences. This allows the user to check high-quality articles automatically generated based on the sent content in various media formats and effectively distribute them to multiple recipients.

[1139] "User terminal" refers to an electronic device used by a user to transmit content, including a smartphone, tablet, personal computer, etc.

[1140] "Server" refers to the central system that receives, analyzes, and stores content sent by users, and automatically generates and distributes articles.

[1141] A "generative AI model" refers to an artificial intelligence model that automatically generates articles based on received data and analysis results, and uses natural language generation technology.

[1142] A "prompt sentence" refers to a sentence that is input into a generative AI model based on the results of data analysis, and includes instructions for generating appropriate captions and stories.

[1143] The "means for receiving" refers to a mechanism by which the server receives multiple contents sent by the user.

[1144] "Means for analyzing" refers to the processes and techniques used to analyze received content and extract significant keywords and tags.

[1145] "Means for automatically generating news-style articles" refers to systems and technologies for automatically creating news-style text based on analyzed data.

[1146] "Means for selecting media format" refers to a mechanism for choosing from multiple formats for the appearance and layout of the generated article.

[1147] "Means for distribution to multiple recipients" refers to a mechanism by which the server transmits the generated article to multiple designated recipients.

[1148] "Natural language processing tools" refers to software and technology for analyzing text data and extracting important keywords and tags.

[1149] "Image recognition technology" refers to technology that analyzes the content of photos and videos and recognizes objects and scenes.

[1150] "Means for automatically generating articles using an artificial intelligence model" refers to a mechanism that uses a generative AI model to automatically create articles based on received and analyzed data.

[1151] "Means for transmitting data analysis results to artificial intelligence as prompt sentences" refers to the processes and techniques for generating the analyzed data as prompt sentences and passing them to an artificial intelligence model.

[1152] The "means for final confirmation" refers to a mechanism by which a user can confirm, edit, or approve the generated article.

[1153] The "means for sending a final preview to a user" refers to a mechanism for sending a preview of the generated article to a user for confirmation.

[1154] The present invention provides a system that allows users to share their experiences and thoughts with others. This system receives and analyzes content sent by users, and generates and distributes articles based on that content, thereby stimulating communication.

[1155] System Overview

[1156] The system consists of the following main components:

[1157] User terminal: A device for transmitting content (such as a smartphone, tablet, or personal computer).

[1158] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[1159] Generative AI model: An artificial intelligence model that automatically generates news-style articles based on the data it receives.

[1160] User operations

[1161] Users send content using dedicated messaging applications. For example, users send photos, videos, and impressions of a sports day using the chat function of the messaging application.

[1162] Server Processing

[1163] The server receives the content (photos, videos, text) sent by the user using the messaging API. After receiving the content, the server performs the following actions:

[1164] 1. Data Analysis: The server analyzes the data sent and extracts important keywords and tags. Specifically, it uses the following techniques:

[1165] Image analysis: The content of the photos and videos sent is analyzed using image recognition technology. Specifically, general image recognition technology is used.

[1166] Text analysis: Use natural language processing tools to analyze the text submitted by the user, such as spaCy or a natural language processing API.

[1167] 2. Calling a generative AI model: Based on the analysis results, a generative AI model (such as a common generative AI model like GPT-4) is called to automatically generate a news-style article.

[1168] Prompt creation: The server compiles the results of image analysis and text analysis into a prompt. Examples of specific prompts are as follows:

[1169] Photo caption: "The touching moment when I came second at the sports day."

[1170] Testimonial: "The children did a great job in beautiful weather. We were all particularly impressed by the high jump."

[1171] Media Format Selection

[1172] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[1173] Article generation and distribution

[1174] The server generates the final article based on the media format selected by the user, and sends the article to the user as a final preview. After the user gives their final confirmation, they click the distribute button, and the server sends the article to multiple specified recipients.

[1175] Specific examples

[1176] Example 1: Sharing your child's growth

[1177] 1. The user opens a dedicated messaging application and sends photos, videos, and impressions of the sports day.

[1178] 2. The server receives this data using a messaging API.

[1179] 3. The server uses image recognition technology to analyze the content of the photo and obtain the information that the photo "came in second place." At the same time, it uses natural language processing tools to analyze the comments and extract important keywords (such as "great weather" and "the kids worked hard").

[1180] 4. The server sends prompts based on the analysis results to the generative AI model.

[1181] 5. The generative AI model generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[1182] 6. The user clicks on the preview link sent by the server and selects the magazine-style media format.

[1183] 7. The server generates a magazine-style article and sends it to the user as a final preview.

[1184] 8. The user checks the preview and clicks the share button, and the server distributes the article to the specified relatives and friends.

[1185] In this way, the present invention provides a concrete means for effectively sharing users' experiences and thoughts and stimulating communication.

[1186] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1187] Step 1: Submit your content

[1188] Users use a dedicated messaging application to send content such as photos, videos, and text. Specifically, users select a photo or video on the messaging application and enter their thoughts and descriptions in text. This content is then sent to the server via the messaging API.

[1189] Input: Photos, videos, text

[1190] Output: Data sent from the user to the server

[1191] Step 2: Receiving content

[1192] The server receives the content sent by the user using the messaging API, and stores it in a database for further analysis.

[1193] Input: User-submitted content (photos, videos, text)

[1194] Output: Content received and stored in a database

[1195] Step 3: Prepare for data analysis

[1196] The server prepares the received content for analysis by placing the data in a queue for passing to the analysis block.

[1197] Input: Content stored in the database

[1198] Output: Content queued for analysis

[1199] Step 4: Data analysis

[1200] The server analyzes the transmitted content in the following way:

[1201] 1. Image analysis: The server uses image recognition technology to analyze the content of the photos and videos you send. For example, it uses common image recognition technology to recognize objects and scenes in the images and gather information for caption generation.

[1202] Input: Image or video in the content

[1203] Output: Information about objects and scenes in the image

[1204] 2. Text Analysis: The server uses natural language processing tools to analyze the text submitted by the user, such as "spaCy" or "a type of natural language processing API," to extract important keywords and tags.

[1205] Input: The text submitted by the user

[1206] Output: Important keywords, tags, and sentiment analysis results

[1207] Step 5: Invoke the generative AI model

[1208] Based on the analysis results, the server calls a generative AI model (such as a common generative AI model like "GPT-4") to automatically generate news-style articles.

[1209] Input: Image analysis results, text analysis results

[1210] Output: Prompt sentence to be passed to the generative AI model

[1211] Specific behavior: The server generates a prompt based on the parsed result:

[1212] Photo caption: "The touching moment when I came second at the sports day."

[1213] Testimonial: "The children did a great job in beautiful weather. We were all particularly impressed by the high jump."

[1214] Step 6: Auto-generating articles

[1215] The generative AI model generates news-style articles based on prompts sent from the server, creates appropriate captions based on the input prompts, and builds the entire story.

[1216] Input: prompt statement

[1217] Output: Auto-generated article

[1218] What it does: A generative AI model generates news articles, including photo captions and the overall story.

[1219] Step 7: Select the media format

[1220] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[1221] Input: Preview link, generated article

[1222] Output: Selected media format

[1223] What happens: The user clicks on the preview link you sent them and selects the media format they want.

[1224] Step 8: Final article generation and confirmation

[1225] The server generates the final article based on the media format selected by the user, and sends the generated article to the user as a final preview.

[1226] Input: Selected media format, generated article

[1227] Output: Final preview article

[1228] What happens: The server formats the article using the selected media format and sends it to the user as a final preview.

[1229] Step 9: Article Distribution

[1230] After the user checks the final preview, they click the distribute button, and the server sends the article to multiple specified recipients.

[1231] Input: Final preview article, recipient list

[1232] Output: Delivery completion notification to the recipient

[1233] What happens: The user clicks the distribute button, and the server distributes the article to the specified recipient list.

[1234] (Application example 1)

[1235] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1236] In systems that allow users to effectively share their experiences and thoughts with others, there are problems such as the complexity of content analysis and article generation, and the inability of users to edit or preview articles, which increases the possibility of articles being distributed that do not match the user's intentions.

[1237] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1238] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, and means for previewing and editing the generated articles. This allows users to edit articles in a way that suits their own intentions and distribute them after final confirmation.

[1239] "User" means an individual or organization that uses the system to submit content to share their experiences and thoughts with others.

[1240] "Content" is a general term for digital data such as photos, videos, and text sent by users.

[1241] "Means for receiving" refers to the method or technology by which a central system such as a server receives the transmitted content.

[1242] "Means for analysis" refers to methods and technologies for understanding the content of received content and extracting important keywords and tags.

[1243] A "keyword" is a word or phrase that indicates important information about the content and characterizes the content.

[1244] A "tag" is a label that indicates the attributes or category of content, and is information that makes it easier for users to organize and search for content.

[1245] A "news-style article" is a document that describes a user's experience in the form of news, including automatically generated content.

[1246] "Automatic generation means" refers to the software and algorithms used to mechanically generate articles based on the analysis results.

[1247] The "media format" refers to the format or style for displaying the generated article, and includes news style, magazine style, album style, and the like.

[1248] A "production method" is a method or technique for producing an article in a selected media format.

[1249] A "delivery means" is a method or technique for sending the final generated article to the intended recipient.

[1250] A "previewing means" is a method or technique for displaying the generated article for final user confirmation.

[1251] "Editing means" refers to methods or techniques that allow users to make changes to generated articles.

[1252] A specific system architecture and its operation for implementing the present invention will now be described.

[1253] System Overview

[1254] The system of the present invention consists of the following main components:

[1255] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[1256] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[1257] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data.

[1258] Communication infrastructure: The Internet that connects user terminals and servers.

[1259] User terminal operation procedure

[1260] Users use a dedicated mobile application to send content, such as photos, videos, and impressions of sports days, using the app's in-app features.

[1261] Server Processing

[1262] The server receives the content sent from the user terminal. The received content is processed as follows:

[1263] 1. Data reception: The server receives the photos, videos, and text data sent by the user.

[1264] 2. Data Analysis: Analyzes the received data and extracts important keywords and tags. Using image recognition technology and natural language processing tools, the content of photos and videos is analyzed in detail.

[1265] 3. Generative AI call: Based on the analysis results, a generative AI model is called to automatically generate news-style articles. Libraries such as Hugging Face Transformers are used.

[1266] Article Generation Process

[1267] The generative AI automatically generates articles based on the received data and analysis results. Specifically, it generates appropriate captions from images and videos, and then generates the sentences that make up the overall story.

[1268] Select and preview media formats

[1269] The generated article is sent as a preview from the server to the user's device. The user can select from multiple media formats, such as news, magazine, and album, and edit the article as needed.

[1270] delivery

[1271] After the user has given their final confirmation, the article is distributed to multiple recipients via LINE or other social media platforms.

[1272] Specific examples

[1273] Example 1: Sharing your child's growth

[1274] 1. The user (parent) opens the dedicated app and sends photos, videos, and impressions of the sports day.

[1275] 2. The server receives this data, checks the format and size of the images and videos, and analyzes the text.

[1276] 3. The AI ​​generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[1277] 4. The user selects the magazine-style media format, and the server sends the magazine-style article as the final preview.

[1278] 5. After user confirms, distribute the article to relatives and friends.

[1279] Example 2: Amateur sports promotion

[1280] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[1281] 2. The server receives and analyzes this data.

[1282] 3. Based on the content of the video, the generative AI generates a caption such as "The next generation star player's amazing play," and assembles the whole video into an article.

[1283] 4. The user selects a news-like media format, and the server sends the news-like article as the final preview.

[1284] 5. After the user has confirmed it, the article is distributed to fans and supporters.

[1285] Prompt Sentence Examples

[1286] "Upload five photos from the sports day and enter the following comment: 'Today was a great sports day. It was impressive to see the kids running around with such energy.'"

[1287] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1288] Step 1:

[1289] The user launches a dedicated mobile application and sends content such as photos, videos, and text.

[1290] Input: Photos, videos, and text data from users.

[1291] Output: The content sent to the server.

[1292] Specific actions: The user uses the app to enter photos and videos from the sports day, as well as their thoughts, and then taps the send button.

[1293] Step 2:

[1294] The server receives the content sent by the user.

[1295] Input: Digital content sent from a user device.

[1296] Output: Data stored in the server's storage.

[1297] Specific operation: The server receives a request from the user terminal and stores the received content in temporary storage.

[1298] Step 3:

[1299] The server analyzes the received content and extracts keywords and tags.

[1300] Input: Photos, videos, and text data stored on the server.

[1301] Output: A list of extracted keywords and tags.

[1302] Specific operation: The server analyzes the content using image recognition technology (e.g., OpenCV) and natural language processing tools (e.g., NLTK) and extracts important keywords and tags.

[1303] Step 4:

[1304] The server calls a generative AI model based on the analysis results and automatically generates news-style articles.

[1305] Input: Analysis results (list of keywords and tags), generative AI model.

[1306] Output: The generated news-style article.

[1307] How it works: The server uses the Hugging Face Transformers library to input the analysis results into a generative AI model and provide a prompt for article generation. It uses a prompt such as, "Today was a great sports day. It was impressive to see the children running around with such energy."

[1308] Step 5:

[1309] The server transmits the generated article to the user terminal as a preview.

[1310] Input: A generated news-style article.

[1311] Output: Article preview displayed on the user's device.

[1312] Specific operation: The server converts the article into HTML format and sends it to the user's device. The user can then preview the article through the app.

[1313] Step 6:

[1314] The user reviews the article and makes edits as necessary.

[1315] Input: The article preview shown to the user.

[1316] Output: The edited article.

[1317] What it does: Users can check the preview, correct typos, or enter additional information. Once they're done editing, they tap the save button.

[1318] Step 7:

[1319] The server delivers the final checked article to multiple recipients.

[1320] Input: The last article edited and reviewed by the user.

[1321] Output: The final article distributed to multiple recipients.

[1322] Specific operation: The server distributes the articles confirmed by the user via LINE or other SNS based on the specified recipient list.

[1323] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1324] Understood. Now, I will describe the "Form for carrying out the invention" based on the "Claims for the invention combining emotion engines."

[1325] This invention provides a system that generates more accurate and emotionally rich articles by combining analysis of content sent by users with an emotion engine that recognizes the user's emotions. This system allows users to effectively share their experiences and thoughts with others, stimulating communication.

[1326] System configuration

[1327] The system consists of the following main components:

[1328] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[1329] Server: The central system that receives, analyzes, stores, generates, and distributes content.

[1330] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data and sentiment data.

[1331] Sentiment Engine: A tool that analyzes and extracts sentiment information from user-submitted content.

[1332] User operation procedure

[1333] Users can send photos, videos, and text from a dedicated LINE account. For example, they can use the LINE chat function to send photos, videos, and comments about a sports day.

[1334] Server Processing

[1335] The server receives the content sent by the user, stores it in a buffer, and then analyzes it using image recognition technology and text analysis tools to extract important keywords and tags.

[1336] Emotion engine processing

[1337] The emotion engine recognizes emotions contained in the received content. For example, it analyzes the user's emotions such as joy, surprise, and sadness from the transmitted text and voice data. This generates emotion tags and assigns them to the content. These emotion tags are taken into account in the article generation process.

[1338] Processing of generated AI

[1339] The generative AI automatically generates articles based on the analyzed data and emotion tags. It creates captions for images and videos and constructs stories that reflect the emotion tags. For example, when generating a caption to express "the touching moment of coming second in a sports day," it reflects the user's emotion tag for joy.

[1340] Selecting a Media Format

[1341] The user can click on the preview link sent by the server to view the generated article, where they can choose from multiple media formats such as magazine, news, and album.

[1342] Article generation and distribution

[1343] The server generates the final article based on the media format selected by the user. This final article is saved on the server and sent back to the user as a preview link. The user can check the final preview and click the share button to distribute the article via LINE to the specified recipient list.

[1344] Specific examples

[1345] Example 1: Sharing your child's growth

[1346] 1. The user (parent) opens a dedicated LINE account and sends photos, videos, and impressions of the sports day.

[1347] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[1348] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[1349] 4. The generative AI generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[1350] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[1351] 6. After user confirms, distribute the article to relatives and friends.

[1352] Example 2: Amateur sports promotion

[1353] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[1354] 2. The server receives and analyzes this data.

[1355] 3. The emotion engine recognizes the user's emotions of "surprise" and "pride" from their post-match impressions and assigns emotion tags.

[1356] 4. The generative AI generates captions such as "The next generation star player's amazing play" and composes articles that reflect the sentiment tags.

[1357] 5. The user selects a news-like media format and the server sends the news-like article as the final preview.

[1358] 6. After the user has confirmed it, the article is distributed to fans and supporters.

[1359] In this way, the present invention provides a system that combines emotion engines to enable users to share their experiences and thoughts in a richer way and achieve more effective communication.

[1360] The processing flow will be explained below.

[1361] Understood. Now, we will explain each processing step based on the "Patent Claims for Inventions Combining Emotion Engines" in detail in the following format.

[1362] Step 1:

[1363] Users use a dedicated LINE account to send content such as photos, videos, and text. For example, a user can upload photos and videos from their child's sports day, or write their thoughts after the event, and click the send button.

[1364] Step 2:

[1365] The server receives the content sent by the user, and the received content is temporarily stored in a buffer.

[1366] Step 3:

[1367] The server analyzes the received content, specifically using image recognition technology to analyze the content of photos and videos, and using text analysis tools to extract keywords and tags from the text data.

[1368] Step 4:

[1369] The emotion engine recognizes the user's emotions from the received and analyzed text and voice data. For example, it recognizes the emotion of "joy" from the positive nuances in the text and generates an emotion tag.

[1370] Step 5:

[1371] The server stores the analyzed data and emotion tags in a database. The stored data includes user IDs, timestamps, keywords, tags, and emotion tags.

[1372] Step 6:

[1373] The server calls the generation AI, which automatically generates news-style articles using content data and emotion tags as input. The generation AI generates captions from images and videos and composes articles that reflect the emotion tags.

[1374] Step 7:

[1375] The AI ​​then creates an entire story based on the generated captions and emotion tags. For example, it incorporates an emotionally rich caption such as "The touching moment when we came in second place at the sports day."

[1376] Step 8:

[1377] The server sends the user a preview link of the generated article, which the user clicks to view the article.

[1378] Step 9:

[1379] The user selects the media format of the article on the preview screen, such as magazine style, news style, or album style.

[1380] Step 10:

[1381] The server generates the final article based on the media format selected by the user, and the final article is stored on the server.

[1382] Step 11:

[1383] The server sends a final preview link back to the user, who then checks the final preview to ensure the article is correct in terms of content and format.

[1384] Step 12:

[1385] The user clicks the distribution button, which causes the server to distribute the article to recipients via LINE based on the distribution list.

[1386] As a result, a system that combines an emotion engine can share users' experiences and thoughts more accurately and emotionally.

[1387] Example 2

[1388] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1389] Conventional content generation systems are limited to analyzing the content submitted by users and do not adequately consider emotional expression. As a result, the generated articles do not effectively reflect users' emotions and thoughts, resulting in insufficient content sharing and communication. To solve this problem, a system that analyzes emotions and reflects them in article generation is needed.

[1390] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1391] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for recognizing emotions from the received content and generating emotion tags, means for automatically generating news-style articles based on the analyzed data and emotion tags, means for selecting a media format for the generated article, and means for generating articles based on the selected media format and distributing them to multiple recipients. This enables article generation that reflects the user's emotions, enabling more effective content sharing and communication.

[1392] "User" means an individual or organization that utilizes the system to submit content, review generated articles, and distribute them.

[1393] "Content" refers to information such as photos, videos, and text that users submit to the system.

[1394] The "receiving means" is a mechanism by which the server receives the content sent by the user and temporarily stores it.

[1395] "Analysis means" refers to a system that analyzes received content using image recognition technology and text analysis tools to extract important keywords and tags.

[1396] The "emotion engine" is a mechanism that recognizes emotions within received content and generates emotion tags based on those emotions.

[1397] An "emotion tag" is an identifier that indicates the emotional information analyzed by the emotion engine and is used to generate articles.

[1398] "Generative AI" is artificial intelligence that automatically generates news-style articles based on analyzed data and emotion tags.

[1399] "Media format" refers to the layout and style in which an article is displayed.

[1400] The "distribution means" is a mechanism for transmitting the generated article to multiple recipients.

[1401] The "final confirmation means" is a mechanism by which a user can finally confirm the content of a generated article.

[1402] Okay, so based on the information we have so far, we'll create a detailed description.

[1403] This invention provides a system that generates more accurate and emotionally rich articles by combining analysis of content sent by users with an emotion engine that recognizes the user's emotions. This system allows users to effectively share their experiences and thoughts with others, stimulating communication.

[1404] System configuration

[1405] The system consists of the following main components:

[1406] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[1407] Server: The central system that receives, analyzes, stores, generates, and distributes content.

[1408] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data and sentiment data.

[1409] Sentiment Engine: A tool that analyzes and extracts sentiment information from user-submitted content.

[1410] User operation procedure

[1411] Users send photos, videos, and text from a dedicated messaging application. For example, when sending photos, videos, or impressions of a sports day, they use the chat function of the messaging application.

[1412] Server Processing

[1413] The server receives the content sent by the user. After receiving the content, it first temporarily stores it in a buffer. Then, the server analyzes the data using image recognition technology (e.g., a general image recognition API) and text analysis tools (e.g., a general text analysis API) to extract important keywords and tags.

[1414] Emotion engine processing

[1415] The emotion engine recognizes emotions contained in the received content. For example, it analyzes the user's emotions such as joy, surprise, and sadness from the transmitted text and voice data. This generates emotion tags and assigns them to the content. These emotion tags are taken into account in the article generation process.

[1416] Processing of generated AI

[1417] The generative AI automatically generates articles based on the analyzed data and emotion tags. It creates captions for images and videos and constructs stories that reflect the emotion tags. For example, when generating a caption to express "the touching moment of coming second in a sports day," it reflects the user's emotion tag of "joy."

[1418] Selecting a Media Format

[1419] The user can click on the preview link sent by the server to view the generated article, where they can choose from multiple media formats such as magazine, news, and album.

[1420] Article generation and distribution

[1421] The server generates the final article based on the media format selected by the user, saves the final article on the server, and sends it back to the user as a preview link. The user can view the final preview and click the distribute button, which distributes the article via a messaging application based on the specified recipient list.

[1422] Specific examples

[1423] Sharing your child's growth

[1424] 1. The user (parent) opens a dedicated messaging application and sends photos, videos, and impressions of the sports day.

[1425] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[1426] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[1427] 4. The generative AI generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[1428] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[1429] 6. After user confirms, distribute the article to relatives and friends.

[1430] Amateur sports promotion

[1431] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[1432] 2. The server receives and analyzes this data.

[1433] 3. The emotion engine recognizes the user's emotions of "surprise" and "pride" from their post-match impressions and assigns emotion tags.

[1434] 4. The generative AI generates captions such as "The next generation star player's amazing play" and composes articles that reflect the sentiment tags.

[1435] 5. The user selects a news-like media format and the server sends the news-like article as the final preview.

[1436] 6. After the user has confirmed it, the article is distributed to fans and supporters.

[1437] This system enables users to effectively share their experiences and thoughts, enabling richer communication.

[1438] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1439] I understand. Now, I will explain the program processing flow of this system by dividing it into specific steps.

[1440] Program processing flow

[1441] Step 1: User submits content

[1442] A user sends content such as photos, videos, and text using a dedicated messaging application.

[1443] Input: Photos, videos, and text data sent by users.

[1444] Output: The content data sent to the server.

[1445] Specific operation: The user opens a message sending application on a smartphone or PC, enters "Photos from the sports day," "Video from the sports day," and "I was so happy to come in second place at the sports day," and presses the send button.

[1446] Step 2: The server receives and stores the content

[1447] The server receives the content sent by the user and temporarily stores it in a buffer.

[1448] Input: Content data submitted by the user.

[1449] Output: The content data stored in the buffer.

[1450] Specific operation: The server separates the received photos, videos, and text into their respective data formats and temporarily stores them in buffer storage with a file name such as "userID_12345".

[1451] Step 3: The server parses the content

[1452] The server uses image recognition technology and text analysis tools to analyze the stored content and extract important keywords and tags.

[1453] Input: The content data stored in the buffer.

[1454] Output: Extracted keywords, tag list.

[1455] Specific operation: The server uses an image recognition API to extract keywords such as "sports day" and "podium" from the photo, and uses a text analysis API to extract keywords such as "second place" and "I was happy" from the text data.

[1456] Step 4: The emotion engine recognizes emotions and assigns tags

[1457] The emotion engine analyzes the emotions contained in the received content, generates emotion tags, and assigns them to the content.

[1458] Input: Extracted keywords, tag list.

[1459] Output: Emotion-tagged data.

[1460] Specific operation: The emotion engine uses a text analysis API to identify the user's emotion, such as "joy" or "surprise," generate tags based on that emotion, and assign the "joy" emotion tag to the text data.

[1461] Step 5: Generative AI generates articles

[1462] Generative AI automatically generates news-style articles based on the analyzed data and sentiment tags.

[1463] Input: Emotion-tagged data.

[1464] Output: The generated news-style article.

[1465] Specific operation: The generative AI (for example, GPT-3) generates captions such as "The touching moment when we came in second place at the sports day" and creates articles that reflect the emotion tag "joy."

[1466] Step 6: User selects media format

[1467] The user clicks on the preview link sent from the server to check the generated article and selects a media format such as magazine style, news style, or album style.

[1468] Input: A generated news-style article.

[1469] Output: Information for the selected media format.

[1470] What happens: The user clicks the preview link to see the generated article and selects the "magazine-style layout."

[1471] Step 7: The server generates the final article

[1472] The server generates the final article based on the media format selected by the user.

[1473] Input: Selected media format information, generated news format article.

[1474] Output: The final generated article.

[1475] What it does: The server uses a magazine-style layout template to position photos, captions, and text appropriately to generate the final article.

[1476] Step 8: Server sends final article

[1477] The server sends the final article to the user as a preview link.

[1478] Input: The final generated article.

[1479] Output: Preview link.

[1480] Specific operation: The server saves the generated final article in storage, generates a preview link, and sends it to the user.

[1481] Step 9: User publishes article

[1482] The user sees the final preview and distributes the article to multiple recipients.

[1483] Input: Preview link.

[1484] Output: A list of recipients to whom the article will be distributed.

[1485] Specific operation: The user checks the final preview and clicks the "Distribute" button to send the article to the specified recipients. The server then distributes the article via a messaging application such as LINE based on the recipient list.

[1486] Through the above processing steps, the system generates articles that more fully reflect the user's experiences and emotions, achieving effective communication.

[1487] (Application example 2)

[1488] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1489] In recent years, there has been a growing demand for emotionally rich communication with others via the Internet. However, conventional content generation systems have difficulty automatically generating articles that accurately reflect users' emotions, and as a result, users' experiences and thoughts cannot be fully shared. Furthermore, the generated articles tend to be flat and unattractive, limiting their impact on the recipient. There is a need for a system that can solve these issues and automatically generate and distribute compelling content that reflects users' emotions.

[1490] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, means for analyzing emotions in the content and assigning emotion tags, and means for generating articles that reflect the emotion tags. This makes it possible for users to share their experiences and thoughts with others in an emotionally rich way, providing an emotional impact on recipients.

[1491] "Content" is a general term for digital data such as photos, videos, and text sent by users.

[1492] An "emotion tag" is an information tag that is assigned based on the emotion analyzed from the content sent by the user.

[1493] "Generative AI" is artificial intelligence that automatically generates news-style articles and captions based on received data and emotional data.

[1494] The "emotion engine" is a tool that analyzes emotions within content submitted by users and extracts that information.

[1495] "Media format" refers to the display format and design of the generated article, and includes news format, magazine format, album format, and the like.

[1496] A "recipient list" is a set of designated recipients to whom a generated article is to be distributed.

[1497] "Keywords" are important words or phrases extracted by analyzing content.

[1498] A "tag" is classification information or identification information that is assigned to content.

[1499] "Stories" are emotionally rich, sequential articles and content created based on analyzed data and sentiment tags.

[1500] A "preview link" is a URL link provided to allow a user to see a preview of the generated article.

[1501] This invention is a system that analyzes content sent by users, recognizes their emotions, and automatically generates and distributes articles that reflect those emotions. Below, we will explain in detail the system and processing procedures for implementing this invention.

[1502] System configuration

[1503] User device:

[1504] Users use devices such as smartphones, tablets, and PCs to send content.

[1505] server:

[1506] This is the central system that receives, analyzes, stores, generates articles, and distributes content. The server uses the following software and libraries:

[1507] TextBlob: A library for analyzing text sentiment

[1508] Pillow (PIL): A library for image processing

[1509] Generative AI model: Artificial intelligence that generates articles based on content and sentiment data

[1510] Emotion Engine: A tool that analyzes emotions in user-submitted content

[1511] User operation procedure

[1512] Users can send photos, videos, and text from a dedicated smartphone app. For example, they can use the app's input interface to send photos, videos, and comments about a sports day.

[1513] Server Processing

[1514] The server receives the content sent by the user and first stores it in a temporary buffer. Next, it uses TextBlob to analyze the emotion of the received text data and generates emotion tags based on the results. It also uses Pillow to obtain basic information about the received image and video data.

[1515] Emotion engine processing

[1516] The emotion engine analyzes the user's emotions, such as joy, surprise, and sadness, from the received text and voice data, generates emotion tags, and assigns them to the content. These emotion tags are taken into account in the article generation process.

[1517] Processing of generated AI

[1518] The generative AI model automatically generates articles based on the analyzed data and sentiment tags, for example by adding captions to incoming photos and constructing stories that reflect the sentiment tags.

[1519] Preview and distribute content

[1520] The user can click the preview link sent by the server to check the generated article, where they can choose from multiple media formats such as news, magazine, album, etc. After final confirmation, the user clicks the distribution button, and the article is distributed to the specified recipient list.

[1521] Specific examples

[1522] Example 1: Sharing your child's progress

[1523] 1. The user (parent) opens a dedicated smartphone app and sends photos and videos of the sports day, along with their thoughts such as "Today was a fun sports day!"

[1524] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[1525] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[1526] 4. The generative AI model generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[1527] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[1528] 6. After user confirms, distribute the article to relatives and friends.

[1529] Prompt Sentence Examples

[1530] "We had a fun family sports day today! We took lots of photos and videos. Let us know what you think of it!"

[1531] In this way, this system allows users to share their experiences and thoughts with others in an emotionally rich way, realizing more effective communication.

[1532] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1533] Step 1:

[1534] The user launches the smartphone app, enters photos, videos, and text, and clicks the send button.

[1535] Input: photos, videos, text

[1536] Output: Transmitted data

[1537] Specific operation: The user enters, for example, photos and videos from the sports day and a comment such as "Today was a fun sports day!" into the smartphone app's input interface, and then presses the send button to send the data to the server.

[1538] Step 2:

[1539] The server receives the data sent by the user and temporarily stores it in a buffer.

[1540] Input: Send data

[1541] Output: Received data

[1542] Specific operation: The server detects the reception of transmitted data and temporarily stores it in buffer memory. Here, photos and videos are stored as binary data, and text is stored as character string data.

[1543] Step 3:

[1544] The server parses the incoming data and uses TextBlob to analyze the sentiment of the text and extract keywords and tags.

[1545] Input: Received data

[1546] Output: sentiment tags, keywords, tags

[1547] Specific operation: The server sends the saved string data to the TextBlob library for sentiment analysis. For example, from the text "Today was a fun sports day!", a "joy" tag is generated, and important keywords such as "sports day" and "fun" are extracted.

[1548] Step 4:

[1549] The server uses Pillow to obtain basic information about the received image and video data.

[1550] Input: Received data (images, videos)

[1551] Output: Basic information (image / video format, size, etc.)

[1552] What it does: The server opens the saved binary data using the Pillow library and retrieves basic information such as image format and size. It also analyzes video metadata.

[1553] Step 5:

[1554] The emotion engine analyzes emotions in the received data and assigns emotion tags.

[1555] Input: text data, image data, video data

[1556] Output: Emotion tag

[1557] Specific operation: The emotion engine in the server analyzes emotions from the entire received data. For example, it analyzes the content of a video and generates emotion tags such as "surprise" or "emotion" from specific scenes and assigns them to the data.

[1558] Step 6:

[1559] The generative AI model automatically generates articles based on the analyzed data and sentiment tags.

[1560] Input: emotion tags, keywords, basic information about images and videos

[1561] Output: Auto-generated article

[1562] Specific operation: The generative AI model on the server generates captions such as "The touching moment when we came in second place at the sports day" based on emotion tags and keywords, and creates an emotionally rich story-style article.

[1563] Step 7:

[1564] The server allows the user to select the media format of the generated article and then finalizes the article based on the selection.

[1565] Input: Auto-generated article

[1566] Output: Final generated article

[1567] Specific operation: The server sends the preview link to the user and lets them choose from media formats such as magazine format, news format, etc. Then it receives the selection results and constructs the final article.

[1568] Step 8:

[1569] The user checks the final generated article and clicks the distribution button.

[1570] Input: Final generated article

[1571] Output: Delivery instructions

[1572] Specific operation: The user clicks the preview link to check the final generated article, and then, if satisfied, clicks the publish button to instruct the article to be published.

[1573] Step 9:

[1574] The server distributes the final generated article based on a selected list of recipients.

[1575] Input: Delivery instructions, recipient list

[1576] Output: Delivery complete

[1577] Specific operation: The server receives distribution instructions and sends the article via email or social media based on the specified recipient list.

[1578] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1579] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1580] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1581] [Fourth embodiment]

[1582] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1583] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1584] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1585] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1586] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1587] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1588] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1589] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1590] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1591] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1592] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1593] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1594] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1595] Understood. Now, let's write the "Form for carrying out the invention."

[1596] The present invention provides a system that allows users to share their experiences and thoughts with others. This system receives and analyzes content sent by users, and generates and distributes articles based on that content, thereby stimulating communication.

[1597] System Overview

[1598] The system consists of the following main components:

[1599] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[1600] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[1601] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data.

[1602] User operation procedure

[1603] Users can use a dedicated LINE account to send content, such as photos, videos, and comments from a sports day, using the LINE chat function.

[1604] Server Processing

[1605] The server receives content sent from LINE accounts. After receiving the data, the server analyzes it and extracts important keywords and tags. It uses image recognition technology to understand the content of photos and videos, and natural language processing tools to analyze text data. It then invokes a generation AI based on the analysis results to automatically generate news-style articles.

[1606] Processing of generated AI

[1607] The generative AI automatically generates articles based on the received data and analysis results. First, it generates appropriate captions from images and videos, and then it generates the sentences that make up the overall story. In this process, it takes into account keywords and tags to create articles that reflect the user's intentions.

[1608] Selecting a Media Format

[1609] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[1610] Article generation and distribution

[1611] The server generates the final article based on the media format selected by the user. The generated article is sent to the user as a final preview. After the user confirms it, they click the distribution button, and the server distributes the article via LINE based on the specified recipient list.

[1612] Specific examples

[1613] Example 1: Sharing your child's growth

[1614] 1. The user (parent) opens a dedicated LINE account and sends photos, videos, and impressions of the sports day.

[1615] 2. The server receives this data, checks the format and size of the images and videos, and analyzes the text.

[1616] 3. The AI ​​generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[1617] 4. The user selects the magazine-style media format, and the server sends the magazine-style article as the final preview.

[1618] 5. After user confirms, distribute the article to relatives and friends.

[1619] Example 2: Amateur sports promotion

[1620] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[1621] 2. The server receives and analyzes this data.

[1622] 3. Based on the content of the video, the generative AI generates a caption such as "The next generation star player's amazing play," and assembles the whole video into an article.

[1623] 4. The user selects a news-like media format, and the server sends the news-like article as the final preview.

[1624] 5. After the user has confirmed it, the article is distributed to fans and supporters.

[1625] Through the above-described embodiments, the present invention provides specific means for effectively sharing users' experiences and thoughts and for activating communication.

[1626] The processing flow will be explained below.

[1627] Okay, now I'll explain the process step by step as you instructed.

[1628] Step 1:

[1629] The user opens a dedicated LINE account, where they are authenticated to use the system.

[1630] Step 2:

[1631] A user sends content (photos, videos, text) using the LINE chat function. At this time, the user checks the content to be sent and clicks the send button at the appropriate time.

[1632] Step 3:

[1633] The server receives the content from the LINE account and temporarily stores the received data in a buffer.

[1634] Step 4:

[1635] The server analyzes the received content, using image recognition technology to extract important elements in the case of images, and text analysis tools to extract keywords and tags from the text.

[1636] Step 5:

[1637] The server stores the parsed data in a database, including the user ID, timestamp, and extracted keywords and tags.

[1638] Step 6:

[1639] The server calls the AI ​​generator, which uses the saved data as input to automatically generate news-style articles. The AI ​​analyzes the content of photos and videos and generates appropriate captions.

[1640] Step 7:

[1641] The generative AI generates captions and stories and determines the overall structure. For example, it creates a caption such as "The moving moment when we came in second place at the sports day" and reflects it in the article.

[1642] Step 8:

[1643] The server sends a preview link of the generated article to the user, who clicks the preview link to check the content of the article.

[1644] Step 9:

[1645] The user selects the media format of the article on the preview screen. Options include magazine, news, album, etc.

[1646] Step 10:

[1647] The server generates the final article based on the media format selected by the user, and the final article is stored on the server.

[1648] Step 11:

[1649] The server sends the final preview link back to the user, who then checks the final preview and verifies that there are no problems with the content or format of the article.

[1650] Step 12:

[1651] The user clicks the distribution button, which causes the server to distribute the article to recipients via LINE based on the distribution list.

[1652] Through each of the above steps, this system completes the process of effectively generating and sharing user experiences and thoughts as articles.

[1653] Example 1

[1654] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1655] Conventional content sharing systems have limited the means by which users can effectively share their experiences and thoughts with others, often requiring them to manually create articles. This places a heavy burden on users and limits the quality and quantity of information they want to share. It has also been difficult to generate articles compatible with multiple media formats and to effectively distribute them to recipients. To solve these issues, a system that can automatically analyze user experiences and share them effectively is needed.

[1656] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1657] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, means for performing text analysis using a natural language processing tool, means for analyzing photos and videos using image recognition technology, means for automatically generating articles using an artificial intelligence model, and means for sending the results of the data analysis to the artificial intelligence as prompt sentences. This allows the user to check high-quality articles automatically generated based on the sent content in various media formats and effectively distribute them to multiple recipients.

[1658] "User terminal" refers to an electronic device used by a user to transmit content, including a smartphone, tablet, personal computer, etc.

[1659] "Server" refers to the central system that receives, analyzes, and stores content sent by users, and automatically generates and distributes articles.

[1660] A "generative AI model" refers to an artificial intelligence model that automatically generates articles based on received data and analysis results, and uses natural language generation technology.

[1661] A "prompt sentence" refers to a sentence that is input into a generative AI model based on the results of data analysis, and includes instructions for generating appropriate captions and stories.

[1662] The "means for receiving" refers to a mechanism by which the server receives multiple contents sent by the user.

[1663] "Means for analyzing" refers to the processes and techniques used to analyze received content and extract significant keywords and tags.

[1664] "Means for automatically generating news-style articles" refers to systems and technologies for automatically creating news-style text based on analyzed data.

[1665] "Means for selecting media format" refers to a mechanism for choosing from multiple formats for the appearance and layout of the generated article.

[1666] "Means for distribution to multiple recipients" refers to a mechanism by which the server transmits the generated article to multiple designated recipients.

[1667] "Natural language processing tools" refers to software and technology for analyzing text data and extracting important keywords and tags.

[1668] "Image recognition technology" refers to technology that analyzes the content of photos and videos and recognizes objects and scenes.

[1669] "Means for automatically generating articles using an artificial intelligence model" refers to a mechanism that uses a generative AI model to automatically create articles based on received and analyzed data.

[1670] "Means for transmitting data analysis results to artificial intelligence as prompt sentences" refers to the processes and techniques for generating the analyzed data as prompt sentences and passing them to an artificial intelligence model.

[1671] The "means for final confirmation" refers to a mechanism by which a user can confirm, edit, or approve the generated article.

[1672] The "means for sending a final preview to a user" refers to a mechanism for sending a preview of the generated article to a user for confirmation.

[1673] The present invention provides a system that allows users to share their experiences and thoughts with others. This system receives and analyzes content sent by users, and generates and distributes articles based on that content, thereby stimulating communication.

[1674] System Overview

[1675] The system consists of the following main components:

[1676] User terminal: A device for transmitting content (such as a smartphone, tablet, or personal computer).

[1677] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[1678] Generative AI model: An artificial intelligence model that automatically generates news-style articles based on the data it receives.

[1679] User operations

[1680] Users send content using dedicated messaging applications. For example, users send photos, videos, and impressions of a sports day using the chat function of the messaging application.

[1681] Server Processing

[1682] The server receives the content (photos, videos, text) sent by the user using the messaging API. After receiving the content, the server performs the following actions:

[1683] 1. Data Analysis: The server analyzes the data sent and extracts important keywords and tags. Specifically, it uses the following techniques:

[1684] Image analysis: The content of the photos and videos sent is analyzed using image recognition technology. Specifically, general image recognition technology is used.

[1685] Text analysis: Use natural language processing tools to analyze the text submitted by the user, such as spaCy or a natural language processing API.

[1686] 2. Calling a generative AI model: Based on the analysis results, a generative AI model (such as a common generative AI model like GPT-4) is called to automatically generate a news-style article.

[1687] Prompt creation: The server compiles the results of image analysis and text analysis into a prompt. Examples of specific prompts are as follows:

[1688] Photo caption: "The touching moment when I came second at the sports day."

[1689] Testimonial: "The children did a great job in beautiful weather. We were all particularly impressed by the high jump."

[1690] Media Format Selection

[1691] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[1692] Article generation and distribution

[1693] The server generates the final article based on the media format selected by the user, and sends the article to the user as a final preview. After the user gives their final confirmation, they click the distribute button, and the server sends the article to multiple specified recipients.

[1694] Specific examples

[1695] Example 1: Sharing your child's growth

[1696] 1. The user opens a dedicated messaging application and sends photos, videos, and impressions of the sports day.

[1697] 2. The server receives this data using a messaging API.

[1698] 3. The server uses image recognition technology to analyze the content of the photo and obtain the information that the photo "came in second place." At the same time, it uses natural language processing tools to analyze the comments and extract important keywords (such as "great weather" and "the kids worked hard").

[1699] 4. The server sends prompts based on the analysis results to the generative AI model.

[1700] 5. The generative AI model generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[1701] 6. The user clicks on the preview link sent by the server and selects the magazine-style media format.

[1702] 7. The server generates a magazine-style article and sends it to the user as a final preview.

[1703] 8. The user checks the preview and clicks the share button, and the server distributes the article to the specified relatives and friends.

[1704] In this way, the present invention provides a concrete means for effectively sharing users' experiences and thoughts and stimulating communication.

[1705] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1706] Step 1: Submit your content

[1707] Users use a dedicated messaging application to send content such as photos, videos, and text. Specifically, users select a photo or video on the messaging application and enter their thoughts and descriptions in text. This content is then sent to the server via the messaging API.

[1708] Input: Photos, videos, text

[1709] Output: Data sent from the user to the server

[1710] Step 2: Receiving content

[1711] The server receives the content sent by the user using the messaging API, and stores it in a database for further analysis.

[1712] Input: User-submitted content (photos, videos, text)

[1713] Output: Content received and stored in a database

[1714] Step 3: Prepare for data analysis

[1715] The server prepares the received content for analysis by placing the data in a queue for passing to the analysis block.

[1716] Input: Content stored in the database

[1717] Output: Content queued for analysis

[1718] Step 4: Data analysis

[1719] The server analyzes the transmitted content in the following way:

[1720] 1. Image analysis: The server uses image recognition technology to analyze the content of the photos and videos you send. For example, it uses common image recognition technology to recognize objects and scenes in the images and gather information for caption generation.

[1721] Input: Image or video in the content

[1722] Output: Information about objects and scenes in the image

[1723] 2. Text Analysis: The server uses natural language processing tools to analyze the text submitted by the user, such as "spaCy" or "a type of natural language processing API," to extract important keywords and tags.

[1724] Input: The text submitted by the user

[1725] Output: Important keywords, tags, and sentiment analysis results

[1726] Step 5: Invoke the generative AI model

[1727] Based on the analysis results, the server calls a generative AI model (such as a common generative AI model like "GPT-4") to automatically generate news-style articles.

[1728] Input: Image analysis results, text analysis results

[1729] Output: Prompt sentence to be passed to the generative AI model

[1730] Specific behavior: The server generates a prompt based on the parsed result:

[1731] Photo caption: "The touching moment when I came second at the sports day."

[1732] Testimonial: "The children did a great job in beautiful weather. We were all particularly impressed by the high jump."

[1733] Step 6: Auto-generating articles

[1734] The generative AI model generates news-style articles based on prompts sent from the server, creates appropriate captions based on the input prompts, and builds the entire story.

[1735] Input: prompt statement

[1736] Output: Auto-generated article

[1737] What it does: A generative AI model generates news articles, including photo captions and the overall story.

[1738] Step 7: Select the media format

[1739] The user clicks on the preview link sent by the server to view the generated article, at which point the user can choose from multiple media formats, including magazine, news, and album formats.

[1740] Input: Preview link, generated article

[1741] Output: Selected media format

[1742] What happens: The user clicks on the preview link you sent them and selects the media format they want.

[1743] Step 8: Final article generation and confirmation

[1744] The server generates the final article based on the media format selected by the user, and sends the generated article to the user as a final preview.

[1745] Input: Selected media format, generated article

[1746] Output: Final preview article

[1747] What happens: The server formats the article using the selected media format and sends it to the user as a final preview.

[1748] Step 9: Article Distribution

[1749] After the user checks the final preview, they click the distribute button, and the server sends the article to multiple specified recipients.

[1750] Input: Final preview article, recipient list

[1751] Output: Delivery completion notification to the recipient

[1752] What happens: The user clicks the distribute button, and the server distributes the article to the specified recipient list.

[1753] (Application example 1)

[1754] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1755] In systems that allow users to effectively share their experiences and thoughts with others, there are problems such as the complexity of content analysis and article generation, and the inability of users to edit or preview articles, which increases the possibility of articles being distributed that do not match the user's intentions.

[1756] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1757] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, and means for previewing and editing the generated articles. This allows users to edit articles in a way that suits their own intentions and distribute them after final confirmation.

[1758] "User" means an individual or organization that uses the system to submit content to share their experiences and thoughts with others.

[1759] "Content" is a general term for digital data such as photos, videos, and text sent by users.

[1760] "Means for receiving" refers to the method or technology by which a central system such as a server receives the transmitted content.

[1761] "Means for analysis" refers to methods and technologies for understanding the content of received content and extracting important keywords and tags.

[1762] A "keyword" is a word or phrase that indicates important information about the content and characterizes the content.

[1763] A "tag" is a label that indicates the attributes or category of content, and is information that makes it easier for users to organize and search for content.

[1764] A "news-style article" is a document that describes a user's experience in the form of news, including automatically generated content.

[1765] "Automatic generation means" refers to the software and algorithms used to mechanically generate articles based on the analysis results.

[1766] The "media format" refers to the format or style for displaying the generated article, and includes news style, magazine style, album style, and the like.

[1767] A "production method" is a method or technique for producing an article in a selected media format.

[1768] A "delivery means" is a method or technique for sending the final generated article to the intended recipient.

[1769] A "previewing means" is a method or technique for displaying the generated article for final user confirmation.

[1770] "Editing means" refers to methods or techniques that allow users to make changes to generated articles.

[1771] A specific system architecture and its operation for implementing the present invention will now be described.

[1772] System Overview

[1773] The system of the present invention consists of the following main components:

[1774] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[1775] Server: The central system that receives, analyzes, stores, generates articles, and distributes content.

[1776] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data.

[1777] Communication infrastructure: The Internet that connects user terminals and servers.

[1778] User terminal operation procedure

[1779] Users use a dedicated mobile application to send content, such as photos, videos, and impressions of sports days, using the app's in-app features.

[1780] Server Processing

[1781] The server receives the content sent from the user terminal. The received content is processed as follows:

[1782] 1. Data reception: The server receives the photos, videos, and text data sent by the user.

[1783] 2. Data Analysis: Analyzes the received data and extracts important keywords and tags. Using image recognition technology and natural language processing tools, the content of photos and videos is analyzed in detail.

[1784] 3. Generative AI call: Based on the analysis results, a generative AI model is called to automatically generate news-style articles. Libraries such as Hugging Face Transformers are used.

[1785] Article Generation Process

[1786] The generative AI automatically generates articles based on the received data and analysis results. Specifically, it generates appropriate captions from images and videos, and then generates the sentences that make up the overall story.

[1787] Select and preview media formats

[1788] The generated article is sent as a preview from the server to the user's device. The user can select from multiple media formats, such as news, magazine, and album, and edit the article as needed.

[1789] delivery

[1790] After the user has given their final confirmation, the article is distributed to multiple recipients via LINE or other social media platforms.

[1791] Specific examples

[1792] Example 1: Sharing your child's growth

[1793] 1. The user (parent) opens the dedicated app and sends photos, videos, and impressions of the sports day.

[1794] 2. The server receives this data, checks the format and size of the images and videos, and analyzes the text.

[1795] 3. The AI ​​generates a caption for the photo, such as "The touching moment when we came in second place at the sports day," and summarizes the entire story in news format.

[1796] 4. The user selects the magazine-style media format, and the server sends the magazine-style article as the final preview.

[1797] 5. After user confirms, distribute the article to relatives and friends.

[1798] Example 2: Amateur sports promotion

[1799] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[1800] 2. The server receives and analyzes this data.

[1801] 3. Based on the content of the video, the generative AI generates a caption such as "The next generation star player's amazing play," and assembles the whole video into an article.

[1802] 4. The user selects a news-like media format, and the server sends the news-like article as the final preview.

[1803] 5. After the user has confirmed it, the article is distributed to fans and supporters.

[1804] Prompt Sentence Examples

[1805] "Upload five photos from the sports day and enter the following comment: 'Today was a great sports day. It was impressive to see the kids running around with such energy.'"

[1806] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1807] Step 1:

[1808] The user launches a dedicated mobile application and sends content such as photos, videos, and text.

[1809] Input: Photos, videos, and text data from users.

[1810] Output: The content sent to the server.

[1811] Specific actions: The user uses the app to enter photos and videos from the sports day, as well as their thoughts, and then taps the send button.

[1812] Step 2:

[1813] The server receives the content sent by the user.

[1814] Input: Digital content sent from a user device.

[1815] Output: Data stored in the server's storage.

[1816] Specific operation: The server receives a request from the user terminal and stores the received content in temporary storage.

[1817] Step 3:

[1818] The server analyzes the received content and extracts keywords and tags.

[1819] Input: Photos, videos, and text data stored on the server.

[1820] Output: A list of extracted keywords and tags.

[1821] Specific operation: The server analyzes the content using image recognition technology (e.g., OpenCV) and natural language processing tools (e.g., NLTK) and extracts important keywords and tags.

[1822] Step 4:

[1823] The server calls a generative AI model based on the analysis results and automatically generates news-style articles.

[1824] Input: Analysis results (list of keywords and tags), generative AI model.

[1825] Output: The generated news-style article.

[1826] How it works: The server uses the Hugging Face Transformers library to input the analysis results into a generative AI model and provide a prompt for article generation. It uses a prompt such as, "Today was a great sports day. It was impressive to see the children running around with such energy."

[1827] Step 5:

[1828] The server transmits the generated article to the user terminal as a preview.

[1829] Input: A generated news-style article.

[1830] Output: Article preview displayed on the user's device.

[1831] Specific operation: The server converts the article into HTML format and sends it to the user's device. The user can then preview the article through the app.

[1832] Step 6:

[1833] The user reviews the article and makes edits as necessary.

[1834] Input: The article preview shown to the user.

[1835] Output: The edited article.

[1836] What it does: Users can check the preview, correct typos, or enter additional information. Once they're done editing, they tap the save button.

[1837] Step 7:

[1838] The server delivers the final checked article to multiple recipients.

[1839] Input: The last article edited and reviewed by the user.

[1840] Output: The final article distributed to multiple recipients.

[1841] Specific operation: The server distributes the articles confirmed by the user via LINE or other SNS based on the specified recipient list.

[1842] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1843] Understood. Now, I will describe the "Form for carrying out the invention" based on the "Claims for the invention combining emotion engines."

[1844] This invention provides a system that generates more accurate and emotionally rich articles by combining analysis of content sent by users with an emotion engine that recognizes the user's emotions. This system allows users to effectively share their experiences and thoughts with others, stimulating communication.

[1845] System configuration

[1846] The system consists of the following main components:

[1847] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[1848] Server: The central system that receives, analyzes, stores, generates, and distributes content.

[1849] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data and sentiment data.

[1850] Sentiment Engine: A tool that analyzes and extracts sentiment information from user-submitted content.

[1851] User operation procedure

[1852] Users can send photos, videos, and text from a dedicated LINE account. For example, they can use the LINE chat function to send photos, videos, and comments about a sports day.

[1853] Server Processing

[1854] The server receives the content sent by the user, stores it in a buffer, and then analyzes it using image recognition technology and text analysis tools to extract important keywords and tags.

[1855] Emotion engine processing

[1856] The emotion engine recognizes emotions contained in the received content. For example, it analyzes the user's emotions such as joy, surprise, and sadness from the transmitted text and voice data. This generates emotion tags and assigns them to the content. These emotion tags are taken into account in the article generation process.

[1857] Processing of generated AI

[1858] The generative AI automatically generates articles based on the analyzed data and emotion tags. It creates captions for images and videos and constructs stories that reflect the emotion tags. For example, when generating a caption to express "the touching moment of coming second in a sports day," it reflects the user's emotion tag for joy.

[1859] Selecting a Media Format

[1860] The user can click on the preview link sent by the server to view the generated article, where they can choose from multiple media formats such as magazine, news, and album.

[1861] Article generation and distribution

[1862] The server generates the final article based on the media format selected by the user. This final article is saved on the server and sent back to the user as a preview link. The user can check the final preview and click the share button to distribute the article via LINE to the specified recipient list.

[1863] Specific examples

[1864] Example 1: Sharing your child's growth

[1865] 1. The user (parent) opens a dedicated LINE account and sends photos, videos, and impressions of the sports day.

[1866] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[1867] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[1868] 4. The generative AI generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[1869] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[1870] 6. After user confirms, distribute the article to relatives and friends.

[1871] Example 2: Amateur sports promotion

[1872] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[1873] 2. The server receives and analyzes this data.

[1874] 3. The emotion engine recognizes the user's emotions of "surprise" and "pride" from their post-match impressions and assigns emotion tags.

[1875] 4. The generative AI generates captions such as "The next generation star player's amazing play" and composes articles that reflect the sentiment tags.

[1876] 5. The user selects a news-like media format and the server sends the news-like article as the final preview.

[1877] 6. After the user has confirmed it, the article is distributed to fans and supporters.

[1878] In this way, the present invention provides a system that combines emotion engines to enable users to share their experiences and thoughts in a richer way and achieve more effective communication.

[1879] The processing flow will be explained below.

[1880] Understood. Now, we will explain each processing step based on the "Patent Claims for Inventions Combining Emotion Engines" in detail in the following format.

[1881] Step 1:

[1882] Users use a dedicated LINE account to send content such as photos, videos, and text. For example, a user can upload photos and videos from their child's sports day, or write their thoughts after the event, and click the send button.

[1883] Step 2:

[1884] The server receives the content sent by the user, and the received content is temporarily stored in a buffer.

[1885] Step 3:

[1886] The server analyzes the received content, specifically using image recognition technology to analyze the content of photos and videos, and using text analysis tools to extract keywords and tags from the text data.

[1887] Step 4:

[1888] The emotion engine recognizes the user's emotions from the received and analyzed text and voice data. For example, it recognizes the emotion of "joy" from the positive nuances in the text and generates an emotion tag.

[1889] Step 5:

[1890] The server stores the analyzed data and emotion tags in a database. The stored data includes user IDs, timestamps, keywords, tags, and emotion tags.

[1891] Step 6:

[1892] The server calls the generation AI, which automatically generates news-style articles using content data and emotion tags as input. The generation AI generates captions from images and videos and composes articles that reflect the emotion tags.

[1893] Step 7:

[1894] The AI ​​then creates an entire story based on the generated captions and emotion tags. For example, it incorporates an emotionally rich caption such as "The touching moment when we came in second place at the sports day."

[1895] Step 8:

[1896] The server sends the user a preview link of the generated article, which the user clicks to view the article.

[1897] Step 9:

[1898] The user selects the media format of the article on the preview screen, such as magazine style, news style, or album style.

[1899] Step 10:

[1900] The server generates the final article based on the media format selected by the user, and the final article is stored on the server.

[1901] Step 11:

[1902] The server sends a final preview link back to the user, who then checks the final preview to ensure the article is correct in terms of content and format.

[1903] Step 12:

[1904] The user clicks the distribution button, which causes the server to distribute the article to recipients via LINE based on the distribution list.

[1905] As a result, a system that combines an emotion engine can share users' experiences and thoughts more accurately and emotionally.

[1906] Example 2

[1907] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1908] Conventional content generation systems are limited to analyzing the content submitted by users and do not adequately consider emotional expression. As a result, the generated articles do not effectively reflect users' emotions and thoughts, resulting in insufficient content sharing and communication. To solve this problem, a system that analyzes emotions and reflects them in article generation is needed.

[1909] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1910] In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for recognizing emotions from the received content and generating emotion tags, means for automatically generating news-style articles based on the analyzed data and emotion tags, means for selecting a media format for the generated article, and means for generating articles based on the selected media format and distributing them to multiple recipients. This enables article generation that reflects the user's emotions, enabling more effective content sharing and communication.

[1911] "User" means an individual or organization that utilizes the system to submit content, review generated articles, and distribute them.

[1912] "Content" refers to information such as photos, videos, and text that users submit to the system.

[1913] The "receiving means" is a mechanism by which the server receives the content sent by the user and temporarily stores it.

[1914] "Analysis means" refers to a system that analyzes received content using image recognition technology and text analysis tools to extract important keywords and tags.

[1915] The "emotion engine" is a mechanism that recognizes emotions within received content and generates emotion tags based on those emotions.

[1916] An "emotion tag" is an identifier that indicates the emotional information analyzed by the emotion engine and is used to generate articles.

[1917] "Generative AI" is artificial intelligence that automatically generates news-style articles based on analyzed data and emotion tags.

[1918] "Media format" refers to the layout and style in which an article is displayed.

[1919] The "distribution means" is a mechanism for transmitting the generated article to multiple recipients.

[1920] The "final confirmation means" is a mechanism by which a user can finally confirm the content of a generated article.

[1921] Okay, so based on the information we have so far, we'll create a detailed description.

[1922] This invention provides a system that generates more accurate and emotionally rich articles by combining analysis of content sent by users with an emotion engine that recognizes the user's emotions. This system allows users to effectively share their experiences and thoughts with others, stimulating communication.

[1923] System configuration

[1924] The system consists of the following main components:

[1925] User terminal: A device for transmitting content (smartphone, tablet, PC, etc.).

[1926] Server: The central system that receives, analyzes, stores, generates, and distributes content.

[1927] Generative AI: Artificial intelligence that automatically generates news-style articles based on received data and sentiment data.

[1928] Sentiment Engine: A tool that analyzes and extracts sentiment information from user-submitted content.

[1929] User operation procedure

[1930] Users send photos, videos, and text from a dedicated messaging application. For example, when sending photos, videos, or impressions of a sports day, they use the chat function of the messaging application.

[1931] Server Processing

[1932] The server receives the content sent by the user. After receiving the content, it first temporarily stores it in a buffer. Then, the server analyzes the data using image recognition technology (e.g., a general image recognition API) and text analysis tools (e.g., a general text analysis API) to extract important keywords and tags.

[1933] Emotion engine processing

[1934] The emotion engine recognizes emotions contained in the received content. For example, it analyzes the user's emotions such as joy, surprise, and sadness from the transmitted text and voice data. This generates emotion tags and assigns them to the content. These emotion tags are taken into account in the article generation process.

[1935] Processing of generated AI

[1936] The generative AI automatically generates articles based on the analyzed data and emotion tags. It creates captions for images and videos and constructs stories that reflect the emotion tags. For example, when generating a caption to express "the touching moment of coming second in a sports day," it reflects the user's emotion tag of "joy."

[1937] Selecting a Media Format

[1938] The user can click on the preview link sent by the server to view the generated article, where they can choose from multiple media formats such as magazine, news, and album.

[1939] Article generation and distribution

[1940] The server generates the final article based on the media format selected by the user, saves the final article on the server, and sends it back to the user as a preview link. The user can view the final preview and click the distribute button, which distributes the article via a messaging application based on the specified recipient list.

[1941] Specific examples

[1942] Sharing your child's growth

[1943] 1. The user (parent) opens a dedicated messaging application and sends photos, videos, and impressions of the sports day.

[1944] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[1945] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[1946] 4. The generative AI generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[1947] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[1948] 6. After user confirms, distribute the article to relatives and friends.

[1949] Amateur sports promotion

[1950] 1. The user (amateur athlete) sends highlight videos, photos, and impressions of the game.

[1951] 2. The server receives and analyzes this data.

[1952] 3. The emotion engine recognizes the user's emotions of "surprise" and "pride" from their post-match impressions and assigns emotion tags.

[1953] 4. The generative AI generates captions such as "The next generation star player's amazing play" and composes articles that reflect the sentiment tags.

[1954] 5. The user selects a news-like media format and the server sends the news-like article as the final preview.

[1955] 6. After the user has confirmed it, the article is distributed to fans and supporters.

[1956] This system enables users to effectively share their experiences and thoughts, enabling richer communication.

[1957] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1958] I understand. Now, I will explain the program processing flow of this system by dividing it into specific steps.

[1959] Program processing flow

[1960] Step 1: User submits content

[1961] A user sends content such as photos, videos, and text using a dedicated messaging application.

[1962] Input: Photos, videos, and text data sent by users.

[1963] Output: The content data sent to the server.

[1964] Specific operation: The user opens a message sending application on a smartphone or PC, enters "Photos from the sports day," "Video from the sports day," and "I was so happy to come in second place at the sports day," and presses the send button.

[1965] Step 2: The server receives and stores the content

[1966] The server receives the content sent by the user and temporarily stores it in a buffer.

[1967] Input: Content data submitted by the user.

[1968] Output: The content data stored in the buffer.

[1969] Specific operation: The server separates the received photos, videos, and text into their respective data formats and temporarily stores them in buffer storage with a file name such as "userID_12345".

[1970] Step 3: The server parses the content

[1971] The server uses image recognition technology and text analysis tools to analyze the stored content and extract important keywords and tags.

[1972] Input: The content data stored in the buffer.

[1973] Output: Extracted keywords, tag list.

[1974] Specific operation: The server uses an image recognition API to extract keywords such as "sports day" and "podium" from the photo, and uses a text analysis API to extract keywords such as "second place" and "I was happy" from the text data.

[1975] Step 4: The emotion engine recognizes emotions and assigns tags

[1976] The emotion engine analyzes the emotions contained in the received content, generates emotion tags, and assigns them to the content.

[1977] Input: Extracted keywords, tag list.

[1978] Output: Emotion-tagged data.

[1979] Specific operation: The emotion engine uses a text analysis API to identify the user's emotion, such as "joy" or "surprise," generate tags based on that emotion, and assign the "joy" emotion tag to the text data.

[1980] Step 5: Generative AI generates articles

[1981] Generative AI automatically generates news-style articles based on the analyzed data and sentiment tags.

[1982] Input: Emotion-tagged data.

[1983] Output: The generated news-style article.

[1984] Specific operation: The generative AI (for example, GPT-3) generates captions such as "The touching moment when we came in second place at the sports day" and creates articles that reflect the emotion tag "joy."

[1985] Step 6: User selects media format

[1986] The user clicks on the preview link sent from the server to check the generated article and selects a media format such as magazine style, news style, or album style.

[1987] Input: A generated news-style article.

[1988] Output: Information for the selected media format.

[1989] What happens: The user clicks the preview link to see the generated article and selects the "magazine-style layout."

[1990] Step 7: The server generates the final article

[1991] The server generates the final article based on the media format selected by the user.

[1992] Input: Selected media format information, generated news format article.

[1993] Output: The final generated article.

[1994] What it does: The server uses a magazine-style layout template to position photos, captions, and text appropriately to generate the final article.

[1995] Step 8: Server sends final article

[1996] The server sends the final article to the user as a preview link.

[1997] Input: The final generated article.

[1998] Output: Preview link.

[1999] Specific operation: The server saves the generated final article in storage, generates a preview link, and sends it to the user.

[2000] Step 9: User publishes article

[2001] The user sees the final preview and distributes the article to multiple recipients.

[2002] Input: Preview link.

[2003] Output: A list of recipients to whom the article will be distributed.

[2004] Specific operation: The user checks the final preview and clicks the "Distribute" button to send the article to the specified recipients. The server then distributes the article via a messaging application such as LINE based on the recipient list.

[2005] Through the above processing steps, the system generates articles that more fully reflect the user's experiences and emotions, achieving effective communication.

[2006] (Application example 2)

[2007] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2008] In recent years, there has been a growing demand for emotionally rich communication with others via the Internet. However, conventional content generation systems have difficulty automatically generating articles that accurately reflect users' emotions, and as a result, users' experiences and thoughts cannot be fully shared. Furthermore, the generated articles tend to be flat and unattractive, limiting their impact on the recipient. There is a need for a system that can solve these issues and automatically generate and distribute compelling content that reflects users' emotions.

[2009] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving multiple pieces of content sent by a user, means for analyzing the received content and extracting keywords and tags, means for automatically generating news-style articles based on the analyzed data, means for selecting a media format for the generated article, means for generating articles based on the selected media format and distributing them to multiple recipients, means for analyzing emotions in the content and assigning emotion tags, and means for generating articles that reflect the emotion tags. This makes it possible for users to share their experiences and thoughts with others in an emotionally rich way, providing an emotional impact on recipients.

[2010] "Content" is a general term for digital data such as photos, videos, and text sent by users.

[2011] An "emotion tag" is an information tag that is assigned based on the emotion analyzed from the content sent by the user.

[2012] "Generative AI" is artificial intelligence that automatically generates news-style articles and captions based on received data and emotional data.

[2013] The "emotion engine" is a tool that analyzes emotions within content submitted by users and extracts that information.

[2014] "Media format" refers to the display format and design of the generated article, and includes news format, magazine format, album format, and the like.

[2015] A "recipient list" is a set of designated recipients to whom a generated article is to be distributed.

[2016] "Keywords" are important words or phrases extracted by analyzing content.

[2017] A "tag" is classification information or identification information that is assigned to content.

[2018] "Stories" are emotionally rich, sequential articles and content created based on analyzed data and sentiment tags.

[2019] A "preview link" is a URL link provided to allow a user to see a preview of the generated article.

[2020] This invention is a system that analyzes content sent by users, recognizes their emotions, and automatically generates and distributes articles that reflect those emotions. Below, we will explain in detail the system and processing procedures for implementing this invention.

[2021] System configuration

[2022] User device:

[2023] Users use devices such as smartphones, tablets, and PCs to send content.

[2024] server:

[2025] This is the central system that receives, analyzes, stores, generates articles, and distributes content. The server uses the following software and libraries:

[2026] TextBlob: A library for analyzing text sentiment

[2027] Pillow (PIL): A library for image processing

[2028] Generative AI model: Artificial intelligence that generates articles based on content and sentiment data

[2029] Emotion Engine: A tool that analyzes emotions in user-submitted content

[2030] User operation procedure

[2031] Users can send photos, videos, and text from a dedicated smartphone app. For example, they can use the app's input interface to send photos, videos, and comments about a sports day.

[2032] Server Processing

[2033] The server receives the content sent by the user and first stores it in a temporary buffer. Next, it uses TextBlob to analyze the emotion of the received text data and generates emotion tags based on the results. It also uses Pillow to obtain basic information about the received image and video data.

[2034] Emotion engine processing

[2035] The emotion engine analyzes the user's emotions, such as joy, surprise, and sadness, from the received text and voice data, generates emotion tags, and assigns them to the content. These emotion tags are taken into account in the article generation process.

[2036] Processing of generated AI

[2037] The generative AI model automatically generates articles based on the analyzed data and sentiment tags, for example by adding captions to incoming photos and constructing stories that reflect the sentiment tags.

[2038] Preview and distribute content

[2039] The user can click the preview link sent by the server to check the generated article, where they can choose from multiple media formats such as news, magazine, album, etc. After final confirmation, the user clicks the distribution button, and the article is distributed to the specified recipient list.

[2040] Specific examples

[2041] Example 1: Sharing your child's progress

[2042] 1. The user (parent) opens a dedicated smartphone app and sends photos and videos of the sports day, along with their thoughts such as "Today was a fun sports day!"

[2043] 2. The server receives the data, analyzes it, and extracts keywords and tags.

[2044] 3. The emotion engine recognizes the user's emotion of "joy" from the text sent and assigns an emotion tag.

[2045] 4. The generative AI model generates captions for the photos, such as "The touching moment when we came in second place at the sports day," and compiles articles in news format that reflect the emotional tags.

[2046] 5. The user selects the magazine-style media format and the server sends the magazine-style article as the final preview.

[2047] 6. After user confirms, distribute the article to relatives and friends.

[2048] Prompt Sentence Examples

[2049] "We had a fun family sports day today! We took lots of photos and videos. Let us know what you think of it!"

[2050] In this way, this system allows users to share their experiences and thoughts with others in an emotionally rich way, realizing more effective communication.

[2051] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2052] Step 1:

[2053] The user launches the smartphone app, enters photos, videos, and text, and clicks the send button.

[2054] Input: photos, videos, text

[2055] Output: Transmitted data

[2056] Specific operation: The user enters, for example, photos and videos from the sports day and a comment such as "Today was a fun sports day!" into the smartphone app's input interface, and then presses the send button to send the data to the server.

[2057] Step 2:

[2058] The server receives the data sent by the user and temporarily stores it in a buffer.

[2059] Input: Send data

[2060] Output: Received data

[2061] Specific operation: The server detects the reception of transmitted data and temporarily stores it in buffer memory. Here, photos and videos are stored as binary data, and text is stored as character string data.

[2062] Step 3:

[2063] The server parses the incoming data and uses TextBlob to analyze the sentiment of the text and extract keywords and tags.

[2064] Input: Received data

[2065] Output: sentiment tags, keywords, tags

[2066] Specific operation: The server sends the saved string data to the TextBlob library for sentiment analysis. For example, from the text "Today was a fun sports day!", a "joy" tag is generated, and important keywords such as "sports day" and "fun" are extracted.

[2067] Step 4:

[2068] The server uses Pillow to obtain basic information about the received image and video data.

[2069] Input: Received data (images, videos)

[2070] Output: Basic information (image / video format, size, etc.)

[2071] What it does: The server opens the saved binary data using the Pillow library and retrieves basic information such as image format and size. It also analyzes video metadata.

[2072] Step 5:

[2073] The emotion engine analyzes emotions in the received data and assigns emotion tags.

[2074] Input: text data, image data, video data

[2075] Output: Emotion tag

[2076] Specific operation: The emotion engine in the server analyzes emotions from the entire received data. For example, it analyzes the content of a video and generates emotion tags such as "surprise" or "emotion" from specific scenes and assigns them to the data.

[2077] Step 6:

[2078] The generative AI model automatically generates articles based on the analyzed data and sentiment tags.

[2079] Input: emotion tags, keywords, basic information about images and videos

[2080] Output: Auto-generated article

[2081] Specific operation: The generative AI model on the server generates captions such as "The touching moment when we came in second place at the sports day" based on emotion tags and keywords, and creates an emotionally rich story-style article.

[2082] Step 7:

[2083] The server allows the user to select the media format of the generated article and then finalizes the article based on the selection.

[2084] Input: Auto-generated article

[2085] Output: Final generated article

[2086] Specific operation: The server sends the preview link to the user and lets them choose from media formats such as magazine format, news format, etc. Then it receives the selection results and constructs the final article.

[2087] Step 8:

[2088] The user checks the final generated article and clicks the distribution button.

[2089] Input: Final generated article

[2090] Output: Delivery instructions

[2091] Specific operation: The user clicks the preview link to check the final generated article, and then, if satisfied, clicks the publish button to instruct the article to be published.

[2092] Step 9:

[2093] The server distributes the final generated article based on a selected list of recipients.

[2094] Input: Delivery instructions, recipient list

[2095] Output: Delivery complete

[2096] Specific operation: The server receives distribution instructions and sends the article via email or social media based on the specified recipient list.

[2097] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2098] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2099] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2100] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2101] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2102] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2103] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2104] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2105] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2106] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2107] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2108] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2109] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2110] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2111] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2112] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2113] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2114] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2115] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2116] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2117] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2118] The following is further disclosed regarding the above embodiment.

[2119] Understood. Now, I will write the claims as instructed.

[2120] (Claim 1)

[2121] means for receiving a plurality of contents sent by a user;

[2122] A means for analyzing received content and extracting keywords and tags;

[2123] A means to automatically generate news-style articles based on the analyzed data, and

[2124] a means for selecting a media format for the generated article;

[2125] means for generating and distributing the article based on the selected media format to multiple recipients;

[2126] A system including:

[2127] (Claim 2)

[2128] A means for a user to make a final confirmation of the generated article;

[2129] a means for delivering the article after user confirmation;

[2130] The system of claim 1 further comprising:

[2131] (Claim 3)

[2132] means for performing image recognition and text analysis on the received content;

[2133] A means to automatically generate captions from parts of the content;

[2134] A means for constructing an article based on the generated captions;

[2135] The system of claim 1 further comprising:

[2136] "Example 1"

[2137] (Claim 1)

[2138] means for receiving a plurality of contents sent by a user;

[2139] A means for analyzing received content and extracting keywords and tags;

[2140] A means to automatically generate news-style articles based on the analyzed data, and

[2141] a means for selecting a media format for the generated article;

[2142] means for generating and distributing the article based on the selected media format to multiple recipients;

[2143] a means for performing text analysis using natural language processing tools;

[2144] A means of analyzing photos and videos using image recognition technology,

[2145] A means for automatically generating articles using an artificial intelligence model;

[2146] A means for transmitting the data analysis results to the artificial intelligence as a prompt sentence;

[2147] A system including:

[2148] (Claim 2)

[2149] A means for a user to make a final confirmation of the generated article;

[2150] a means for delivering the article after user confirmation;

[2151] means for sending a final preview of the generated article to the user;

[2152] The system of claim 1 further comprising:

[2153] (Claim 3)

[2154] means for performing image recognition and text analysis on the received content;

[2155] A means to automatically generate captions from parts of the content;

[2156] A means for constructing an article based on the generated captions;

[2157] A means to automatically generate news-style articles using generative AI models;

[2158] The system of claim 1 further comprising:

[2159] "Application Example 1"

[2160] (Claim 1)

[2161] means for receiving a plurality of contents sent by a user;

[2162] A means for analyzing received content and extracting keywords and tags;

[2163] A means to automatically generate news-style articles based on the analyzed data, and

[2164] a means for selecting a media format for the generated article;

[2165] means for generating and distributing the article based on the selected media format to multiple recipients;

[2166] A means to preview and edit the generated articles;

[2167] A system including:

[2168] (Claim 2)

[2169] A means for a user to make a final confirmation of the generated article;

[2170] a means for delivering the article after user confirmation;

[2171] The system of claim 1 further comprising:

[2172] (Claim 3)

[2173] means for performing image recognition and text analysis on the received content;

[2174] A means to automatically generate captions from parts of the content;

[2175] A means for constructing an article based on the generated captions;

[2176] The system of claim 1 further comprising:

[2177] "Example 2: Combining Emotion Engines"

[2178] Okay, so let's rewrite the claims to incorporate the distinctive features of this system.

[2179] (Claim 1)

[2180] means for receiving a plurality of contents sent by a user;

[2181] A means for analyzing received content and extracting keywords and tags;

[2182] means for recognizing emotions from received content and generating emotion tags;

[2183] A means to automatically generate news-style articles based on the analyzed data and emotion tags;

[2184] a means for selecting a media format for the generated article;

[2185] means for generating and distributing the article based on the selected media format to multiple recipients;

[2186] A system including:

[2187] (Claim 2)

[2188] 2. The system according to claim 1, wherein the user performs a final check on the generated article, and distributes the article after the check.

[2189] (Claim 3)

[2190] means for performing image recognition and text analysis on the received content;

[2191] A means to automatically generate captions from parts of the content;

[2192] a means for composing articles based on the generated captions and sentiment tags;

[2193] The system of claim 1 further comprising:

[2194] "Application example 2 when combining emotion engines"

[2195] (Claim 1)

[2196] means for receiving a plurality of contents sent by a user;

[2197] A means for analyzing received content and extracting keywords and tags;

[2198] A means to automatically generate news-style articles based on the analyzed data, and

[2199] a means for selecting a media format for the generated article;

[2200] means for generating and distributing the article based on the selected media format to multiple recipients;

[2201] A means for analyzing emotions in content and assigning emotion tags;

[2202] A means for generating articles that reflect sentiment tags;

[2203] A system including:

[2204] (Claim 2)

[2205] A means for a user to make a final confirmation of the generated article;

[2206] a means for delivering the article after user confirmation;

[2207] 10. The system of claim 1.

[2208] (Claim 3)

[2209] means for performing image recognition and text analysis on the received content;

[2210] A means to automatically generate captions from parts of the content;

[2211] A means for constructing an article based on the generated captions;

[2212] A method for creating story-style articles based on emotion tags,

[2213] 10. The system of claim 1. [Explanation of symbols]

[2214] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving a plurality of contents sent by a user; A means for analyzing received content and extracting keywords and tags; A means to automatically generate news-style articles based on the analyzed data, and a means for selecting a media format for the generated article; means for generating and distributing the article based on the selected media format to multiple recipients; A system including:

2. A means for a user to make a final confirmation of the generated article; a means for delivering the article after user confirmation; The system of claim 1 further comprising:

3. means for performing image recognition and text analysis on the received content; A means to automatically generate captions from parts of the content; A means for constructing an article based on the generated captions; The system of claim 1 further comprising:

Images