system

Abstract

We provide the system. [Solution] A means for collecting user behavior data, A means of analyzing user areas of interest based on collected behavioral data, A means of automatically selecting relevant information based on the analysis results, A method for automatically generating articles based on selected information, A means of laying out the generated articles and outputting them as a newspaper, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventionally, means for a user to easily collect customized information based on their interests and create it as an individual newspaper have been limited. Also, the labor for article generation and layout has been large, requiring time and effort. Furthermore, it has been difficult to utilize personal data such as photos as articles. There is a need for a system that solves these problems and allows users to create, share, and save their own newspapers without much effort.

Means for Solving the Problems

[0005] This invention provides a means for collecting user behavior data and analyzing the user's areas of interest based on the collected data. Furthermore, it constructs a system that includes means for automatically selecting relevant information based on the analysis results and automatically generating articles based on that information. It also provides means for laying out the generated articles and outputting them as a newspaper, enabling users to easily create customized newspapers. In addition, it includes means for analyzing images from a photo folder and generating articles based on the images, thereby realizing article generation that utilizes personal data. Furthermore, it includes means for outputting the newspaper selected by the user in a printable format, allowing users to preserve important memories as physical objects for a long time.

[0006] "Behavioral data" refers to information such as a user's operation history, browsing history, search history, and interaction history when using the system.

[0007] "Areas of interest" refer to the themes and categories that a user is interested in, as identified through the analysis of behavioral data.

[0008] "Information" is a general term for content related to the user's areas of interest, such as news articles, social media posts, photos, and videos.

[0009] An "article" is text-based content generated by the system, and is specific content created based on news or user photos.

[0010] "Layout" refers to the technique of visually organizing generated articles and arranging them in a newspaper format, a method of arranging articles in a way that is easy for users to read.

[0011] A "newspaper" is a document consisting of multiple articles arranged in a specific layout, and is a collection of information provided in print or digital format.

[0012] A "photo folder" is a directory or storage location that stores image data saved on the user's device.

[0013] "Image analysis" is a technical method that analyzes image data within a photo folder to identify and extract its contents.

[0014] "Format" refers to the structure and format for outputting a newspaper in print or digital format. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention relates to a system that automatically generates customized newspapers by collecting user behavior data and analyzing the user's areas of interest based on that data. The system consists of a server, a user terminal, and an AI engine.

[0037] Collection of user information

[0038] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, that history is saved in the database.

[0039] Specific example:

[0040] When a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[0041] Interest analysis and news selection

[0042] An AI engine on the server analyzes collected user behavior data to identify the user's areas of interest. Based on these identified areas of interest, the AI ​​engine retrieves relevant and up-to-date information from an external news API. This selects articles that are likely to be of interest to the user.

[0043] Specific example:

[0044] The AI ​​engine analyzes that the user has a strong interest in environmental issues and retrieves the latest environmental articles from the news API.

[0045] Generating content from photo folders

[0046] The server accesses the user's photo folder and extracts data using image analysis technology. Based on the analysis results, it automatically generates relevant articles from the user's photos. The necessary access permissions are obtained from the user in advance.

[0047] Specific example:

[0048] The server detects the user's travel photos and automatically generates a "user's travelogue" based on them.

[0049] Automatic article generation and layout

[0050] The AI ​​engine automatically generates text-based content based on articles created from selected information and photos. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[0051] Specific example:

[0052] The AI ​​engine generates articles on environmental issues and travelogues, and arranges them according to the user's interests. For example, it might place articles on environmental issues at the top of the newspaper, followed by travelogues.

[0053] Newspaper generation and preview display

[0054] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make corrections or changes as needed.

[0055] Specific example:

[0056] The server generates the newspaper as an A4-sized PDF and displays a preview on the user's device. The user checks the preview and, if satisfied with the content, saves or prints it.

[0057] Providing printing options

[0058] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0059] Specific example:

[0060] The user reviews the previewed content and clicks the "Print" button. This causes the server to download the newspaper as a PDF and print it from the user's printer.

[0061] As described above, the system of the present invention significantly reduces the effort required from the user, making it possible to easily create, share, and save customized newspapers.

[0062] The following describes the processing flow.

[0063] Step 1:

[0064] The server collects user behavior data. This includes click data on articles the user views, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, the article ID, viewing start time, and viewing end time are stored in the database.

[0065] Step 2:

[0066] The AI ​​engine on the server analyzes collected behavioral data to identify the user's areas of interest. This analysis uses machine learning algorithms and natural language processing techniques. For example, if a user reads many articles about environmental issues, the AI ​​engine will identify that the user is interested in environmental issues.

[0067] Step 3:

[0068] The AI ​​engine retrieves relevant and up-to-date information from a news API based on identified areas of interest. The news API is a service that provides the latest news articles from various sources on the internet. For example, it retrieves the latest articles on environmental issues.

[0069] Step 4:

[0070] The server accesses the user's photo folder and scans the image data within the folder using image analysis technology. The server obtains the necessary access permissions from the user beforehand. For example, the server can detect the user's travel photos and generate article ideas based on them.

[0071] Step 5:

[0072] The AI ​​engine automatically generates articles based on information obtained from a news API and ideas generated from photo folders. The generated articles are created using natural language generation (NLG) technology. For example, it can automatically generate both news articles on environmental issues and travelogues.

[0073] Step 6:

[0074] The AI ​​engine applies an algorithm to sort the generated articles according to the user's interests. This places the articles the user is most interested in at the top. For example, articles on environmental issues might be placed at the top of the newspaper, followed by travelogues.

[0075] Step 7:

[0076] The server generates a PDF or web version of the original newspaper based on the rearranged articles. The generated newspaper is displayed as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[0077] Step 8:

[0078] Users can view a preview of the newspaper and select the option to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0079] By following these steps, users can easily create, preview, and print a customized newspaper based on their interests.

[0080] (Example 1)

[0081] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0082] Current news delivery methods make it difficult to effectively provide information tailored to individual user interests and concerns, requiring users to sift through a large amount of information themselves. As a result, users waste time and effort, and their information consumption becomes incomplete. Furthermore, there are limitations to the means of easily saving and sharing generated information in physical format.

[0083] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0084] In this invention, the server includes means for collecting operation data performed on the user's terminal, means for analyzing the collected operation data and identifying the user's areas of interest, means for acquiring relevant information from an external data source based on the identified areas of interest, means for generating text content based on the acquired information, means for rearranging the generated text content in order of the user's interests and laying it out in newspaper format, and means for previewing the laid-out newspaper on an electronic device. This makes it possible to automatically generate a customized newspaper according to the user's interests, enabling them to effectively enjoy information while significantly reducing time and effort.

[0085] "User operation data" refers to data recorded on the user's device, such as click data, browsing time, search history, and interactions.

[0086] "Areas of interest" refer to the fields or topics that a user is primarily interested in, and are analyzed from the user's behavioral data.

[0087] "External data sources" refer to external news APIs and databases used to retrieve relevant information.

[0088] "Text content" refers to text-based content generated by an AI engine, including news articles and essays.

[0089] "Newspaper format" refers to a layout in which generated text content is neatly arranged and presented in an easy-to-read format.

[0090] "Electronic devices" refers to display devices such as smartphones, tablets, and personal computers.

[0091] "Image analysis technology" refers to techniques that extract information from image data using computer vision and machine learning algorithms.

[0092] "Preview display" refers to a function that allows users to check the generated content, such as newspapers, in its final output format beforehand.

[0093] A "printable format" refers to a format suitable for physical printing (for example, PDF or DOC format).

[0094] Modes for carrying out the invention

[0095] This invention relates to a system that collects user behavior data, analyzes the user's areas of interest based on that data, and automatically generates a customized newspaper. The system consists of a server, a user terminal, and a generation AI model.

[0096] Collection of user information

[0097] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. Specifically, if a user searches for "environmental protection," clicks on an article, and reads it, the click data and viewing time are collected and stored in the database. MySQL® and PostgreSQL are used as databases.

[0098] Interest analysis and news selection

[0099] An AI engine on the server analyzes collected user behavior data to identify the user's areas of interest. The AI ​​engine runs using Python libraries (e.g., scikit-learn and TENSORFLOW®). Once areas of interest are identified, the server retrieves relevant and up-to-date information through news APIs (e.g., Google® News API or NewsAPI). Specifically, if the AI ​​engine analyzes that the user has a strong interest in environmental issues, it will retrieve the latest articles on environmental issues from the news API.

[0100] Generating content from photo folders

[0101] The server accesses the user's photo folder and extracts data using image analysis techniques. Necessary access permissions are obtained from the user in advance. OpenCV and TensorFlow libraries are used for image analysis. For example, if the user's travel photos are detected, related articles such as "User's Travelogue" are automatically generated based on them.

[0102] Automatic article generation and layout

[0103] The AI ​​engine automatically generates text-based content from selected information and photos. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout. This uses natural language generation technologies such as GPT-3® and BERT. For example, if an article on environmental issues and a travelogue are generated, the article on environmental issues will be placed at the top, followed by the travelogue.

[0104] Newspaper generation and preview display

[0105] The server generates the original newspaper in PDF or HTML format based on the generated articles and displays it as a preview on the user's device. Specifically, it generates the layout using LaTeX or HTML templates and displays it on the user's smartphone or PC. The user can check this preview and make corrections or changes as needed.

[0106] Providing printing options

[0107] After reviewing the previewed newspaper, users can select a print option. The server outputs the newspaper in a printable format (e.g., PDF) and provides a download link to the user's device. It can also initiate direct printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server generates the newspaper as a PDF, sends it to the user's printer, and prints it.

[0108] Examples of prompt statements

[0109] Prompt example 1:

[0110] "Automatically generate news articles based on topics that users are recently interested in. For example, if a user is interested in environmental protection, retrieve the latest environmental news."

[0111] Prompt example 2:

[0112] "Please analyze travel photos from the user's photo folder and automatically generate a travelogue based on that analysis. For example, if there are many photos of Paris, please generate a travelogue about Paris."

[0113] Prompt example 3:

[0114] "Create a customized newspaper based on articles of interest to the user and display it as a preview in PDF format. For example, if the user is interested in environmental issues, create a newspaper focusing on environmentally related articles."

[0115] As a result, the system of the present invention significantly reduces the effort required from the user, making it possible to easily create, share, and save customized newspapers.

[0116] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0117] Step 1:

[0118] The user logs into the application and generates operation data on their device. Inputs include the user's login operation and article clicks and search actions within the application. Outputs are the generated operation data, which includes click data, viewing time, search history, and interactions. Specifically, the user accesses the application using a smartphone or PC, searches for an article using the keyword "environmental protection," and clicks to read it.

[0119] Step 2:

[0120] The server collects user activity data and stores it in a database. The input is the activity data generated in step 1. The output is the activity data stored in the database. Specifically, the server collects user click data, browsing time, and search history, and stores the data in a database such as MySQL or PostgreSQL.

[0121] Step 3:

[0122] An AI engine on the server analyzes collected interaction data to identify the user's areas of interest. The input is the interaction data stored in the database. The output is the identified areas of interest. Specifically, the AI ​​engine uses Python libraries (such as scikit-learn or TensorFlow) to analyze the interaction data and identify that the user is interested in environmental issues.

[0123] Step 4:

[0124] The server retrieves relevant and up-to-date information from external data sources based on identified areas of interest. The input is the user's areas of interest. The output is the retrieved latest information (articles). Specifically, the server uses a news API (e.g., Google News API or NewsAPI) to retrieve the latest articles on environmental issues of interest.

[0125] Step 5:

[0126] The server accesses the user's photo folder, extracts data using image analysis techniques, and automatically generates related articles. The input is the user's photo data. The output is the image analysis results and the automatically generated articles. Specifically, the server uses OpenCV or TensorFlow to analyze travel photos and generates a "user travelogue" based on that analysis. Access permission to the photo folder is obtained from the user beforehand.

[0127] Step 6:

[0128] An AI engine on the server automatically generates text content based on articles created from selected information and photos, and then arranges them in a newspaper format, sorted according to the user's interests. The input is the latest acquired information and image analysis results. The output is a pre-layout newspaper-style article. Specifically, the AI ​​engine uses GPT-3 and BERT to generate articles, placing articles on environmental issues at the top and travelogues next.

[0129] Step 7:

[0130] The server generates a newspaper and displays a preview of it on the user's device. The input is a pre-layout newspaper article. The output is the preview of the newspaper displayed on the device. Specifically, the server generates a newspaper in PDF or HTML format using LaTeX or HTML templates and displays a preview of it on the user's smartphone or PC.

[0131] Step 8:

[0132] The user reviews the previewed newspaper and selects a print option. The server outputs the newspaper in a printable format and provides a download link. It also initiates printing directly with the user's printer. The input is the previewed newspaper. The output is the newspaper in a printable format and a download link. Specifically, when the user clicks the "Print" button, the server generates the newspaper in PDF format, sends it to the printer, and performs the printing.

[0133] (Application Example 1)

[0134] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0135] In recent years, there has been a growing demand for customized services based on users' purchasing behavior and interests, but meeting this demand requires advanced data analysis and automated generation technologies. In particular, e-commerce sites lack mechanisms to efficiently recommend products likely to interest users and provide related news and information. Existing systems fail to fully utilize user behavior data, making it difficult to provide information tailored to individual users.

[0136] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0137] In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for automatically selecting relevant information based on the analysis results, means for automatically generating articles based on the selected information, means for laying out the generated articles and outputting them as a newspaper, means for recommending products on an e-commerce site based on the user's behavior data, and means for outputting articles generated based on the user's areas of interest in PDF format. This makes it possible to provide customized information and product recommendations that are tailored to the interests and purchasing behavior of individual users, thereby improving user satisfaction and increasing purchasing intent.

[0138] "User behavior data" refers to data on actions taken by users on applications and websites, such as click history, browsing history, search history, and purchase history.

[0139] "Areas of interest" refers to the fields or topics that a user is particularly interested in, as determined by analyzing their behavioral data.

[0140] "Relevant information" refers to information that is valuable to the user, such as news articles and product information selected based on the user's areas of interest.

[0141] "Methods for automatically generating articles" refers to algorithms and systems that generate text based on selected relevant information.

[0142] "Methods for outputting as a newspaper" refers to a system that automatically generates articles and lays them out in a specific format (e.g., PDF or web page) for output.

[0143] An "online shopping site" refers to a website that sells goods and services via the internet.

[0144] "A means of recommending products" refers to a system that selects and displays products that a user is likely to be interested in, based on their behavioral data.

[0145] "Methods for outputting in PDF format" refers to systems that save or display generated articles or information as files in PDF format.

[0146] This invention is a system that collects user behavior data and recommends customized information and products based on that data. Specifically, it consists of a server, a user terminal, and an AI engine.

[0147] Collection of user information

[0148] The server collects data on user actions on the e-commerce site. This data includes product click history, browsing time, search history, and purchase history. For example, if a user clicks on and views products related to "summer fashion," that history is recorded on the server.

[0149] Interest analysis and product recommendations

[0150] The AI ​​engine on the server analyzes collected user behavior data to identify the user's areas of interest. Based on these identified areas of interest, the AI ​​engine retrieves relevant products from an external database. This allows for the recommendation of products that the user is likely to be interested in.

[0151] Inspiration from photo folders

[0152] The server accesses the user's photo folder and extracts data using image analysis technology. Based on the analysis results, it automatically generates related products and articles from the user's photos. For example, it can generate "recommended travel items" based on the user's travel photos. The necessary access permissions are obtained from the user in advance.

[0153] Generating and Layout Custom Articles

[0154] The AI ​​engine automatically generates text-based content based on articles created from selected product information and photos. The generated articles are sorted according to the user's interests and compiled into a PDF format with an optimal layout. For example, articles about summer fashion are placed at the top, followed by articles about travel goods.

[0155] PDF generation and preview display

[0156] The server generates an original PDF based on the generated article and displays it as a preview on the user's device. The user can review this preview and make corrections or changes as needed. For example, if the user reviews the generated PDF and is satisfied with the content, they can save or print it as is.

[0157] Providing printing options

[0158] After reviewing the preview, users can choose to print. The server outputs the PDF in a printable format and provides a download link to the user's device. It can also directly initiate printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server generates a PDF, which the user can then print as a poster or flyer on their own printer.

[0159] For example, if a user searches for "summer fashion items" and leaves a click and browsing history, the server collects this data, and the AI ​​engine analyzes the user's interests. Based on this, the server recommends the latest summer fashion items and related news, and generates a customized PDF. This PDF may include articles such as "This Summer's Trendy Items" or "Recommended Beach Gear."

[0160] Examples of prompt statements to input into a generative AI model are as follows:

[0161] Based on the user's search history, browsing history, and purchase history, recommend products and related news articles that the user might be interested in. The following is the user's past data.

[0162] Search history: ['Summer dress', 'Flip-flops', 'Travel suitcase']

[0163] Browsing history: ['New beach sandals', 'Summer clothes made of cooling material']

[0164] Purchase history: ['Travel bag', 'Sunglasses']

[0165] Based on the data mentioned above, generate news articles related to products that users are likely to be interested in.

[0166] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0167] Step 1:

[0168] Collect user behavior data.

[0169] The server collects user actions on the e-commerce site (product clicks, browsing, search history, purchase history) and stores them in a database. The collected data includes product IDs, operation times, and operation details. This reveals user behavior patterns.

[0170] Step 2:

[0171] Analyze the user's areas of interest.

[0172] The server sends the collected user behavior data to the AI ​​engine for analysis. The AI ​​engine uses TF-IDF vectorization and KMeans clustering techniques to identify categories and topics of high user interest. The input is user behavior data, and the output is cluster data related to areas of interest.

[0173] Step 3:

[0174] Select related products.

[0175] The AI ​​engine selects products related to the user's areas of interest from an external database based on the analysis results. For example, if it determines that the user is interested in summer fashion, it will retrieve the latest information on summer items. The input is cluster data of areas of interest, and the output is a list of recommended products.

[0176] Step 4:

[0177] Analyze the photo folder.

[0178] The server accesses the photo folder stored on the user's device and extracts data using image analysis technology. Specifically, it analyzes image files to generate tags, and then automatically generates related products and articles based on that tag information. The input is the user's photo data, and the output is image tags and their associated product information.

[0179] Step 5:

[0180] Automatically generate articles.

[0181] The server uses an AI engine to generate text-based content based on selected product information and photo analysis results. For example, it can automatically create articles introducing new flip-flops or recommending travel goods. The input is product information and image tag information, and the output is customized text content.

[0182] Step 6:

[0183] Layout the generated articles.

[0184] The server lays out the generated text content in PDF format. Articles are sorted according to user interest and compiled into a PDF with optimal layout. The input is the generated text content, and the output is a pre-layout PDF file.

[0185] Step 7:

[0186] Preview the PDF.

[0187] The terminal visually displays the generated PDF to the user, providing a preview. The user can review the content and make corrections or changes as needed. The input is a pre-layout PDF file, and the output is a preview displayed on the terminal screen.

[0188] Step 8:

[0189] Print the PDF.

[0190] After the user reviews the preview, they select the print option. The server outputs the PDF in a printable format, and the user's device initiates printing to the printer. The input is the previewed PDF file, and the output is the printed document.

[0191] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0192] This invention is a system that collects user behavior data, analyzes the user's areas of interest based on that data, and automatically generates a customized newspaper. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves even more personalized article generation. This system consists of a server, a user terminal, and an AI engine.

[0193] Collection of user information

[0194] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, that history is saved in the database.

[0195] Specific example:

[0196] When a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[0197] Emotion recognition by an emotion engine

[0198] The server collects user emotion data through emotion recognition devices (such as cameras and microphones) connected to the user's terminal. The emotion engine recognizes the user's emotions in real time using facial expression analysis and voice analysis. For example, it analyzes the user's facial expressions and tone of voice while they are reading an article to determine whether the user is in an emotional state such as happy, sad, or interested.

[0199] Specific example:

[0200] When a user is reading an article about environmental issues, the camera captures the user's facial expressions, and an emotion engine analyzes those expressions to recognize that the user finds them interesting.

[0201] Interest analysis and news selection

[0202] An AI engine on the server analyzes behavioral and emotional data to identify the user's areas of interest and emotional state. Based on this analysis, it retrieves relevant and up-to-date information from the news API. For example, if a user has a strong interest in environmental issues and shows positive emotions towards that topic, the AI ​​engine will select environmental articles that are relevant to the user.

[0203] Specific example:

[0204] The AI ​​engine retrieves the latest environmental articles from the news API if the user is interested in environmental issues and expresses positive feelings towards them.

[0205] Generating content from photo folders

[0206] The server accesses the user's photo folder and extracts data using image analysis technology. The server obtains the necessary access permissions from the user in advance. For example, the server can detect the user's travel photos and use them to generate article ideas.

[0207] Specific example:

[0208] The server detects the user's travel photos and generates a "user's travelogue" based on those photos.

[0209] Automatic article generation and layout

[0210] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. The generated articles are created using natural language generation (NLG) technology and include sentences and expressions based on specific sentiment data. Furthermore, the generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[0211] Specific example:

[0212] The AI ​​engine generates an article titled "The Future of the Environment" in a positive tone, based on the positive emotions of the user recognized by the emotion engine, and places it at the top.

[0213] Newspaper generation and preview display

[0214] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[0215] Specific example:

[0216] The server generates the newspaper as an A4-sized PDF and displays a preview on the user's device. The user checks the preview and, if satisfied with the content, saves or prints it.

[0217] Providing printing options

[0218] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0219] Specific example:

[0220] The user reviews the previewed content and clicks the "Print" button. This causes the server to download the newspaper as a PDF and print it from the user's printer.

[0221] As described above, the system of the present invention significantly reduces the effort required from the user, enabling them to easily create, share, and save customized newspapers. Furthermore, by recognizing the user's emotions and generating articles based on those emotions, it can provide a more personalized experience.

[0222] The following describes the processing flow.

[0223] Step 1:

[0224] The server collects user behavior data. This includes click data on articles the user views, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, the article ID, viewing start time, and viewing end time are stored in the database.

[0225] Step 2:

[0226] The server collects user emotion data through emotion recognition devices (such as cameras and microphones) connected to the user's terminal. The emotion engine recognizes the user's emotions in real time using facial expression analysis and voice analysis. For example, it analyzes the user's facial expressions and tone of voice while they are reading an article to determine whether the user is in an emotional state such as happy, sad, or interested.

[0227] Step 3:

[0228] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. This analysis uses machine learning algorithms and natural language processing techniques. For example, if a user reads many articles about environmental issues and expresses positive emotions while doing so, the AI ​​engine will identify that the user is interested in and has a favorable view of environmental issues.

[0229] Step 4:

[0230] The AI ​​engine retrieves relevant and up-to-date information from a news API based on identified areas of interest and emotional state. The news API is a service that provides the latest news articles from various sources on the internet. For example, it retrieves the latest articles on environmental issues and prioritizes selecting those with positive content.

[0231] Step 5:

[0232] The server accesses the user's photo folder and scans the image data within the folder using image analysis technology. The server obtains the necessary access permissions from the user beforehand. For example, the server can detect the user's travel photos and use them to generate article ideas.

[0233] Step 6:

[0234] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. The generated articles are created using natural language generation (NLG) technology. For example, if a user is interested in environmental issues and expresses positive feelings towards them, the AI ​​engine will create an article about environmental issues in a positive tone.

[0235] Step 7:

[0236] The AI ​​engine applies an algorithm to sort the generated articles according to the user's interests. This places the articles the user is most interested in at the top. For example, articles on environmental issues might be placed at the top of the newspaper, followed by travelogues.

[0237] Step 8:

[0238] The server generates a PDF or web version of the original newspaper based on the rearranged articles. The generated newspaper is displayed as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[0239] Step 9:

[0240] Users can view a preview of the newspaper and select the option to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0241] By following these steps, users can easily create, preview, and print a customized newspaper based on their interests and feelings.

[0242] (Example 2)

[0243] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0244] Traditional systems provided personalized content based solely on user behavior data, making it difficult to reflect user emotions or subtle shifts in interests. Furthermore, the lack of a function to generate content from photo folders prevented the delivery of more personalized articles. Additionally, there was a lack of easy ways to print the generated content.

[0245] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0246] In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for collecting and analyzing user sentiment data, means for automatically generating articles based on selected information and sentiment data, and means for laying out the generated articles and outputting them as a newspaper. This makes it possible to utilize both user behavior data and sentiment data to generate more accurate personalized articles and to provide newspapers with content that meets the individual needs of the user.

[0247] "User behavior data" refers to information about the actions and interactions that users perform on the application, and specifically includes article click data, viewing time, search history, and interactions with followers.

[0248] "Areas of interest" refer to the fields or topics that a user is particularly interested in, and are identified from behavioral and emotional data.

[0249] "Emotional data" refers to information about a user's emotional state collected from their facial expressions and voice, including their facial expressions and tone of voice while reading an article.

[0250] "Automatic article generation" refers to the process by which an AI engine automatically creates articles using natural language generation (NLG) technology based on collected and analyzed data.

[0251] "Layout" refers to the process of arranging generated articles in a specific order and format, and includes the process of compiling them into an optimal newspaper format.

[0252] "Outputting as a newspaper" means providing the generated articles to the user's device as a newspaper in PDF or web format.

[0253] "Access permission" refers to the permission a server has to access specific data on a user's device (for example, their photo folder), and this permission is approved by the user in advance.

[0254] A "News API" is an application programming interface used to retrieve the latest news information.

[0255] Natural Language Generation (NLG) is a technology that uses an AI engine to mechanically generate data and output it in a natural language format.

[0256] The "print option" is a feature that outputs the generated newspaper in a printable format (e.g., PDF), allowing the user to physically print it.

[0257] Modes for carrying out the invention

[0258] This invention relates to a system that automatically generates personalized newspapers using user behavioral data and emotional data. This system consists of multiple components, including a server, a user terminal, and an AI engine.

[0259] Collection of user behavior data

[0260] The server collects data on user actions within the application. This data includes article click data, viewing time, search history, and interactions with followers. For example, if a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[0261] Collection of emotional data

[0262] The server collects user emotion data in real time using emotion recognition devices (such as cameras and microphones) connected to the user's device. The emotion engine recognizes the user's emotions using facial expression analysis and voice analysis and stores them in a database. For example, by analyzing the user's facial expressions and tone of voice while reading an article, it can recognize that the user is finding it interesting.

[0263] Analysis of user areas of interest

[0264] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Based on this analysis, it retrieves relevant and up-to-date information from the news API. For example, if a user has a strong interest in environmental issues and also shows positive emotions towards that topic, the AI ​​engine will select environmental articles that are relevant to the user.

[0265] Generating content from photo folders

[0266] The server uses access permissions obtained from the user in advance to access the user's photo folder and extracts data using image analysis technology. For example, the server detects the user's travel photos and generates article ideas for a "user's travelogue" based on those photos.

[0267] Automatic article generation and layout

[0268] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. This process utilizes natural language generation (NLG) technology. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[0269] Newspaper generation and preview display

[0270] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make any necessary corrections or changes. For example, the server generates the newspaper in A4 size PDF format and displays it as a preview on the user's device. The user reviews the content and, if satisfied, saves or prints it.

[0271] Providing printing options

[0272] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format (PDF) and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server provides a download link for the newspaper as a PDF, and the user prints it directly from their printer.

[0273] Example of a prompt

[0274] "If a user is interested in 'environmental protection' and shows positive sentiments, the system will retrieve the latest environmental news and generate articles with a positive tone."

[0275] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0276] Step 1:

[0277] The server collects user behavior data. Specifically, it collects data on the actions users perform on the application and stores it in a database. Inputs include user actions such as clicking on articles or searching, and based on these, click data, viewing time, search history, etc., are recorded on the server. For example, if a user searches for an article using the search term "environmental protection," clicks on it, and views it for 10 minutes, the search term, article ID, and viewing time will be recorded.

[0278] Step 2:

[0279] The server collects emotional data using emotion recognition devices (camera and microphone) connected to the user's device. The input includes the user's facial expressions and voice while they are reading an article, and the emotion engine analyzes this to determine their emotional state. Specifically, the camera captures the user's smile while they are reading an article, and the emotion engine recognizes this data as a "positive emotion" and outputs it.

[0280] Step 3:

[0281] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Using the collected behavioral and emotional data as input, it performs data calculations to analyze the user's interests. For example, if the analysis reveals that the user is interested in environmental issues and exhibits positive emotions, the AI ​​engine will use this to identify relevant recent articles on environmental issues as output.

[0282] Step 4:

[0283] The server accesses the photo folder and extracts data using image analysis technology. The input is the photo data in the user's photo folder, which the server analyzes. Specifically, the server detects photos of "Paris trip" from the user's photo folder and generates story ideas for an article titled "User's Travelogue" based on these photos.

[0284] Step 5:

[0285] The AI engine automatically generates articles based on the information obtained from the news API, the story ideas generated from the photo folder, and the sentiment data. The input includes the latest information obtained from the news API and various collected data. The AI engine combines these data and uses natural language generation (NLG) technology to generate articles. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout as the output. For example, based on the positive sentiment of the user recognized by the sentiment engine, an article titled "The Future of the Environment" is generated in a positive tone and placed at the top of the newspaper.

[0286] Step 6:

[0287] The server generates a PDF or web format of the original newspaper based on the generated articles and displays it as a preview on the user's terminal. The input is the article data generated by the AI engine, and a newspaper-format file is generated based on this. For example, the server displays a newspaper generated in A4 PDF format on the user's terminal. The user can view the preview and change the order of the articles.

[0288] Step 7:

[0289] After reviewing the previewed newspaper, the user can select a print option. The server outputs the newspaper in a printable format (PDF) and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer. A generated PDF file is provided as input, and when the user clicks the "Print" button, the server provides the PDF as a download link, and printing is initiated directly from the user's printer.

[0290] (Application Example 2)

[0291] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0292] Traditional systems that effectively utilize user behavior data and emotional states to provide information tailored to user interests and emotions are incomplete and require improvement to enhance the user experience. Furthermore, personalized content such as music and news needs to be enhanced to increase user satisfaction.

[0293] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for collecting and recognizing user emotion data, means for selecting and generating information based on the collected and recognized emotion data, means for automatically generating articles and content based on the selected information, and means for laying out and outputting the generated articles and content. This makes it possible to provide personalized information based on the user's interests and emotions.

[0294] "Behavioral data" refers to all records of operations and activities performed by users on the system.

[0295] "Areas of interest" refers to specific fields or topics that a user is interested in, derived from their behavioral data.

[0296] "Emotional data" refers to data that indicates the emotional state of a user, as recognized from their facial expressions, voice, and other factors.

[0297] "Emotion recognition means" refers to technology that uses cameras and microphones to analyze a user's emotional state in real time.

[0298] "Information selection means" refers to the function of a system that selects the most suitable content based on user behavior data and emotional data.

[0299] "Automatic article generation method" refers to a technology that automatically creates articles using natural language generation technology based on selected information.

[0300] "Layout techniques" refer to the methods used to arrange generated articles and content in an easy-to-read format and prepare them for the final output.

[0301] Modes for carrying out the invention

[0302] The present invention is a system that collects user behavioral and emotional data and automatically generates personalized content based on that data. Specific embodiments for carrying out the present invention are described below.

[0303] Collection of behavioral data

[0304] The server collects behavioral data from the user's device, such as operation logs, playback history, browsing time, search history, and likes and skips. This data is used to analyze the user's areas of interest. For example, if a user frequently listens to music in the "rock" genre, that information is stored by the server.

[0305] Collection and recognition of emotional data

[0306] The server collects the user's emotion data using the camera and microphone connected to the user's terminal. The emotion engine analyzes the user's expressions and voice in real time using specific software (e.g., EmotionRecognizer) to recognize the user's emotional state. For example, when the user has a smiling expression while listening to a specific piece of music, that information is recorded as emotion data.

[0307] Analysis of Areas of Interest and Emotional States

[0308] The collected behavioral data and emotion data are stored in a database accumulated within the server. The AI engine within the server analyzes these data to identify the user's areas of interest and emotional states. This includes what themes the user is interested in and in what emotional states the user enjoys those themes.

[0309] Selection and Automatic Generation of Content

[0310] The server obtains relevant information from news APIs and music libraries based on the analysis results. Based on the obtained information, it automatically generates articles and playlists using natural language generation (NLG) technology. Specifically, it uses software such as MusicRecommender to create a playlist that recommends songs optimal for the user's emotional state. For example, when the user is exercising with an energetic emotion, it generates a playlist of upbeat music.

[0311] Output and Preview Display

[0312] The generated articles and playlists are displayed in preview format on the user's terminal. The user can view the generated content and make corrections or changes as needed. For example, if the user is satisfied with the generated playlist, they can play it as is.

[0313] Provision of Printing and Download Options

[0314] User-created content is output in a printable format. The server generates the content in PDF format and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer.

[0315] Example of a prompt

[0316] "Please describe a program that analyzes a user's emotions while they are listening to music and generates the optimal playlist for them."

[0317] Through the above processing, the present invention realizes the provision of personalized content based on user behavioral data and emotional data. By using this system, users can easily obtain content that matches their interests and emotions.

[0318] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0319] Step 1:

[0320] The server collects behavioral data from the user's device. Specifically, it collects data on songs played by the user on the application, playback time, skipped songs, search history, and actions such as liking and sharing, and stores this data in a database. Input data includes operation logs and playback history, while output is behavioral data that serves as the basis for analysis.

[0321] Step 2:

[0322] The server collects and recognizes user emotion data using the terminal's camera and microphone. Specifically, it uses EmotionRecognizer software to analyze the user's facial expressions and voice in real time while they are playing music, and recognizes their emotional state (joy, sadness, excitement, etc.). The input data is real-time facial expressions and voice, and the output is the recognized emotion data.

[0323] Step 3:

[0324] The server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Specifically, it uses an AI engine to analyze what music genres the user is interested in and under what emotional state they enjoy them. The input data consists of behavioral and emotional data, and the output is data related to the user's areas of interest and emotional state.

[0325] Step 4:

[0326] The server selects and automatically generates relevant content based on the analysis results. Specifically, it uses MusicRecommender software to generate a playlist containing songs best suited to the user's emotional state. The input data includes areas of interest and emotional state data, and the output is the generated playlist.

[0327] Step 5:

[0328] The server displays a preview of the generated playlist on the user's device. Specifically, it sends the generated playlist to the device in real time and displays it within the application. The input data is the generated playlist, and the output is the previewed playlist.

[0329] Step 6:

[0330] Users can review a previewed playlist and make corrections or changes as needed. Specifically, they can add, delete, or rearrange songs within the playlist. The input data is the previewed playlist, and the output is the playlist modified by the user.

[0331] Step 7:

[0332] The server outputs the user-selected playlist in a printable format. Specifically, it generates the playlist in PDF format and provides a download link to the user's device. The input data is a playlist modified or changed by the user, and the output is a PDF playlist file.

[0333] Through these processing steps, users can easily enjoy personalized content based on their interests and emotions.

[0334] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0335] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0336] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0337] [Second Embodiment]

[0338] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0339] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0340] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0341] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0342] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0343] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0344] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0345] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0346] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0347] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0348] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0349] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0350] This invention relates to a system that automatically generates customized newspapers by collecting user behavior data and analyzing the user's areas of interest based on that data. The system consists of a server, a user terminal, and an AI engine.

[0351] Collection of user information

[0352] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, that history is saved in the database.

[0353] Specific example:

[0354] When a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[0355] Interest analysis and news selection

[0356] An AI engine on the server analyzes collected user behavior data to identify the user's areas of interest. Based on these identified areas of interest, the AI ​​engine retrieves relevant and up-to-date information from an external news API. This selects articles that are likely to be of interest to the user.

[0357] Specific example:

[0358] The AI ​​engine analyzes that the user has a strong interest in environmental issues and retrieves the latest environmental articles from the news API.

[0359] Generating content from photo folders

[0360] The server accesses the user's photo folder and extracts data using image analysis technology. Based on the analysis results, it automatically generates relevant articles from the user's photos. The necessary access permissions are obtained from the user in advance.

[0361] Specific example:

[0362] The server detects the user's travel photos and automatically generates a "user's travelogue" based on them.

[0363] Automatic article generation and layout

[0364] The AI ​​engine automatically generates text-based content based on articles created from selected information and photos. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[0365] Specific example:

[0366] The AI ​​engine generates articles on environmental issues and travelogues, and arranges them according to the user's interests. For example, it might place articles on environmental issues at the top of the newspaper, followed by travelogues.

[0367] Newspaper generation and preview display

[0368] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make corrections or changes as needed.

[0369] Specific example:

[0370] The server generates the newspaper as an A4-sized PDF and displays a preview on the user's device. The user checks the preview and, if satisfied with the content, saves or prints it.

[0371] Providing printing options

[0372] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0373] Specific example:

[0374] The user reviews the previewed content and clicks the "Print" button. This causes the server to download the newspaper as a PDF and print it from the user's printer.

[0375] As described above, the system of the present invention significantly reduces the effort required from the user, making it possible to easily create, share, and save customized newspapers.

[0376] The following describes the processing flow.

[0377] Step 1:

[0378] The server collects user behavior data. This includes click data on articles the user views, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, the article ID, viewing start time, and viewing end time are stored in the database.

[0379] Step 2:

[0380] The AI ​​engine on the server analyzes collected behavioral data to identify the user's areas of interest. This analysis uses machine learning algorithms and natural language processing techniques. For example, if a user reads many articles about environmental issues, the AI ​​engine will identify that the user is interested in environmental issues.

[0381] Step 3:

[0382] The AI ​​engine retrieves relevant and up-to-date information from a news API based on identified areas of interest. The news API is a service that provides the latest news articles from various sources on the internet. For example, it retrieves the latest articles on environmental issues.

[0383] Step 4:

[0384] The server accesses the user's photo folder and scans the image data within the folder using image analysis technology. The server obtains the necessary access permissions from the user beforehand. For example, the server can detect the user's travel photos and generate article ideas based on them.

[0385] Step 5:

[0386] The AI ​​engine automatically generates articles based on information obtained from a news API and ideas generated from photo folders. The generated articles are created using natural language generation (NLG) technology. For example, it can automatically generate both news articles on environmental issues and travelogues.

[0387] Step 6:

[0388] The AI ​​engine applies an algorithm to sort the generated articles according to the user's interests. This places the articles the user is most interested in at the top. For example, articles on environmental issues might be placed at the top of the newspaper, followed by travelogues.

[0389] Step 7:

[0390] The server generates a PDF or web version of the original newspaper based on the rearranged articles. The generated newspaper is displayed as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[0391] Step 8:

[0392] Users can view a preview of the newspaper and select the option to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0393] By following these steps, users can easily create, preview, and print a customized newspaper based on their interests.

[0394] (Example 1)

[0395] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0396] Current news delivery methods make it difficult to effectively provide information tailored to individual user interests and concerns, requiring users to sift through a large amount of information themselves. As a result, users waste time and effort, and their information consumption becomes incomplete. Furthermore, there are limitations to the means of easily saving and sharing generated information in physical format.

[0397] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0398] In this invention, the server includes means for collecting operation data performed on the user's terminal, means for analyzing the collected operation data and identifying the user's areas of interest, means for acquiring relevant information from an external data source based on the identified areas of interest, means for generating text content based on the acquired information, means for rearranging the generated text content in order of the user's interests and laying it out in newspaper format, and means for previewing the laid-out newspaper on an electronic device. This makes it possible to automatically generate a customized newspaper according to the user's interests, enabling them to effectively enjoy information while significantly reducing time and effort.

[0399] "User operation data" refers to data recorded on the user's device, such as click data, browsing time, search history, and interactions.

[0400] "Areas of interest" refer to the fields or topics that a user is primarily interested in, and are analyzed from the user's behavioral data.

[0401] "External data sources" refer to external news APIs and databases used to retrieve relevant information.

[0402] "Text content" refers to text-based content generated by an AI engine, including news articles and essays.

[0403] "Newspaper format" refers to a layout in which generated text content is neatly arranged and presented in an easy-to-read format.

[0404] "Electronic devices" refers to display devices such as smartphones, tablets, and personal computers.

[0405] "Image analysis technology" refers to techniques that extract information from image data using computer vision and machine learning algorithms.

[0406] "Preview display" refers to a function that allows users to check the generated content, such as newspapers, in its final output format beforehand.

[0407] A "printable format" refers to a format suitable for physical printing (for example, PDF or DOC format).

[0408] Modes for carrying out the invention

[0409] This invention relates to a system that collects user behavior data, analyzes the user's areas of interest based on that data, and automatically generates a customized newspaper. The system consists of a server, a user terminal, and a generation AI model.

[0410] Collection of user information

[0411] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. Specifically, if a user searches for "environmental protection," clicks on an article, and reads it, the click data and viewing time are collected and stored in the database. MySQL or PostgreSQL are used as the database.

[0412] Interest analysis and news selection

[0413] An AI engine on the server analyzes collected user behavior data to identify the user's areas of interest. The AI ​​engine runs using Python libraries (e.g., scikit-learn and TensorFlow). Once areas of interest are identified, the server retrieves relevant and up-to-date information through news APIs (e.g., Google News API or NewsAPI). Specifically, if the AI ​​engine analyzes that the user has a strong interest in environmental issues, it will retrieve the latest articles on environmental issues from the news API.

[0414] Generating content from photo folders

[0415] The server accesses the user's photo folder and extracts data using image analysis techniques. Necessary access permissions are obtained from the user in advance. OpenCV and TensorFlow libraries are used for image analysis. For example, if the user's travel photos are detected, related articles such as "User's Travelogue" are automatically generated based on them.

[0416] Automatic article generation and layout

[0417] The AI ​​engine automatically generates text-based content from selected information and photos. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout. This uses natural language generation technologies such as GPT-3 and BERT. For example, if an article on environmental issues and a travelogue are generated, the article on environmental issues will be placed at the top, followed by the travelogue.

[0418] Newspaper generation and preview display

[0419] The server generates the original newspaper in PDF or HTML format based on the generated articles and displays it as a preview on the user's device. Specifically, it generates the layout using LaTeX or HTML templates and displays it on the user's smartphone or PC. The user can check this preview and make corrections or changes as needed.

[0420] Providing printing options

[0421] After reviewing the previewed newspaper, users can select a print option. The server outputs the newspaper in a printable format (e.g., PDF) and provides a download link to the user's device. It can also initiate direct printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server generates the newspaper as a PDF, sends it to the user's printer, and prints it.

[0422] Examples of prompt statements

[0423] Prompt example 1:

[0424] "Automatically generate news articles based on topics that users are recently interested in. For example, if a user is interested in environmental protection, retrieve the latest environmental news."

[0425] Prompt example 2:

[0426] "Please analyze travel photos from the user's photo folder and automatically generate a travelogue based on that analysis. For example, if there are many photos of Paris, please generate a travelogue about Paris."

[0427] Prompt example 3:

[0428] "Create a customized newspaper based on articles of interest to the user and display it as a preview in PDF format. For example, if the user is interested in environmental issues, create a newspaper focusing on environmentally related articles."

[0429] As a result, the system of the present invention significantly reduces the effort required from the user, making it possible to easily create, share, and save customized newspapers.

[0430] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0431] Step 1:

[0432] The user logs into the application and generates operation data on their device. Inputs include the user's login operation and article clicks and search actions within the application. Outputs are the generated operation data, which includes click data, viewing time, search history, and interactions. Specifically, the user accesses the application using a smartphone or PC, searches for an article using the keyword "environmental protection," and clicks to read it.

[0433] Step 2:

[0434] The server collects user activity data and stores it in a database. The input is the activity data generated in step 1. The output is the activity data stored in the database. Specifically, the server collects user click data, browsing time, and search history, and stores the data in a database such as MySQL or PostgreSQL.

[0435] Step 3:

[0436] An AI engine on the server analyzes collected interaction data to identify the user's areas of interest. The input is the interaction data stored in the database. The output is the identified areas of interest. Specifically, the AI ​​engine uses Python libraries (such as scikit-learn or TensorFlow) to analyze the interaction data and identify that the user is interested in environmental issues.

[0437] Step 4:

[0438] The server retrieves relevant and up-to-date information from external data sources based on identified areas of interest. The input is the user's areas of interest. The output is the retrieved latest information (articles). Specifically, the server uses a news API (e.g., Google News API or NewsAPI) to retrieve the latest articles on environmental issues of interest.

[0439] Step 5:

[0440] The server accesses the user's photo folder, extracts data using image analysis techniques, and automatically generates related articles. The input is the user's photo data. The output is the image analysis results and the automatically generated articles. Specifically, the server uses OpenCV or TensorFlow to analyze travel photos and generates a "user travelogue" based on that analysis. Access permission to the photo folder is obtained from the user beforehand.

[0441] Step 6:

[0442] An AI engine on the server automatically generates text content based on articles created from selected information and photos, and then arranges them in a newspaper format, sorted according to the user's interests. The input is the latest acquired information and image analysis results. The output is a pre-layout newspaper-style article. Specifically, the AI ​​engine uses GPT-3 and BERT to generate articles, placing articles on environmental issues at the top and travelogues next.

[0443] Step 7:

[0444] The server generates a newspaper and displays a preview of it on the user's device. The input is a pre-layout newspaper article. The output is the preview of the newspaper displayed on the device. Specifically, the server generates a newspaper in PDF or HTML format using LaTeX or HTML templates and displays a preview of it on the user's smartphone or PC.

[0445] Step 8:

[0446] The user reviews the previewed newspaper and selects a print option. The server outputs the newspaper in a printable format and provides a download link. It also initiates printing directly with the user's printer. The input is the previewed newspaper. The output is the newspaper in a printable format and a download link. Specifically, when the user clicks the "Print" button, the server generates the newspaper in PDF format, sends it to the printer, and performs the printing.

[0447] (Application Example 1)

[0448] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0449] In recent years, there has been a growing demand for customized services based on users' purchasing behavior and interests, but meeting this demand requires advanced data analysis and automated generation technologies. In particular, e-commerce sites lack mechanisms to efficiently recommend products likely to interest users and provide related news and information. Existing systems fail to fully utilize user behavior data, making it difficult to provide information tailored to individual users.

[0450] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0451] In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for automatically selecting relevant information based on the analysis results, means for automatically generating articles based on the selected information, means for laying out the generated articles and outputting them as a newspaper, means for recommending products on an e-commerce site based on the user's behavior data, and means for outputting articles generated based on the user's areas of interest in PDF format. This makes it possible to provide customized information and product recommendations that are tailored to the interests and purchasing behavior of individual users, thereby improving user satisfaction and increasing purchasing intent.

[0452] "User behavior data" refers to data on actions taken by users on applications and websites, such as click history, browsing history, search history, and purchase history.

[0453] "Areas of interest" refers to the fields or topics that a user is particularly interested in, as determined by analyzing their behavioral data.

[0454] "Relevant information" refers to information that is valuable to the user, such as news articles and product information selected based on the user's areas of interest.

[0455] "Methods for automatically generating articles" refers to algorithms and systems that generate text based on selected relevant information.

[0456] "Methods for outputting as a newspaper" refers to a system that automatically generates articles and lays them out in a specific format (e.g., PDF or web page) for output.

[0457] An "online shopping site" refers to a website that sells goods and services via the internet.

[0458] "A means of recommending products" refers to a system that selects and displays products that a user is likely to be interested in, based on their behavioral data.

[0459] "Methods for outputting in PDF format" refers to systems that save or display generated articles or information as files in PDF format.

[0460] This invention is a system that collects user behavior data and recommends customized information and products based on that data. Specifically, it consists of a server, a user terminal, and an AI engine.

[0461] Collection of user information

[0462] The server collects data on user actions on the e-commerce site. This data includes product click history, browsing time, search history, and purchase history. For example, if a user clicks on and views products related to "summer fashion," that history is recorded on the server.

[0463] Interest analysis and product recommendations

[0464] The AI ​​engine on the server analyzes collected user behavior data to identify the user's areas of interest. Based on these identified areas of interest, the AI ​​engine retrieves relevant products from an external database. This allows for the recommendation of products that the user is likely to be interested in.

[0465] Inspiration from photo folders

[0466] The server accesses the user's photo folder and extracts data using image analysis technology. Based on the analysis results, it automatically generates related products and articles from the user's photos. For example, it can generate "recommended travel items" based on the user's travel photos. The necessary access permissions are obtained from the user in advance.

[0467] Generating and Layout Custom Articles

[0468] The AI ​​engine automatically generates text-based content based on articles created from selected product information and photos. The generated articles are sorted according to the user's interests and compiled into a PDF format with an optimal layout. For example, articles about summer fashion are placed at the top, followed by articles about travel goods.

[0469] PDF generation and preview display

[0470] The server generates an original PDF based on the generated article and displays it as a preview on the user's device. The user can review this preview and make corrections or changes as needed. For example, if the user reviews the generated PDF and is satisfied with the content, they can save or print it as is.

[0471] Providing printing options

[0472] After reviewing the preview, users can choose to print. The server outputs the PDF in a printable format and provides a download link to the user's device. It can also directly initiate printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server generates a PDF, which the user can then print as a poster or flyer on their own printer.

[0473] For example, if a user searches for "summer fashion items" and leaves a click and browsing history, the server collects this data, and the AI ​​engine analyzes the user's interests. Based on this, the server recommends the latest summer fashion items and related news, and generates a customized PDF. This PDF may include articles such as "This Summer's Trendy Items" or "Recommended Beach Gear."

[0474] Examples of prompt statements to input into a generative AI model are as follows:

[0475] Based on the user's search history, browsing history, and purchase history, recommend products and related news articles that the user might be interested in. The following is the user's past data.

[0476] Search history: ['Summer dress', 'Flip-flops', 'Travel suitcase']

[0477] Browsing history: ['New beach sandals', 'Summer clothes made of cooling material']

[0478] Purchase history: ['Travel bag', 'Sunglasses']

[0479] Based on the data mentioned above, generate news articles related to products that users are likely to be interested in.

[0480] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0481] Step 1:

[0482] Collect user behavior data.

[0483] The server collects user actions on the e-commerce site (product clicks, browsing, search history, purchase history) and stores them in a database. The collected data includes product IDs, operation times, and operation details. This reveals user behavior patterns.

[0484] Step 2:

[0485] Analyze the user's areas of interest.

[0486] The server sends the collected user behavior data to the AI ​​engine for analysis. The AI ​​engine uses TF-IDF vectorization and KMeans clustering techniques to identify categories and topics of high user interest. The input is user behavior data, and the output is cluster data related to areas of interest.

[0487] Step 3:

[0488] Select related products.

[0489] The AI ​​engine selects products related to the user's areas of interest from an external database based on the analysis results. For example, if it determines that the user is interested in summer fashion, it will retrieve the latest information on summer items. The input is cluster data of areas of interest, and the output is a list of recommended products.

[0490] Step 4:

[0491] Analyze the photo folder.

[0492] The server accesses the photo folder stored on the user's device and extracts data using image analysis technology. Specifically, it analyzes image files to generate tags, and then automatically generates related products and articles based on that tag information. The input is the user's photo data, and the output is image tags and their associated product information.

[0493] Step 5:

[0494] Automatically generate articles.

[0495] The server uses an AI engine to generate text-based content based on selected product information and photo analysis results. For example, it can automatically create articles introducing new flip-flops or recommending travel goods. The input is product information and image tag information, and the output is customized text content.

[0496] Step 6:

[0497] Layout the generated articles.

[0498] The server lays out the generated text content in PDF format. Articles are sorted according to user interest and compiled into a PDF with optimal layout. The input is the generated text content, and the output is a pre-layout PDF file.

[0499] Step 7:

[0500] Preview the PDF.

[0501] The terminal visually displays the generated PDF to the user, providing a preview. The user can review the content and make corrections or changes as needed. The input is a pre-layout PDF file, and the output is a preview displayed on the terminal screen.

[0502] Step 8:

[0503] Print the PDF.

[0504] After the user reviews the preview, they select the print option. The server outputs the PDF in a printable format, and the user's device initiates printing to the printer. The input is the previewed PDF file, and the output is the printed document.

[0505] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0506] This invention is a system that collects user behavior data, analyzes the user's areas of interest based on that data, and automatically generates a customized newspaper. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves even more personalized article generation. This system consists of a server, a user terminal, and an AI engine.

[0507] Collection of user information

[0508] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, that history is saved in the database.

[0509] Specific example:

[0510] When a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[0511] Emotion recognition by an emotion engine

[0512] The server collects user emotion data through emotion recognition devices (such as cameras and microphones) connected to the user's terminal. The emotion engine recognizes the user's emotions in real time using facial expression analysis and voice analysis. For example, it analyzes the user's facial expressions and tone of voice while they are reading an article to determine whether the user is in an emotional state such as happy, sad, or interested.

[0513] Specific example:

[0514] When a user is reading an article about environmental issues, the camera captures the user's facial expressions, and an emotion engine analyzes those expressions to recognize that the user finds them interesting.

[0515] Interest analysis and news selection

[0516] An AI engine on the server analyzes behavioral and emotional data to identify the user's areas of interest and emotional state. Based on this analysis, it retrieves relevant and up-to-date information from the news API. For example, if a user has a strong interest in environmental issues and shows positive emotions towards that topic, the AI ​​engine will select environmental articles that are relevant to the user.

[0517] Specific example:

[0518] The AI ​​engine retrieves the latest environmental articles from the news API if the user is interested in environmental issues and expresses positive feelings towards them.

[0519] Generating content from photo folders

[0520] The server accesses the user's photo folder and extracts data using image analysis technology. The server obtains the necessary access permissions from the user in advance. For example, the server can detect the user's travel photos and use them to generate article ideas.

[0521] Specific example:

[0522] The server detects the user's travel photos and generates a "user's travelogue" based on those photos.

[0523] Automatic article generation and layout

[0524] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. The generated articles are created using natural language generation (NLG) technology and include sentences and expressions based on specific sentiment data. Furthermore, the generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[0525] Specific example:

[0526] The AI ​​engine generates an article titled "The Future of the Environment" in a positive tone, based on the positive emotions of the user recognized by the emotion engine, and places it at the top.

[0527] Newspaper generation and preview display

[0528] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[0529] Specific example:

[0530] The server generates the newspaper as an A4-sized PDF and displays a preview on the user's device. The user checks the preview and, if satisfied with the content, saves or prints it.

[0531] Providing printing options

[0532] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0533] Specific example:

[0534] The user reviews the previewed content and clicks the "Print" button. This causes the server to download the newspaper as a PDF and print it from the user's printer.

[0535] As described above, the system of the present invention significantly reduces the effort required from the user, enabling them to easily create, share, and save customized newspapers. Furthermore, by recognizing the user's emotions and generating articles based on those emotions, it can provide a more personalized experience.

[0536] The following describes the processing flow.

[0537] Step 1:

[0538] The server collects user behavior data. This includes click data on articles the user views, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, the article ID, viewing start time, and viewing end time are stored in the database.

[0539] Step 2:

[0540] The server collects user emotion data through emotion recognition devices (such as cameras and microphones) connected to the user's terminal. The emotion engine recognizes the user's emotions in real time using facial expression analysis and voice analysis. For example, it analyzes the user's facial expressions and tone of voice while they are reading an article to determine whether the user is in an emotional state such as happy, sad, or interested.

[0541] Step 3:

[0542] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. This analysis uses machine learning algorithms and natural language processing techniques. For example, if a user reads many articles about environmental issues and expresses positive emotions while doing so, the AI ​​engine will identify that the user is interested in and has a favorable view of environmental issues.

[0543] Step 4:

[0544] The AI ​​engine retrieves relevant and up-to-date information from a news API based on identified areas of interest and emotional state. The news API is a service that provides the latest news articles from various sources on the internet. For example, it retrieves the latest articles on environmental issues and prioritizes selecting those with positive content.

[0545] Step 5:

[0546] The server accesses the user's photo folder and scans the image data within the folder using image analysis technology. The server obtains the necessary access permissions from the user beforehand. For example, the server can detect the user's travel photos and use them to generate article ideas.

[0547] Step 6:

[0548] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. The generated articles are created using natural language generation (NLG) technology. For example, if a user is interested in environmental issues and expresses positive feelings towards them, the AI ​​engine will create an article about environmental issues in a positive tone.

[0549] Step 7:

[0550] The AI ​​engine applies an algorithm to sort the generated articles according to the user's interests. This places the articles the user is most interested in at the top. For example, articles on environmental issues might be placed at the top of the newspaper, followed by travelogues.

[0551] Step 8:

[0552] The server generates a PDF or web version of the original newspaper based on the rearranged articles. The generated newspaper is displayed as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[0553] Step 9:

[0554] Users can view a preview of the newspaper and select the option to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0555] By following these steps, users can easily create, preview, and print a customized newspaper based on their interests and feelings.

[0556] (Example 2)

[0557] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0558] Traditional systems provided personalized content based solely on user behavior data, making it difficult to reflect user emotions or subtle shifts in interests. Furthermore, the lack of a function to generate content from photo folders prevented the delivery of more personalized articles. Additionally, there was a lack of easy ways to print the generated content.

[0559] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0560] In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for collecting and analyzing user sentiment data, means for automatically generating articles based on selected information and sentiment data, and means for laying out the generated articles and outputting them as a newspaper. This makes it possible to utilize both user behavior data and sentiment data to generate more accurate personalized articles and to provide newspapers with content that meets the individual needs of the user.

[0561] "User behavior data" refers to information about the actions and interactions that users perform on the application, and specifically includes article click data, viewing time, search history, and interactions with followers.

[0562] "Areas of interest" refer to the fields or topics that a user is particularly interested in, and are identified from behavioral and emotional data.

[0563] "Emotional data" refers to information about a user's emotional state collected from their facial expressions and voice, including their facial expressions and tone of voice while reading an article.

[0564] "Automatic article generation" refers to the process by which an AI engine automatically creates articles using natural language generation (NLG) technology based on collected and analyzed data.

[0565] "Layout" refers to the process of arranging generated articles in a specific order and format, and includes the process of compiling them into an optimal newspaper format.

[0566] "Outputting as a newspaper" means providing the generated articles to the user's device as a newspaper in PDF or web format.

[0567] "Access permission" refers to the permission a server has to access specific data on a user's device (for example, their photo folder), and this permission is approved by the user in advance.

[0568] A "News API" is an application programming interface used to retrieve the latest news information.

[0569] Natural Language Generation (NLG) is a technology that uses an AI engine to mechanically generate data and output it in a natural language format.

[0570] The "print option" is a feature that outputs the generated newspaper in a printable format (e.g., PDF), allowing the user to physically print it.

[0571] Modes for carrying out the invention

[0572] This invention relates to a system that automatically generates personalized newspapers using user behavioral data and emotional data. This system consists of multiple components, including a server, a user terminal, and an AI engine.

[0573] Collection of user behavior data

[0574] The server collects data on user actions within the application. This data includes article click data, viewing time, search history, and interactions with followers. For example, if a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[0575] Collection of emotional data

[0576] The server collects user emotion data in real time using emotion recognition devices (such as cameras and microphones) connected to the user's device. The emotion engine recognizes the user's emotions using facial expression analysis and voice analysis and stores them in a database. For example, by analyzing the user's facial expressions and tone of voice while reading an article, it can recognize that the user is finding it interesting.

[0577] Analysis of user areas of interest

[0578] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Based on this analysis, it retrieves relevant and up-to-date information from the news API. For example, if a user has a strong interest in environmental issues and also shows positive emotions towards that topic, the AI ​​engine will select environmental articles that are relevant to the user.

[0579] Generating content from photo folders

[0580] The server uses access permissions obtained from the user in advance to access the user's photo folder and extracts data using image analysis technology. For example, the server detects the user's travel photos and generates article ideas for a "user's travelogue" based on those photos.

[0581] Automatic article generation and layout

[0582] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. This process utilizes natural language generation (NLG) technology. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[0583] Newspaper generation and preview display

[0584] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make any necessary corrections or changes. For example, the server generates the newspaper in A4 size PDF format and displays it as a preview on the user's device. The user reviews the content and, if satisfied, saves or prints it.

[0585] Providing printing options

[0586] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format (PDF) and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server provides a download link for the newspaper as a PDF, and the user prints it directly from their printer.

[0587] Example of a prompt

[0588] "If a user is interested in 'environmental protection' and shows positive sentiments, the system will retrieve the latest environmental news and generate articles with a positive tone."

[0589] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0590] Step 1:

[0591] The server collects user behavior data. Specifically, it collects data on the actions users perform on the application and stores it in a database. Inputs include user actions such as clicking on articles or searching, and based on these, click data, viewing time, search history, etc., are recorded on the server. For example, if a user searches for an article using the search term "environmental protection," clicks on it, and views it for 10 minutes, the search term, article ID, and viewing time will be recorded.

[0592] Step 2:

[0593] The server collects emotional data using emotion recognition devices (camera and microphone) connected to the user's device. The input includes the user's facial expressions and voice while they are reading an article, and the emotion engine analyzes this to determine their emotional state. Specifically, the camera captures the user's smile while they are reading an article, and the emotion engine recognizes this data as a "positive emotion" and outputs it.

[0594] Step 3:

[0595] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Using the collected behavioral and emotional data as input, it performs data calculations to analyze the user's interests. For example, if the analysis reveals that the user is interested in environmental issues and exhibits positive emotions, the AI ​​engine will use this to identify relevant recent articles on environmental issues as output.

[0596] Step 4:

[0597] The server accesses the photo folder and extracts data using image analysis technology. The input is the photo data in the user's photo folder, which the server analyzes. Specifically, it detects photos of a "trip to Paris" from the user's photo folder and uses this to generate content for an article titled "The User's Travelogue."

[0598] Step 5:

[0599] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data. Input includes the latest information obtained from the news API and various collected data. The AI ​​engine combines this data and generates articles using natural language generation (NLG) technology. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout. For example, based on the user's positive emotions recognized by the sentiment engine, an article titled "The Future of the Environment" is generated in a positive tone and placed at the top of the newspaper.

[0600] Step 6:

[0601] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The input is article data generated by an AI engine, which is used to create the newspaper file. For example, the server displays a newspaper generated in A4-sized PDF format on the user's device. The user can review the preview and change the order of the articles.

[0602] Step 7:

[0603] After reviewing the previewed newspaper, the user can select a print option. The server outputs the newspaper in a printable format (PDF) and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer. A generated PDF file is provided as input, and when the user clicks the "Print" button, the server provides the PDF as a download link, and printing is initiated directly from the user's printer.

[0604] (Application Example 2)

[0605] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0606] Traditional systems that effectively utilize user behavior data and emotional states to provide information tailored to user interests and emotions are incomplete and require improvement to enhance the user experience. Furthermore, personalized content such as music and news needs to be enhanced to increase user satisfaction.

[0607] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for collecting and recognizing user emotion data, means for selecting and generating information based on the collected and recognized emotion data, means for automatically generating articles and content based on the selected information, and means for laying out and outputting the generated articles and content. This makes it possible to provide personalized information based on the user's interests and emotions.

[0608] "Behavioral data" refers to all records of operations and activities performed by users on the system.

[0609] "Areas of interest" refers to specific fields or topics that a user is interested in, derived from their behavioral data.

[0610] "Emotional data" refers to data that indicates the emotional state of a user, as recognized from their facial expressions, voice, and other factors.

[0611] "Emotion recognition means" refers to technology that uses cameras and microphones to analyze a user's emotional state in real time.

[0612] "Information selection means" refers to the function of a system that selects the most suitable content based on user behavior data and emotional data.

[0613] "Automatic article generation method" refers to a technology that automatically creates articles using natural language generation technology based on selected information.

[0614] "Layout techniques" refer to the methods used to arrange generated articles and content in an easy-to-read format and prepare them for the final output.

[0615] Modes for carrying out the invention

[0616] The present invention is a system that collects user behavioral and emotional data and automatically generates personalized content based on that data. Specific embodiments for carrying out the present invention are described below.

[0617] Collection of behavioral data

[0618] The server collects behavioral data from the user's device, such as operation logs, playback history, browsing time, search history, and likes and skips. This data is used to analyze the user's areas of interest. For example, if a user frequently listens to music in the "rock" genre, that information is stored by the server.

[0619] Collection and recognition of emotional data

[0620] The server collects user emotion data using the camera and microphone connected to the user's device. The emotion engine uses specific software (e.g., EmotionRecognizer) to analyze the user's facial expressions and voice in real time and recognize the user's emotional state. For example, if a user is smiling while listening to a particular song, that information is recorded as emotion data.

[0621] Analysis of areas of interest and emotional state

[0622] The collected behavioral and emotional data is stored in a database on the server. The server's AI engine analyzes this data to identify the user's areas of interest and emotional state. This includes what themes the user is interested in and the emotional state in which they enjoy those themes.

[0623] Content selection and automatic generation

[0624] The server retrieves relevant information from news APIs and music libraries based on the analysis results. Using this information, it automatically generates articles and playlists using natural language generation (NLG) technology. Specifically, it uses software like MusicRecommender to create playlists that recommend songs best suited to the user's emotional state. For example, if a user is feeling energetic and exercising, it will generate a playlist of upbeat music.

[0625] Output and preview display

[0626] The generated articles and playlists are displayed on the user's device in preview format. The user can review the generated content and make corrections or changes as needed. For example, if the user is satisfied with the generated playlist, they can play it directly.

[0627] Print and download options available.

[0628] User-created content is output in a printable format. The server generates the content in PDF format and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer.

[0629] Example of a prompt

[0630] "Please describe a program that analyzes a user's emotions while they are listening to music and generates the optimal playlist for them."

[0631] Through the above processing, the present invention realizes the provision of personalized content based on user behavioral data and emotional data. By using this system, users can easily obtain content that matches their interests and emotions.

[0632] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0633] Step 1:

[0634] The server collects behavioral data from the user's device. Specifically, it collects data on songs played by the user on the application, playback time, skipped songs, search history, and actions such as liking and sharing, and stores this data in a database. Input data includes operation logs and playback history, while output is behavioral data that serves as the basis for analysis.

[0635] Step 2:

[0636] The server collects and recognizes user emotion data using the terminal's camera and microphone. Specifically, it uses EmotionRecognizer software to analyze the user's facial expressions and voice in real time while they are playing music, and recognizes their emotional state (joy, sadness, excitement, etc.). The input data is real-time facial expressions and voice, and the output is the recognized emotion data.

[0637] Step 3:

[0638] The server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Specifically, it uses an AI engine to analyze what music genres the user is interested in and under what emotional state they enjoy them. The input data consists of behavioral and emotional data, and the output is data related to the user's areas of interest and emotional state.

[0639] Step 4:

[0640] The server selects and automatically generates relevant content based on the analysis results. Specifically, it uses MusicRecommender software to generate a playlist containing songs best suited to the user's emotional state. The input data includes areas of interest and emotional state data, and the output is the generated playlist.

[0641] Step 5:

[0642] The server displays a preview of the generated playlist on the user's device. Specifically, it sends the generated playlist to the device in real time and displays it within the application. The input data is the generated playlist, and the output is the previewed playlist.

[0643] Step 6:

[0644] Users can review a previewed playlist and make corrections or changes as needed. Specifically, they can add, delete, or rearrange songs within the playlist. The input data is the previewed playlist, and the output is the playlist modified by the user.

[0645] Step 7:

[0646] The server outputs the user-selected playlist in a printable format. Specifically, it generates the playlist in PDF format and provides a download link to the user's device. The input data is a playlist modified or changed by the user, and the output is a PDF playlist file.

[0647] Through these processing steps, users can easily enjoy personalized content based on their interests and emotions.

[0648] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0649] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0650] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0651] [Third Embodiment]

[0652] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0653] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0654] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0655] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0656] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0657] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0658] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0659] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0660] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0661] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0662] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0663] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0664] This invention relates to a system that automatically generates customized newspapers by collecting user behavior data and analyzing the user's areas of interest based on that data. The system consists of a server, a user terminal, and an AI engine.

[0665] Collection of user information

[0666] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, that history is saved in the database.

[0667] Specific example:

[0668] When a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[0669] Interest analysis and news selection

[0670] An AI engine on the server analyzes collected user behavior data to identify the user's areas of interest. Based on these identified areas of interest, the AI ​​engine retrieves relevant and up-to-date information from an external news API. This selects articles that are likely to be of interest to the user.

[0671] Specific example:

[0672] The AI ​​engine analyzes that the user has a strong interest in environmental issues and retrieves the latest environmental articles from the news API.

[0673] Generating content from photo folders

[0674] The server accesses the user's photo folder and extracts data using image analysis technology. Based on the analysis results, it automatically generates relevant articles from the user's photos. The necessary access permissions are obtained from the user in advance.

[0675] Specific example:

[0676] The server detects the user's travel photos and automatically generates a "user's travelogue" based on them.

[0677] Automatic article generation and layout

[0678] The AI ​​engine automatically generates text-based content based on articles created from selected information and photos. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[0679] Specific example:

[0680] The AI ​​engine generates articles on environmental issues and travelogues, and arranges them according to the user's interests. For example, it might place articles on environmental issues at the top of the newspaper, followed by travelogues.

[0681] Newspaper generation and preview display

[0682] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make corrections or changes as needed.

[0683] Specific example:

[0684] The server generates the newspaper as an A4-sized PDF and displays a preview on the user's device. The user checks the preview and, if satisfied with the content, saves or prints it.

[0685] Providing printing options

[0686] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0687] Specific example:

[0688] The user reviews the previewed content and clicks the "Print" button. This causes the server to download the newspaper as a PDF and print it from the user's printer.

[0689] As described above, the system of the present invention significantly reduces the effort required from the user, making it possible to easily create, share, and save customized newspapers.

[0690] The following describes the processing flow.

[0691] Step 1:

[0692] The server collects user behavior data. This includes click data on articles the user views, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, the article ID, viewing start time, and viewing end time are stored in the database.

[0693] Step 2:

[0694] The AI ​​engine on the server analyzes collected behavioral data to identify the user's areas of interest. This analysis uses machine learning algorithms and natural language processing techniques. For example, if a user reads many articles about environmental issues, the AI ​​engine will identify that the user is interested in environmental issues.

[0695] Step 3:

[0696] The AI ​​engine retrieves relevant and up-to-date information from a news API based on identified areas of interest. The news API is a service that provides the latest news articles from various sources on the internet. For example, it retrieves the latest articles on environmental issues.

[0697] Step 4:

[0698] The server accesses the user's photo folder and scans the image data within the folder using image analysis technology. The server obtains the necessary access permissions from the user beforehand. For example, the server can detect the user's travel photos and generate article ideas based on them.

[0699] Step 5:

[0700] The AI ​​engine automatically generates articles based on information obtained from a news API and ideas generated from photo folders. The generated articles are created using natural language generation (NLG) technology. For example, it can automatically generate both news articles on environmental issues and travelogues.

[0701] Step 6:

[0702] The AI ​​engine applies an algorithm to sort the generated articles according to the user's interests. This places the articles the user is most interested in at the top. For example, articles on environmental issues might be placed at the top of the newspaper, followed by travelogues.

[0703] Step 7:

[0704] The server generates a PDF or web version of the original newspaper based on the rearranged articles. The generated newspaper is displayed as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[0705] Step 8:

[0706] Users can view a preview of the newspaper and select the option to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0707] By following these steps, users can easily create, preview, and print a customized newspaper based on their interests.

[0708] (Example 1)

[0709] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0710] Current news delivery methods make it difficult to effectively provide information tailored to individual user interests and concerns, requiring users to sift through a large amount of information themselves. As a result, users waste time and effort, and their information consumption becomes incomplete. Furthermore, there are limitations to the means of easily saving and sharing generated information in physical format.

[0711] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0712] In this invention, the server includes means for collecting operation data performed on the user's terminal, means for analyzing the collected operation data and identifying the user's areas of interest, means for acquiring relevant information from an external data source based on the identified areas of interest, means for generating text content based on the acquired information, means for rearranging the generated text content in order of the user's interests and laying it out in newspaper format, and means for previewing the laid-out newspaper on an electronic device. This makes it possible to automatically generate a customized newspaper according to the user's interests, enabling them to effectively enjoy information while significantly reducing time and effort.

[0713] "User operation data" refers to data recorded on the user's device, such as click data, browsing time, search history, and interactions.

[0714] "Areas of interest" refer to the fields or topics that a user is primarily interested in, and are analyzed from the user's behavioral data.

[0715] "External data sources" refer to external news APIs and databases used to retrieve relevant information.

[0716] "Text content" refers to text-based content generated by an AI engine, including news articles and essays.

[0717] "Newspaper format" refers to a layout in which generated text content is neatly arranged and presented in an easy-to-read format.

[0718] "Electronic devices" refers to display devices such as smartphones, tablets, and personal computers.

[0719] "Image analysis technology" refers to techniques that extract information from image data using computer vision and machine learning algorithms.

[0720] "Preview display" refers to a function that allows users to check the generated content, such as newspapers, in its final output format beforehand.

[0721] A "printable format" refers to a format suitable for physical printing (for example, PDF or DOC format).

[0722] Modes for carrying out the invention

[0723] This invention relates to a system that collects user behavior data, analyzes the user's areas of interest based on that data, and automatically generates a customized newspaper. The system consists of a server, a user terminal, and a generation AI model.

[0724] Collection of user information

[0725] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. Specifically, if a user searches for "environmental protection," clicks on an article, and reads it, the click data and viewing time are collected and stored in the database. MySQL or PostgreSQL are used as the database.

[0726] Interest analysis and news selection

[0727] An AI engine on the server analyzes collected user behavior data to identify the user's areas of interest. The AI ​​engine runs using Python libraries (e.g., scikit-learn and TensorFlow). Once areas of interest are identified, the server retrieves relevant and up-to-date information through news APIs (e.g., Google News API or NewsAPI). Specifically, if the AI ​​engine analyzes that the user has a strong interest in environmental issues, it will retrieve the latest articles on environmental issues from the news API.

[0728] Generating content from photo folders

[0729] The server accesses the user's photo folder and extracts data using image analysis techniques. Necessary access permissions are obtained from the user in advance. OpenCV and TensorFlow libraries are used for image analysis. For example, if the user's travel photos are detected, related articles such as "User's Travelogue" are automatically generated based on them.

[0730] Automatic article generation and layout

[0731] The AI ​​engine automatically generates text-based content from selected information and photos. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout. This uses natural language generation technologies such as GPT-3 and BERT. For example, if an article on environmental issues and a travelogue are generated, the article on environmental issues will be placed at the top, followed by the travelogue.

[0732] Newspaper generation and preview display

[0733] The server generates the original newspaper in PDF or HTML format based on the generated articles and displays it as a preview on the user's device. Specifically, it generates the layout using LaTeX or HTML templates and displays it on the user's smartphone or PC. The user can check this preview and make corrections or changes as needed.

[0734] Providing printing options

[0735] After reviewing the previewed newspaper, users can select a print option. The server outputs the newspaper in a printable format (e.g., PDF) and provides a download link to the user's device. It can also initiate direct printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server generates the newspaper as a PDF, sends it to the user's printer, and prints it.

[0736] Examples of prompt statements

[0737] Prompt example 1:

[0738] "Automatically generate news articles based on topics that users are recently interested in. For example, if a user is interested in environmental protection, retrieve the latest environmental news."

[0739] Prompt example 2:

[0740] "Please analyze travel photos from the user's photo folder and automatically generate a travelogue based on that analysis. For example, if there are many photos of Paris, please generate a travelogue about Paris."

[0741] Prompt example 3:

[0742] "Create a customized newspaper based on articles of interest to the user and display it as a preview in PDF format. For example, if the user is interested in environmental issues, create a newspaper focusing on environmentally related articles."

[0743] As a result, the system of the present invention significantly reduces the effort required from the user, making it possible to easily create, share, and save customized newspapers.

[0744] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0745] Step 1:

[0746] The user logs into the application and generates operation data on their device. Inputs include the user's login operation and article clicks and search actions within the application. Outputs are the generated operation data, which includes click data, viewing time, search history, and interactions. Specifically, the user accesses the application using a smartphone or PC, searches for an article using the keyword "environmental protection," and clicks to read it.

[0747] Step 2:

[0748] The server collects user activity data and stores it in a database. The input is the activity data generated in step 1. The output is the activity data stored in the database. Specifically, the server collects user click data, browsing time, and search history, and stores the data in a database such as MySQL or PostgreSQL.

[0749] Step 3:

[0750] An AI engine on the server analyzes collected interaction data to identify the user's areas of interest. The input is the interaction data stored in the database. The output is the identified areas of interest. Specifically, the AI ​​engine uses Python libraries (such as scikit-learn or TensorFlow) to analyze the interaction data and identify that the user is interested in environmental issues.

[0751] Step 4:

[0752] The server retrieves relevant and up-to-date information from external data sources based on identified areas of interest. The input is the user's areas of interest. The output is the retrieved latest information (articles). Specifically, the server uses a news API (e.g., Google News API or NewsAPI) to retrieve the latest articles on environmental issues of interest.

[0753] Step 5:

[0754] The server accesses the user's photo folder, extracts data using image analysis techniques, and automatically generates related articles. The input is the user's photo data. The output is the image analysis results and the automatically generated articles. Specifically, the server uses OpenCV or TensorFlow to analyze travel photos and generates a "user travelogue" based on that analysis. Access permission to the photo folder is obtained from the user beforehand.

[0755] Step 6:

[0756] An AI engine on the server automatically generates text content based on articles created from selected information and photos, and then arranges them in a newspaper format, sorted according to the user's interests. The input is the latest acquired information and image analysis results. The output is a pre-layout newspaper-style article. Specifically, the AI ​​engine uses GPT-3 and BERT to generate articles, placing articles on environmental issues at the top and travelogues next.

[0757] Step 7:

[0758] The server generates a newspaper and displays a preview of it on the user's device. The input is a pre-layout newspaper article. The output is the preview of the newspaper displayed on the device. Specifically, the server generates a newspaper in PDF or HTML format using LaTeX or HTML templates and displays a preview of it on the user's smartphone or PC.

[0759] Step 8:

[0760] The user reviews the previewed newspaper and selects a print option. The server outputs the newspaper in a printable format and provides a download link. It also initiates printing directly with the user's printer. The input is the previewed newspaper. The output is the newspaper in a printable format and a download link. Specifically, when the user clicks the "Print" button, the server generates the newspaper in PDF format, sends it to the printer, and performs the printing.

[0761] (Application Example 1)

[0762] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0763] In recent years, there has been a growing demand for customized services based on users' purchasing behavior and interests, but meeting this demand requires advanced data analysis and automated generation technologies. In particular, e-commerce sites lack mechanisms to efficiently recommend products likely to interest users and provide related news and information. Existing systems fail to fully utilize user behavior data, making it difficult to provide information tailored to individual users.

[0764] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0765] In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for automatically selecting relevant information based on the analysis results, means for automatically generating articles based on the selected information, means for laying out the generated articles and outputting them as a newspaper, means for recommending products on an e-commerce site based on the user's behavior data, and means for outputting articles generated based on the user's areas of interest in PDF format. This makes it possible to provide customized information and product recommendations that are tailored to the interests and purchasing behavior of individual users, thereby improving user satisfaction and increasing purchasing intent.

[0766] "User behavior data" refers to data on actions taken by users on applications and websites, such as click history, browsing history, search history, and purchase history.

[0767] "Areas of interest" refers to the fields or topics that a user is particularly interested in, as determined by analyzing their behavioral data.

[0768] "Relevant information" refers to information that is valuable to the user, such as news articles and product information selected based on the user's areas of interest.

[0769] "Methods for automatically generating articles" refers to algorithms and systems that generate text based on selected relevant information.

[0770] "Methods for outputting as a newspaper" refers to a system that automatically generates articles and lays them out in a specific format (e.g., PDF or web page) for output.

[0771] An "online shopping site" refers to a website that sells goods and services via the internet.

[0772] "A means of recommending products" refers to a system that selects and displays products that a user is likely to be interested in, based on their behavioral data.

[0773] "Methods for outputting in PDF format" refers to systems that save or display generated articles or information as files in PDF format.

[0774] This invention is a system that collects user behavior data and recommends customized information and products based on that data. Specifically, it consists of a server, a user terminal, and an AI engine.

[0775] Collection of user information

[0776] The server collects data on user actions on the e-commerce site. This data includes product click history, browsing time, search history, and purchase history. For example, if a user clicks on and views products related to "summer fashion," that history is recorded on the server.

[0777] Interest analysis and product recommendations

[0778] The AI ​​engine on the server analyzes collected user behavior data to identify the user's areas of interest. Based on these identified areas of interest, the AI ​​engine retrieves relevant products from an external database. This allows for the recommendation of products that the user is likely to be interested in.

[0779] Inspiration from photo folders

[0780] The server accesses the user's photo folder and extracts data using image analysis technology. Based on the analysis results, it automatically generates related products and articles from the user's photos. For example, it can generate "recommended travel items" based on the user's travel photos. The necessary access permissions are obtained from the user in advance.

[0781] Generating and Layout Custom Articles

[0782] The AI ​​engine automatically generates text-based content based on articles created from selected product information and photos. The generated articles are sorted according to the user's interests and compiled into a PDF format with an optimal layout. For example, articles about summer fashion are placed at the top, followed by articles about travel goods.

[0783] PDF generation and preview display

[0784] The server generates an original PDF based on the generated article and displays it as a preview on the user's device. The user can review this preview and make corrections or changes as needed. For example, if the user reviews the generated PDF and is satisfied with the content, they can save or print it as is.

[0785] Providing printing options

[0786] After reviewing the preview, users can choose to print. The server outputs the PDF in a printable format and provides a download link to the user's device. It can also directly initiate printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server generates a PDF, which the user can then print as a poster or flyer on their own printer.

[0787] For example, if a user searches for "summer fashion items" and leaves a click and browsing history, the server collects this data, and the AI ​​engine analyzes the user's interests. Based on this, the server recommends the latest summer fashion items and related news, and generates a customized PDF. This PDF may include articles such as "This Summer's Trendy Items" or "Recommended Beach Gear."

[0788] Examples of prompt statements to input into a generative AI model are as follows:

[0789] Based on the user's search history, browsing history, and purchase history, recommend products and related news articles that the user might be interested in. The following is the user's past data.

[0790] Search history: ['Summer dress', 'Flip-flops', 'Travel suitcase']

[0791] Browsing history: ['New beach sandals', 'Summer clothes made of cooling material']

[0792] Purchase history: ['Travel bag', 'Sunglasses']

[0793] Based on the data mentioned above, generate news articles related to products that users are likely to be interested in.

[0794] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0795] Step 1:

[0796] Collect user behavior data.

[0797] The server collects user actions on the e-commerce site (product clicks, browsing, search history, purchase history) and stores them in a database. The collected data includes product IDs, operation times, and operation details. This reveals user behavior patterns.

[0798] Step 2:

[0799] Analyze the user's areas of interest.

[0800] The server sends the collected user behavior data to the AI ​​engine for analysis. The AI ​​engine uses TF-IDF vectorization and KMeans clustering techniques to identify categories and topics of high user interest. The input is user behavior data, and the output is cluster data related to areas of interest.

[0801] Step 3:

[0802] Select related products.

[0803] The AI ​​engine selects products related to the user's areas of interest from an external database based on the analysis results. For example, if it determines that the user is interested in summer fashion, it will retrieve the latest information on summer items. The input is cluster data of areas of interest, and the output is a list of recommended products.

[0804] Step 4:

[0805] Analyze the photo folder.

[0806] The server accesses the photo folder stored on the user's device and extracts data using image analysis technology. Specifically, it analyzes image files to generate tags, and then automatically generates related products and articles based on that tag information. The input is the user's photo data, and the output is image tags and their associated product information.

[0807] Step 5:

[0808] Automatically generate articles.

[0809] The server uses an AI engine to generate text-based content based on selected product information and photo analysis results. For example, it can automatically create articles introducing new flip-flops or recommending travel goods. The input is product information and image tag information, and the output is customized text content.

[0810] Step 6:

[0811] Layout the generated articles.

[0812] The server lays out the generated text content in PDF format. Articles are sorted according to user interest and compiled into a PDF with optimal layout. The input is the generated text content, and the output is a pre-layout PDF file.

[0813] Step 7:

[0814] Preview the PDF.

[0815] The terminal visually displays the generated PDF to the user, providing a preview. The user can review the content and make corrections or changes as needed. The input is a pre-layout PDF file, and the output is a preview displayed on the terminal screen.

[0816] Step 8:

[0817] Print the PDF.

[0818] After the user reviews the preview, they select the print option. The server outputs the PDF in a printable format, and the user's device initiates printing to the printer. The input is the previewed PDF file, and the output is the printed document.

[0819] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0820] This invention is a system that collects user behavior data, analyzes the user's areas of interest based on that data, and automatically generates a customized newspaper. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves even more personalized article generation. This system consists of a server, a user terminal, and an AI engine.

[0821] Collection of user information

[0822] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, that history is saved in the database.

[0823] Specific example:

[0824] When a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[0825] Emotion recognition by an emotion engine

[0826] The server collects user emotion data through emotion recognition devices (such as cameras and microphones) connected to the user's terminal. The emotion engine recognizes the user's emotions in real time using facial expression analysis and voice analysis. For example, it analyzes the user's facial expressions and tone of voice while they are reading an article to determine whether the user is in an emotional state such as happy, sad, or interested.

[0827] Specific example:

[0828] When a user is reading an article about environmental issues, the camera captures the user's facial expressions, and an emotion engine analyzes those expressions to recognize that the user finds them interesting.

[0829] Interest analysis and news selection

[0830] An AI engine on the server analyzes behavioral and emotional data to identify the user's areas of interest and emotional state. Based on this analysis, it retrieves relevant and up-to-date information from the news API. For example, if a user has a strong interest in environmental issues and shows positive emotions towards that topic, the AI ​​engine will select environmental articles that are relevant to the user.

[0831] Specific example:

[0832] The AI ​​engine retrieves the latest environmental articles from the news API if the user is interested in environmental issues and expresses positive feelings towards them.

[0833] Generating content from photo folders

[0834] The server accesses the user's photo folder and extracts data using image analysis technology. The server obtains the necessary access permissions from the user in advance. For example, the server can detect the user's travel photos and use them to generate article ideas.

[0835] Specific example:

[0836] The server detects the user's travel photos and generates a "user's travelogue" based on those photos.

[0837] Automatic article generation and layout

[0838] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. The generated articles are created using natural language generation (NLG) technology and include sentences and expressions based on specific sentiment data. Furthermore, the generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[0839] Specific example:

[0840] The AI ​​engine generates an article titled "The Future of the Environment" in a positive tone, based on the positive emotions of the user recognized by the emotion engine, and places it at the top.

[0841] Newspaper generation and preview display

[0842] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[0843] Specific example:

[0844] The server generates the newspaper as an A4-sized PDF and displays a preview on the user's device. The user checks the preview and, if satisfied with the content, saves or prints it.

[0845] Providing printing options

[0846] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0847] Specific example:

[0848] The user reviews the previewed content and clicks the "Print" button. This causes the server to download the newspaper as a PDF and print it from the user's printer.

[0849] As described above, the system of the present invention significantly reduces the effort required from the user, enabling them to easily create, share, and save customized newspapers. Furthermore, by recognizing the user's emotions and generating articles based on those emotions, it can provide a more personalized experience.

[0850] The following describes the processing flow.

[0851] Step 1:

[0852] The server collects user behavior data. This includes click data on articles the user views, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, the article ID, viewing start time, and viewing end time are stored in the database.

[0853] Step 2:

[0854] The server collects user emotion data through emotion recognition devices (such as cameras and microphones) connected to the user's terminal. The emotion engine recognizes the user's emotions in real time using facial expression analysis and voice analysis. For example, it analyzes the user's facial expressions and tone of voice while they are reading an article to determine whether the user is in an emotional state such as happy, sad, or interested.

[0855] Step 3:

[0856] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. This analysis uses machine learning algorithms and natural language processing techniques. For example, if a user reads many articles about environmental issues and expresses positive emotions while doing so, the AI ​​engine will identify that the user is interested in and has a favorable view of environmental issues.

[0857] Step 4:

[0858] The AI ​​engine retrieves relevant and up-to-date information from a news API based on identified areas of interest and emotional state. The news API is a service that provides the latest news articles from various sources on the internet. For example, it retrieves the latest articles on environmental issues and prioritizes selecting those with positive content.

[0859] Step 5:

[0860] The server accesses the user's photo folder and scans the image data within the folder using image analysis technology. The server obtains the necessary access permissions from the user beforehand. For example, the server can detect the user's travel photos and use them to generate article ideas.

[0861] Step 6:

[0862] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. The generated articles are created using natural language generation (NLG) technology. For example, if a user is interested in environmental issues and expresses positive feelings towards them, the AI ​​engine will create an article about environmental issues in a positive tone.

[0863] Step 7:

[0864] The AI ​​engine applies an algorithm to sort the generated articles according to the user's interests. This places the articles the user is most interested in at the top. For example, articles on environmental issues might be placed at the top of the newspaper, followed by travelogues.

[0865] Step 8:

[0866] The server generates a PDF or web version of the original newspaper based on the rearranged articles. The generated newspaper is displayed as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[0867] Step 9:

[0868] Users can view a preview of the newspaper and select the option to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[0869] By following these steps, users can easily create, preview, and print a customized newspaper based on their interests and feelings.

[0870] (Example 2)

[0871] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0872] Traditional systems provided personalized content based solely on user behavior data, making it difficult to reflect user emotions or subtle shifts in interests. Furthermore, the lack of a function to generate content from photo folders prevented the delivery of more personalized articles. Additionally, there was a lack of easy ways to print the generated content.

[0873] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0874] In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for collecting and analyzing user sentiment data, means for automatically generating articles based on selected information and sentiment data, and means for laying out the generated articles and outputting them as a newspaper. This makes it possible to utilize both user behavior data and sentiment data to generate more accurate personalized articles and to provide newspapers with content that meets the individual needs of the user.

[0875] "User behavior data" refers to information about the actions and interactions that users perform on the application, and specifically includes article click data, viewing time, search history, and interactions with followers.

[0876] "Areas of interest" refer to the fields or topics that a user is particularly interested in, and are identified from behavioral and emotional data.

[0877] "Emotional data" refers to information about a user's emotional state collected from their facial expressions and voice, including their facial expressions and tone of voice while reading an article.

[0878] "Automatic article generation" refers to the process by which an AI engine automatically creates articles using natural language generation (NLG) technology based on collected and analyzed data.

[0879] "Layout" refers to the process of arranging generated articles in a specific order and format, and includes the process of compiling them into an optimal newspaper format.

[0880] "Outputting as a newspaper" means providing the generated articles to the user's device as a newspaper in PDF or web format.

[0881] "Access permission" refers to the permission a server has to access specific data on a user's device (for example, their photo folder), and this permission is approved by the user in advance.

[0882] A "News API" is an application programming interface used to retrieve the latest news information.

[0883] Natural Language Generation (NLG) is a technology that uses an AI engine to mechanically generate data and output it in a natural language format.

[0884] The "print option" is a feature that outputs the generated newspaper in a printable format (e.g., PDF), allowing the user to physically print it.

[0885] Modes for carrying out the invention

[0886] This invention relates to a system that automatically generates personalized newspapers using user behavioral data and emotional data. This system consists of multiple components, including a server, a user terminal, and an AI engine.

[0887] Collection of user behavior data

[0888] The server collects data on user actions within the application. This data includes article click data, viewing time, search history, and interactions with followers. For example, if a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[0889] Collection of emotional data

[0890] The server collects user emotion data in real time using emotion recognition devices (such as cameras and microphones) connected to the user's device. The emotion engine recognizes the user's emotions using facial expression analysis and voice analysis and stores them in a database. For example, by analyzing the user's facial expressions and tone of voice while reading an article, it can recognize that the user is finding it interesting.

[0891] Analysis of user areas of interest

[0892] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Based on this analysis, it retrieves relevant and up-to-date information from the news API. For example, if a user has a strong interest in environmental issues and also shows positive emotions towards that topic, the AI ​​engine will select environmental articles that are relevant to the user.

[0893] Generating content from photo folders

[0894] The server uses access permissions obtained from the user in advance to access the user's photo folder and extracts data using image analysis technology. For example, the server detects the user's travel photos and generates article ideas for a "user's travelogue" based on those photos.

[0895] Automatic article generation and layout

[0896] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. This process utilizes natural language generation (NLG) technology. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[0897] Newspaper generation and preview display

[0898] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make any necessary corrections or changes. For example, the server generates the newspaper in A4 size PDF format and displays it as a preview on the user's device. The user reviews the content and, if satisfied, saves or prints it.

[0899] Providing printing options

[0900] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format (PDF) and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server provides a download link for the newspaper as a PDF, and the user prints it directly from their printer.

[0901] Example of a prompt

[0902] "If a user is interested in 'environmental protection' and shows positive sentiments, the system will retrieve the latest environmental news and generate articles with a positive tone."

[0903] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0904] Step 1:

[0905] The server collects user behavior data. Specifically, it collects data on the actions users perform on the application and stores it in a database. Inputs include user actions such as clicking on articles or searching, and based on these, click data, viewing time, search history, etc., are recorded on the server. For example, if a user searches for an article using the search term "environmental protection," clicks on it, and views it for 10 minutes, the search term, article ID, and viewing time will be recorded.

[0906] Step 2:

[0907] The server collects emotional data using emotion recognition devices (camera and microphone) connected to the user's device. The input includes the user's facial expressions and voice while they are reading an article, and the emotion engine analyzes this to determine their emotional state. Specifically, the camera captures the user's smile while they are reading an article, and the emotion engine recognizes this data as a "positive emotion" and outputs it.

[0908] Step 3:

[0909] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Using the collected behavioral and emotional data as input, it performs data calculations to analyze the user's interests. For example, if the analysis reveals that the user is interested in environmental issues and exhibits positive emotions, the AI ​​engine will use this to identify relevant recent articles on environmental issues as output.

[0910] Step 4:

[0911] The server accesses the photo folder and extracts data using image analysis technology. The input is the photo data in the user's photo folder, which the server analyzes. Specifically, it detects photos of a "trip to Paris" from the user's photo folder and uses this to generate content for an article titled "The User's Travelogue."

[0912] Step 5:

[0913] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data. Input includes the latest information obtained from the news API and various collected data. The AI ​​engine combines this data and generates articles using natural language generation (NLG) technology. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout. For example, based on the user's positive emotions recognized by the sentiment engine, an article titled "The Future of the Environment" is generated in a positive tone and placed at the top of the newspaper.

[0914] Step 6:

[0915] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The input is article data generated by an AI engine, which is used to create the newspaper file. For example, the server displays a newspaper generated in A4-sized PDF format on the user's device. The user can review the preview and change the order of the articles.

[0916] Step 7:

[0917] After reviewing the previewed newspaper, the user can select a print option. The server outputs the newspaper in a printable format (PDF) and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer. A generated PDF file is provided as input, and when the user clicks the "Print" button, the server provides the PDF as a download link, and printing is initiated directly from the user's printer.

[0918] (Application Example 2)

[0919] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0920] Traditional systems that effectively utilize user behavior data and emotional states to provide information tailored to user interests and emotions are incomplete and require improvement to enhance the user experience. Furthermore, personalized content such as music and news needs to be enhanced to increase user satisfaction.

[0921] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for collecting and recognizing user emotion data, means for selecting and generating information based on the collected and recognized emotion data, means for automatically generating articles and content based on the selected information, and means for laying out and outputting the generated articles and content. This makes it possible to provide personalized information based on the user's interests and emotions.

[0922] "Behavioral data" refers to all records of operations and activities performed by users on the system.

[0923] "Areas of interest" refers to specific fields or topics that a user is interested in, derived from their behavioral data.

[0924] "Emotional data" refers to data that indicates the emotional state of a user, as recognized from their facial expressions, voice, and other factors.

[0925] "Emotion recognition means" refers to technology that uses cameras and microphones to analyze a user's emotional state in real time.

[0926] "Information selection means" refers to the function of a system that selects the most suitable content based on user behavior data and emotional data.

[0927] "Automatic article generation method" refers to a technology that automatically creates articles using natural language generation technology based on selected information.

[0928] "Layout techniques" refer to the methods used to arrange generated articles and content in an easy-to-read format and prepare them for the final output.

[0929] Modes for carrying out the invention

[0930] The present invention is a system that collects user behavioral and emotional data and automatically generates personalized content based on that data. Specific embodiments for carrying out the present invention are described below.

[0931] Collection of behavioral data

[0932] The server collects behavioral data from the user's device, such as operation logs, playback history, browsing time, search history, and likes and skips. This data is used to analyze the user's areas of interest. For example, if a user frequently listens to music in the "rock" genre, that information is stored by the server.

[0933] Collection and recognition of emotional data

[0934] The server collects user emotion data using the camera and microphone connected to the user's device. The emotion engine uses specific software (e.g., EmotionRecognizer) to analyze the user's facial expressions and voice in real time and recognize the user's emotional state. For example, if a user is smiling while listening to a particular song, that information is recorded as emotion data.

[0935] Analysis of areas of interest and emotional state

[0936] The collected behavioral and emotional data is stored in a database on the server. The server's AI engine analyzes this data to identify the user's areas of interest and emotional state. This includes what themes the user is interested in and the emotional state in which they enjoy those themes.

[0937] Content selection and automatic generation

[0938] The server retrieves relevant information from news APIs and music libraries based on the analysis results. Using this information, it automatically generates articles and playlists using natural language generation (NLG) technology. Specifically, it uses software like MusicRecommender to create playlists that recommend songs best suited to the user's emotional state. For example, if a user is feeling energetic and exercising, it will generate a playlist of upbeat music.

[0939] Output and preview display

[0940] The generated articles and playlists are displayed on the user's device in preview format. The user can review the generated content and make corrections or changes as needed. For example, if the user is satisfied with the generated playlist, they can play it directly.

[0941] Print and download options available.

[0942] User-created content is output in a printable format. The server generates the content in PDF format and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer.

[0943] Example of a prompt

[0944] "Please describe a program that analyzes a user's emotions while they are listening to music and generates the optimal playlist for them."

[0945] Through the above processing, the present invention realizes the provision of personalized content based on user behavioral data and emotional data. By using this system, users can easily obtain content that matches their interests and emotions.

[0946] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0947] Step 1:

[0948] The server collects behavioral data from the user's device. Specifically, it collects data on songs played by the user on the application, playback time, skipped songs, search history, and actions such as liking and sharing, and stores this data in a database. Input data includes operation logs and playback history, while output is behavioral data that serves as the basis for analysis.

[0949] Step 2:

[0950] The server collects and recognizes user emotion data using the terminal's camera and microphone. Specifically, it uses EmotionRecognizer software to analyze the user's facial expressions and voice in real time while they are playing music, and recognizes their emotional state (joy, sadness, excitement, etc.). The input data is real-time facial expressions and voice, and the output is the recognized emotion data.

[0951] Step 3:

[0952] The server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Specifically, it uses an AI engine to analyze what music genres the user is interested in and under what emotional state they enjoy them. The input data consists of behavioral and emotional data, and the output is data related to the user's areas of interest and emotional state.

[0953] Step 4:

[0954] The server selects and automatically generates relevant content based on the analysis results. Specifically, it uses MusicRecommender software to generate a playlist containing songs best suited to the user's emotional state. The input data includes areas of interest and emotional state data, and the output is the generated playlist.

[0955] Step 5:

[0956] The server displays a preview of the generated playlist on the user's device. Specifically, it sends the generated playlist to the device in real time and displays it within the application. The input data is the generated playlist, and the output is the previewed playlist.

[0957] Step 6:

[0958] Users can review a previewed playlist and make corrections or changes as needed. Specifically, they can add, delete, or rearrange songs within the playlist. The input data is the previewed playlist, and the output is the playlist modified by the user.

[0959] Step 7:

[0960] The server outputs the user-selected playlist in a printable format. Specifically, it generates the playlist in PDF format and provides a download link to the user's device. The input data is a playlist modified or changed by the user, and the output is a PDF playlist file.

[0961] Through these processing steps, users can easily enjoy personalized content based on their interests and emotions.

[0962] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0963] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0964] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0965] [Fourth Embodiment]

[0966] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0967] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0968] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0969] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0970] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0971] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0972] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0973] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0974] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0975] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0976] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0977] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0978] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0979] This invention relates to a system that automatically generates customized newspapers by collecting user behavior data and analyzing the user's areas of interest based on that data. The system consists of a server, a user terminal, and an AI engine.

[0980] Collection of user information

[0981] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, that history is saved in the database.

[0982] Specific example:

[0983] When a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[0984] Interest analysis and news selection

[0985] An AI engine on the server analyzes collected user behavior data to identify the user's areas of interest. Based on these identified areas of interest, the AI ​​engine retrieves relevant and up-to-date information from an external news API. This selects articles that are likely to be of interest to the user.

[0986] Specific example:

[0987] The AI ​​engine analyzes that the user has a strong interest in environmental issues and retrieves the latest environmental articles from the news API.

[0988] Generating content from photo folders

[0989] The server accesses the user's photo folder and extracts data using image analysis technology. Based on the analysis results, it automatically generates relevant articles from the user's photos. The necessary access permissions are obtained from the user in advance.

[0990] Specific example:

[0991] The server detects the user's travel photos and automatically generates a "user's travelogue" based on them.

[0992] Automatic article generation and layout

[0993] The AI ​​engine automatically generates text-based content based on articles created from selected information and photos. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[0994] Specific example:

[0995] The AI ​​engine generates articles on environmental issues and travelogues, and arranges them according to the user's interests. For example, it might place articles on environmental issues at the top of the newspaper, followed by travelogues.

[0996] Newspaper generation and preview display

[0997] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make corrections or changes as needed.

[0998] Specific example:

[0999] The server generates the newspaper as an A4-sized PDF and displays a preview on the user's device. The user checks the preview and, if satisfied with the content, saves or prints it.

[1000] Providing printing options

[1001] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[1002] Specific example:

[1003] The user reviews the previewed content and clicks the "Print" button. This causes the server to download the newspaper as a PDF and print it from the user's printer.

[1004] As described above, the system of the present invention significantly reduces the effort required from the user, making it possible to easily create, share, and save customized newspapers.

[1005] The following describes the processing flow.

[1006] Step 1:

[1007] The server collects user behavior data. This includes click data on articles the user views, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, the article ID, viewing start time, and viewing end time are stored in the database.

[1008] Step 2:

[1009] The AI ​​engine on the server analyzes collected behavioral data to identify the user's areas of interest. This analysis uses machine learning algorithms and natural language processing techniques. For example, if a user reads many articles about environmental issues, the AI ​​engine will identify that the user is interested in environmental issues.

[1010] Step 3:

[1011] The AI ​​engine retrieves relevant and up-to-date information from a news API based on identified areas of interest. The news API is a service that provides the latest news articles from various sources on the internet. For example, it retrieves the latest articles on environmental issues.

[1012] Step 4:

[1013] The server accesses the user's photo folder and scans the image data within the folder using image analysis technology. The server obtains the necessary access permissions from the user beforehand. For example, the server can detect the user's travel photos and generate article ideas based on them.

[1014] Step 5:

[1015] The AI ​​engine automatically generates articles based on information obtained from a news API and ideas generated from photo folders. The generated articles are created using natural language generation (NLG) technology. For example, it can automatically generate both news articles on environmental issues and travelogues.

[1016] Step 6:

[1017] The AI ​​engine applies an algorithm to sort the generated articles according to the user's interests. This places the articles the user is most interested in at the top. For example, articles on environmental issues might be placed at the top of the newspaper, followed by travelogues.

[1018] Step 7:

[1019] The server generates a PDF or web version of the original newspaper based on the rearranged articles. The generated newspaper is displayed as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[1020] Step 8:

[1021] Users can view a preview of the newspaper and select the option to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[1022] By following these steps, users can easily create, preview, and print a customized newspaper based on their interests.

[1023] (Example 1)

[1024] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1025] Current news delivery methods make it difficult to effectively provide information tailored to individual user interests and concerns, requiring users to sift through a large amount of information themselves. As a result, users waste time and effort, and their information consumption becomes incomplete. Furthermore, there are limitations to the means of easily saving and sharing generated information in physical format.

[1026] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1027] In this invention, the server includes means for collecting operation data performed on the user's terminal, means for analyzing the collected operation data and identifying the user's areas of interest, means for acquiring relevant information from an external data source based on the identified areas of interest, means for generating text content based on the acquired information, means for rearranging the generated text content in order of the user's interests and laying it out in newspaper format, and means for previewing the laid-out newspaper on an electronic device. This makes it possible to automatically generate a customized newspaper according to the user's interests, enabling them to effectively enjoy information while significantly reducing time and effort.

[1028] "User operation data" refers to data recorded on the user's device, such as click data, browsing time, search history, and interactions.

[1029] "Areas of interest" refer to the fields or topics that a user is primarily interested in, and are analyzed from the user's behavioral data.

[1030] "External data sources" refer to external news APIs and databases used to retrieve relevant information.

[1031] "Text content" refers to text-based content generated by an AI engine, including news articles and essays.

[1032] "Newspaper format" refers to a layout in which generated text content is neatly arranged and presented in an easy-to-read format.

[1033] "Electronic devices" refers to display devices such as smartphones, tablets, and personal computers.

[1034] "Image analysis technology" refers to techniques that extract information from image data using computer vision and machine learning algorithms.

[1035] "Preview display" refers to a function that allows users to check the generated content, such as newspapers, in its final output format beforehand.

[1036] A "printable format" refers to a format suitable for physical printing (for example, PDF or DOC format).

[1037] Modes for carrying out the invention

[1038] This invention relates to a system that collects user behavior data, analyzes the user's areas of interest based on that data, and automatically generates a customized newspaper. The system consists of a server, a user terminal, and a generation AI model.

[1039] Collection of user information

[1040] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. Specifically, if a user searches for "environmental protection," clicks on an article, and reads it, the click data and viewing time are collected and stored in the database. MySQL or PostgreSQL are used as the database.

[1041] Interest analysis and news selection

[1042] An AI engine on the server analyzes collected user behavior data to identify the user's areas of interest. The AI ​​engine runs using Python libraries (e.g., scikit-learn and TensorFlow). Once areas of interest are identified, the server retrieves relevant and up-to-date information through news APIs (e.g., Google News API or NewsAPI). Specifically, if the AI ​​engine analyzes that the user has a strong interest in environmental issues, it will retrieve the latest articles on environmental issues from the news API.

[1043] Generating content from photo folders

[1044] The server accesses the user's photo folder and extracts data using image analysis techniques. Necessary access permissions are obtained from the user in advance. OpenCV and TensorFlow libraries are used for image analysis. For example, if the user's travel photos are detected, related articles such as "User's Travelogue" are automatically generated based on them.

[1045] Automatic article generation and layout

[1046] The AI ​​engine automatically generates text-based content from selected information and photos. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout. This uses natural language generation technologies such as GPT-3 and BERT. For example, if an article on environmental issues and a travelogue are generated, the article on environmental issues will be placed at the top, followed by the travelogue.

[1047] Newspaper generation and preview display

[1048] The server generates the original newspaper in PDF or HTML format based on the generated articles and displays it as a preview on the user's device. Specifically, it generates the layout using LaTeX or HTML templates and displays it on the user's smartphone or PC. The user can check this preview and make corrections or changes as needed.

[1049] Providing printing options

[1050] After reviewing the previewed newspaper, users can select a print option. The server outputs the newspaper in a printable format (e.g., PDF) and provides a download link to the user's device. It can also initiate direct printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server generates the newspaper as a PDF, sends it to the user's printer, and prints it.

[1051] Examples of prompt statements

[1052] Prompt example 1:

[1053] "Automatically generate news articles based on topics that users are recently interested in. For example, if a user is interested in environmental protection, retrieve the latest environmental news."

[1054] Prompt example 2:

[1055] "Please analyze travel photos from the user's photo folder and automatically generate a travelogue based on that analysis. For example, if there are many photos of Paris, please generate a travelogue about Paris."

[1056] Prompt example 3:

[1057] "Create a customized newspaper based on articles of interest to the user and display it as a preview in PDF format. For example, if the user is interested in environmental issues, create a newspaper focusing on environmentally related articles."

[1058] As a result, the system of the present invention significantly reduces the effort required from the user, making it possible to easily create, share, and save customized newspapers.

[1059] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1060] Step 1:

[1061] The user logs into the application and generates operation data on their device. Inputs include the user's login operation and article clicks and search actions within the application. Outputs are the generated operation data, which includes click data, viewing time, search history, and interactions. Specifically, the user accesses the application using a smartphone or PC, searches for an article using the keyword "environmental protection," and clicks to read it.

[1062] Step 2:

[1063] The server collects user activity data and stores it in a database. The input is the activity data generated in step 1. The output is the activity data stored in the database. Specifically, the server collects user click data, browsing time, and search history, and stores the data in a database such as MySQL or PostgreSQL.

[1064] Step 3:

[1065] An AI engine on the server analyzes collected interaction data to identify the user's areas of interest. The input is the interaction data stored in the database. The output is the identified areas of interest. Specifically, the AI ​​engine uses Python libraries (such as scikit-learn or TensorFlow) to analyze the interaction data and identify that the user is interested in environmental issues.

[1066] Step 4:

[1067] The server retrieves relevant and up-to-date information from external data sources based on identified areas of interest. The input is the user's areas of interest. The output is the retrieved latest information (articles). Specifically, the server uses a news API (e.g., Google News API or NewsAPI) to retrieve the latest articles on environmental issues of interest.

[1068] Step 5:

[1069] The server accesses the user's photo folder, extracts data using image analysis techniques, and automatically generates related articles. The input is the user's photo data. The output is the image analysis results and the automatically generated articles. Specifically, the server uses OpenCV or TensorFlow to analyze travel photos and generates a "user travelogue" based on that analysis. Access permission to the photo folder is obtained from the user beforehand.

[1070] Step 6:

[1071] An AI engine on the server automatically generates text content based on articles created from selected information and photos, and then arranges them in a newspaper format, sorted according to the user's interests. The input is the latest acquired information and image analysis results. The output is a pre-layout newspaper-style article. Specifically, the AI ​​engine uses GPT-3 and BERT to generate articles, placing articles on environmental issues at the top and travelogues next.

[1072] Step 7:

[1073] The server generates a newspaper and displays a preview of it on the user's device. The input is a pre-layout newspaper article. The output is the preview of the newspaper displayed on the device. Specifically, the server generates a newspaper in PDF or HTML format using LaTeX or HTML templates and displays a preview of it on the user's smartphone or PC.

[1074] Step 8:

[1075] The user reviews the previewed newspaper and selects a print option. The server outputs the newspaper in a printable format and provides a download link. It also initiates printing directly with the user's printer. The input is the previewed newspaper. The output is the newspaper in a printable format and a download link. Specifically, when the user clicks the "Print" button, the server generates the newspaper in PDF format, sends it to the printer, and performs the printing.

[1076] (Application Example 1)

[1077] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1078] In recent years, there has been a growing demand for customized services based on users' purchasing behavior and interests, but meeting this demand requires advanced data analysis and automated generation technologies. In particular, e-commerce sites lack mechanisms to efficiently recommend products likely to interest users and provide related news and information. Existing systems fail to fully utilize user behavior data, making it difficult to provide information tailored to individual users.

[1079] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1080] In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for automatically selecting relevant information based on the analysis results, means for automatically generating articles based on the selected information, means for laying out the generated articles and outputting them as a newspaper, means for recommending products on an e-commerce site based on the user's behavior data, and means for outputting articles generated based on the user's areas of interest in PDF format. This makes it possible to provide customized information and product recommendations that are tailored to the interests and purchasing behavior of individual users, thereby improving user satisfaction and increasing purchasing intent.

[1081] "User behavior data" refers to data on actions taken by users on applications and websites, such as click history, browsing history, search history, and purchase history.

[1082] "Areas of interest" refers to the fields or topics that a user is particularly interested in, as determined by analyzing their behavioral data.

[1083] "Relevant information" refers to information that is valuable to the user, such as news articles and product information selected based on the user's areas of interest.

[1084] "Methods for automatically generating articles" refers to algorithms and systems that generate text based on selected relevant information.

[1085] "Methods for outputting as a newspaper" refers to a system that automatically generates articles and lays them out in a specific format (e.g., PDF or web page) for output.

[1086] An "online shopping site" refers to a website that sells goods and services via the internet.

[1087] "A means of recommending products" refers to a system that selects and displays products that a user is likely to be interested in, based on their behavioral data.

[1088] "Methods for outputting in PDF format" refers to systems that save or display generated articles or information as files in PDF format.

[1089] This invention is a system that collects user behavior data and recommends customized information and products based on that data. Specifically, it consists of a server, a user terminal, and an AI engine.

[1090] Collection of user information

[1091] The server collects data on user actions on the e-commerce site. This data includes product click history, browsing time, search history, and purchase history. For example, if a user clicks on and views products related to "summer fashion," that history is recorded on the server.

[1092] Interest analysis and product recommendations

[1093] The AI ​​engine on the server analyzes collected user behavior data to identify the user's areas of interest. Based on these identified areas of interest, the AI ​​engine retrieves relevant products from an external database. This allows for the recommendation of products that the user is likely to be interested in.

[1094] Inspiration from photo folders

[1095] The server accesses the user's photo folder and extracts data using image analysis technology. Based on the analysis results, it automatically generates related products and articles from the user's photos. For example, it can generate "recommended travel items" based on the user's travel photos. The necessary access permissions are obtained from the user in advance.

[1096] Generating and Layout Custom Articles

[1097] The AI ​​engine automatically generates text-based content based on articles created from selected product information and photos. The generated articles are sorted according to the user's interests and compiled into a PDF format with an optimal layout. For example, articles about summer fashion are placed at the top, followed by articles about travel goods.

[1098] PDF generation and preview display

[1099] The server generates an original PDF based on the generated article and displays it as a preview on the user's device. The user can review this preview and make corrections or changes as needed. For example, if the user reviews the generated PDF and is satisfied with the content, they can save or print it as is.

[1100] Providing printing options

[1101] After reviewing the preview, users can choose to print. The server outputs the PDF in a printable format and provides a download link to the user's device. It can also directly initiate printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server generates a PDF, which the user can then print as a poster or flyer on their own printer.

[1102] For example, if a user searches for "summer fashion items" and leaves a click and browsing history, the server collects this data, and the AI ​​engine analyzes the user's interests. Based on this, the server recommends the latest summer fashion items and related news, and generates a customized PDF. This PDF may include articles such as "This Summer's Trendy Items" or "Recommended Beach Gear."

[1103] Examples of prompt statements to input into a generative AI model are as follows:

[1104] Based on the user's search history, browsing history, and purchase history, recommend products and related news articles that the user might be interested in. The following is the user's past data.

[1105] Search history: ['Summer dress', 'Flip-flops', 'Travel suitcase']

[1106] Browsing history: ['New beach sandals', 'Summer clothes made of cooling material']

[1107] Purchase history: ['Travel bag', 'Sunglasses']

[1108] Based on the data mentioned above, generate news articles related to products that users are likely to be interested in.

[1109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1110] Step 1:

[1111] Collect user behavior data.

[1112] The server collects user actions on the e-commerce site (product clicks, browsing, search history, purchase history) and stores them in a database. The collected data includes product IDs, operation times, and operation details. This reveals user behavior patterns.

[1113] Step 2:

[1114] Analyze the user's areas of interest.

[1115] The server sends the collected user behavior data to the AI ​​engine for analysis. The AI ​​engine uses TF-IDF vectorization and KMeans clustering techniques to identify categories and topics of high user interest. The input is user behavior data, and the output is cluster data related to areas of interest.

[1116] Step 3:

[1117] Select related products.

[1118] The AI ​​engine selects products related to the user's areas of interest from an external database based on the analysis results. For example, if it determines that the user is interested in summer fashion, it will retrieve the latest information on summer items. The input is cluster data of areas of interest, and the output is a list of recommended products.

[1119] Step 4:

[1120] Analyze the photo folder.

[1121] The server accesses the photo folder stored on the user's device and extracts data using image analysis technology. Specifically, it analyzes image files to generate tags, and then automatically generates related products and articles based on that tag information. The input is the user's photo data, and the output is image tags and their associated product information.

[1122] Step 5:

[1123] Automatically generate articles.

[1124] The server uses an AI engine to generate text-based content based on selected product information and photo analysis results. For example, it can automatically create articles introducing new flip-flops or recommending travel goods. The input is product information and image tag information, and the output is customized text content.

[1125] Step 6:

[1126] Layout the generated articles.

[1127] The server lays out the generated text content in PDF format. Articles are sorted according to user interest and compiled into a PDF with optimal layout. The input is the generated text content, and the output is a pre-layout PDF file.

[1128] Step 7:

[1129] Preview the PDF.

[1130] The terminal visually displays the generated PDF to the user, providing a preview. The user can review the content and make corrections or changes as needed. The input is a pre-layout PDF file, and the output is a preview displayed on the terminal screen.

[1131] Step 8:

[1132] Print the PDF.

[1133] After the user reviews the preview, they select the print option. The server outputs the PDF in a printable format, and the user's device initiates printing to the printer. The input is the previewed PDF file, and the output is the printed document.

[1134] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1135] This invention is a system that collects user behavior data, analyzes the user's areas of interest based on that data, and automatically generates a customized newspaper. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it achieves even more personalized article generation. This system consists of a server, a user terminal, and an AI engine.

[1136] Collection of user information

[1137] The server collects data on user actions within the application. This includes article click data, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, that history is saved in the database.

[1138] Specific example:

[1139] When a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[1140] Emotion recognition by an emotion engine

[1141] The server collects user emotion data through emotion recognition devices (such as cameras and microphones) connected to the user's terminal. The emotion engine recognizes the user's emotions in real time using facial expression analysis and voice analysis. For example, it analyzes the user's facial expressions and tone of voice while they are reading an article to determine whether the user is in an emotional state such as happy, sad, or interested.

[1142] Specific example:

[1143] When a user is reading an article about environmental issues, the camera captures the user's facial expressions, and an emotion engine analyzes those expressions to recognize that the user finds them interesting.

[1144] Interest analysis and news selection

[1145] An AI engine on the server analyzes behavioral and emotional data to identify the user's areas of interest and emotional state. Based on this analysis, it retrieves relevant and up-to-date information from the news API. For example, if a user has a strong interest in environmental issues and shows positive emotions towards that topic, the AI ​​engine will select environmental articles that are relevant to the user.

[1146] Specific example:

[1147] The AI ​​engine retrieves the latest environmental articles from the news API if the user is interested in environmental issues and expresses positive feelings towards them.

[1148] Generating content from photo folders

[1149] The server accesses the user's photo folder and extracts data using image analysis technology. The server obtains the necessary access permissions from the user in advance. For example, the server can detect the user's travel photos and use them to generate article ideas.

[1150] Specific example:

[1151] The server detects the user's travel photos and generates a "user's travelogue" based on those photos.

[1152] Automatic article generation and layout

[1153] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. The generated articles are created using natural language generation (NLG) technology and include sentences and expressions based on specific sentiment data. Furthermore, the generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[1154] Specific example:

[1155] The AI ​​engine generates an article titled "The Future of the Environment" in a positive tone, based on the positive emotions of the user recognized by the emotion engine, and places it at the top.

[1156] Newspaper generation and preview display

[1157] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[1158] Specific example:

[1159] The server generates the newspaper as an A4-sized PDF and displays a preview on the user's device. The user checks the preview and, if satisfied with the content, saves or prints it.

[1160] Providing printing options

[1161] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[1162] Specific example:

[1163] The user reviews the previewed content and clicks the "Print" button. This causes the server to download the newspaper as a PDF and print it from the user's printer.

[1164] As described above, the system of the present invention significantly reduces the effort required from the user, enabling them to easily create, share, and save customized newspapers. Furthermore, by recognizing the user's emotions and generating articles based on those emotions, it can provide a more personalized experience.

[1165] The following describes the processing flow.

[1166] Step 1:

[1167] The server collects user behavior data. This includes click data on articles the user views, viewing time, search history, and interactions with followers. For example, if a user clicks on and reads an article about environmental issues, the article ID, viewing start time, and viewing end time are stored in the database.

[1168] Step 2:

[1169] The server collects user emotion data through emotion recognition devices (such as cameras and microphones) connected to the user's terminal. The emotion engine recognizes the user's emotions in real time using facial expression analysis and voice analysis. For example, it analyzes the user's facial expressions and tone of voice while they are reading an article to determine whether the user is in an emotional state such as happy, sad, or interested.

[1170] Step 3:

[1171] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. This analysis uses machine learning algorithms and natural language processing techniques. For example, if a user reads many articles about environmental issues and expresses positive emotions while doing so, the AI ​​engine will identify that the user is interested in and has a favorable view of environmental issues.

[1172] Step 4:

[1173] The AI ​​engine retrieves relevant and up-to-date information from a news API based on identified areas of interest and emotional state. The news API is a service that provides the latest news articles from various sources on the internet. For example, it retrieves the latest articles on environmental issues and prioritizes selecting those with positive content.

[1174] Step 5:

[1175] The server accesses the user's photo folder and scans the image data within the folder using image analysis technology. The server obtains the necessary access permissions from the user beforehand. For example, the server can detect the user's travel photos and use them to generate article ideas.

[1176] Step 6:

[1177] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. The generated articles are created using natural language generation (NLG) technology. For example, if a user is interested in environmental issues and expresses positive feelings towards them, the AI ​​engine will create an article about environmental issues in a positive tone.

[1178] Step 7:

[1179] The AI ​​engine applies an algorithm to sort the generated articles according to the user's interests. This places the articles the user is most interested in at the top. For example, articles on environmental issues might be placed at the top of the newspaper, followed by travelogues.

[1180] Step 8:

[1181] The server generates a PDF or web version of the original newspaper based on the rearranged articles. The generated newspaper is displayed as a preview on the user's device. The user can review this preview and make any necessary corrections or changes.

[1182] Step 9:

[1183] Users can view a preview of the newspaper and select the option to print it. The server outputs the newspaper in a printable format and provides a download link to the user's device. It is also possible to initiate printing directly by connecting to the user's printer.

[1184] By following these steps, users can easily create, preview, and print a customized newspaper based on their interests and feelings.

[1185] (Example 2)

[1186] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1187] Traditional systems provided personalized content based solely on user behavior data, making it difficult to reflect user emotions or subtle shifts in interests. Furthermore, the lack of a function to generate content from photo folders prevented the delivery of more personalized articles. Additionally, there was a lack of easy ways to print the generated content.

[1188] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1189] In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for collecting and analyzing user sentiment data, means for automatically generating articles based on selected information and sentiment data, and means for laying out the generated articles and outputting them as a newspaper. This makes it possible to utilize both user behavior data and sentiment data to generate more accurate personalized articles and to provide newspapers with content that meets the individual needs of the user.

[1190] "User behavior data" refers to information about the actions and interactions that users perform on the application, and specifically includes article click data, viewing time, search history, and interactions with followers.

[1191] "Areas of interest" refer to the fields or topics that a user is particularly interested in, and are identified from behavioral and emotional data.

[1192] "Emotional data" refers to information about a user's emotional state collected from their facial expressions and voice, including their facial expressions and tone of voice while reading an article.

[1193] "Automatic article generation" refers to the process by which an AI engine automatically creates articles using natural language generation (NLG) technology based on collected and analyzed data.

[1194] "Layout" refers to the process of arranging generated articles in a specific order and format, and includes the process of compiling them into an optimal newspaper format.

[1195] "Outputting as a newspaper" means providing the generated articles to the user's device as a newspaper in PDF or web format.

[1196] "Access permission" refers to the permission a server has to access specific data on a user's device (for example, their photo folder), and this permission is approved by the user in advance.

[1197] A "News API" is an application programming interface used to retrieve the latest news information.

[1198] Natural Language Generation (NLG) is a technology that uses an AI engine to mechanically generate data and output it in a natural language format.

[1199] The "print option" is a feature that outputs the generated newspaper in a printable format (e.g., PDF), allowing the user to physically print it.

[1200] Modes for carrying out the invention

[1201] This invention relates to a system that automatically generates personalized newspapers using user behavioral data and emotional data. This system consists of multiple components, including a server, a user terminal, and an AI engine.

[1202] Collection of user behavior data

[1203] The server collects data on user actions within the application. This data includes article click data, viewing time, search history, and interactions with followers. For example, if a user searches for an article using the keyword "environmental protection" and clicks to read it, the server records the search term, article ID, and viewing time.

[1204] Collection of emotional data

[1205] The server collects user emotion data in real time using emotion recognition devices (such as cameras and microphones) connected to the user's device. The emotion engine recognizes the user's emotions using facial expression analysis and voice analysis and stores them in a database. For example, by analyzing the user's facial expressions and tone of voice while reading an article, it can recognize that the user is finding it interesting.

[1206] Analysis of user areas of interest

[1207] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Based on this analysis, it retrieves relevant and up-to-date information from the news API. For example, if a user has a strong interest in environmental issues and also shows positive emotions towards that topic, the AI ​​engine will select environmental articles that are relevant to the user.

[1208] Generating content from photo folders

[1209] The server uses access permissions obtained from the user in advance to access the user's photo folder and extracts data using image analysis technology. For example, the server detects the user's travel photos and generates article ideas for a "user's travelogue" based on those photos.

[1210] Automatic article generation and layout

[1211] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data obtained from a sentiment engine. This process utilizes natural language generation (NLG) technology. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout.

[1212] Newspaper generation and preview display

[1213] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The user can review this preview and make any necessary corrections or changes. For example, the server generates the newspaper in A4 size PDF format and displays it as a preview on the user's device. The user reviews the content and, if satisfied, saves or prints it.

[1214] Providing printing options

[1215] After reviewing the previewed newspaper, users can choose to print it. The server outputs the newspaper in a printable format (PDF) and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer. For example, when the user clicks the "Print" button, the server provides a download link for the newspaper as a PDF, and the user prints it directly from their printer.

[1216] Example of a prompt

[1217] "If a user is interested in 'environmental protection' and shows positive sentiments, the system will retrieve the latest environmental news and generate articles with a positive tone."

[1218] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1219] Step 1:

[1220] The server collects user behavior data. Specifically, it collects data on the actions users perform on the application and stores it in a database. Inputs include user actions such as clicking on articles or searching, and based on these, click data, viewing time, search history, etc., are recorded on the server. For example, if a user searches for an article using the search term "environmental protection," clicks on it, and views it for 10 minutes, the search term, article ID, and viewing time will be recorded.

[1221] Step 2:

[1222] The server collects emotional data using emotion recognition devices (camera and microphone) connected to the user's device. The input includes the user's facial expressions and voice while they are reading an article, and the emotion engine analyzes this to determine their emotional state. Specifically, the camera captures the user's smile while they are reading an article, and the emotion engine recognizes this data as a "positive emotion" and outputs it.

[1223] Step 3:

[1224] The AI ​​engine on the server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Using the collected behavioral and emotional data as input, it performs data calculations to analyze the user's interests. For example, if the analysis reveals that the user is interested in environmental issues and exhibits positive emotions, the AI ​​engine will use this to identify relevant recent articles on environmental issues as output.

[1225] Step 4:

[1226] The server accesses the photo folder and extracts data using image analysis technology. The input is the photo data in the user's photo folder, which the server analyzes. Specifically, it detects photos of a "trip to Paris" from the user's photo folder and uses this to generate content for an article titled "The User's Travelogue."

[1227] Step 5:

[1228] The AI ​​engine automatically generates articles based on information obtained from a news API, material generated from photo folders, and sentiment data. Input includes the latest information obtained from the news API and various collected data. The AI ​​engine combines this data and generates articles using natural language generation (NLG) technology. The generated articles are sorted according to the user's interests and compiled into a newspaper format with an optimal layout. For example, based on the user's positive emotions recognized by the sentiment engine, an article titled "The Future of the Environment" is generated in a positive tone and placed at the top of the newspaper.

[1229] Step 6:

[1230] The server generates a PDF or web version of the original newspaper based on the generated articles and displays it as a preview on the user's device. The input is article data generated by an AI engine, which is used to create the newspaper file. For example, the server displays a newspaper generated in A4-sized PDF format on the user's device. The user can review the preview and change the order of the articles.

[1231] Step 7:

[1232] After reviewing the previewed newspaper, the user can select a print option. The server outputs the newspaper in a printable format (PDF) and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer. A generated PDF file is provided as input, and when the user clicks the "Print" button, the server provides the PDF as a download link, and printing is initiated directly from the user's printer.

[1233] (Application Example 2)

[1234] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1235] Traditional systems that effectively utilize user behavior data and emotional states to provide information tailored to user interests and emotions are incomplete and require improvement to enhance the user experience. Furthermore, personalized content such as music and news needs to be enhanced to increase user satisfaction.

[1236] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavior data, means for analyzing the user's areas of interest based on the collected behavior data, means for collecting and recognizing user emotion data, means for selecting and generating information based on the collected and recognized emotion data, means for automatically generating articles and content based on the selected information, and means for laying out and outputting the generated articles and content. This makes it possible to provide personalized information based on the user's interests and emotions.

[1237] "Behavioral data" refers to all records of operations and activities performed by users on the system.

[1238] "Areas of interest" refers to specific fields or topics that a user is interested in, derived from their behavioral data.

[1239] "Emotional data" refers to data that indicates the emotional state of a user, as recognized from their facial expressions, voice, and other factors.

[1240] "Emotion recognition means" refers to technology that uses cameras and microphones to analyze a user's emotional state in real time.

[1241] "Information selection means" refers to the function of a system that selects the most suitable content based on user behavior data and emotional data.

[1242] "Automatic article generation method" refers to a technology that automatically creates articles using natural language generation technology based on selected information.

[1243] "Layout techniques" refer to the methods used to arrange generated articles and content in an easy-to-read format and prepare them for the final output.

[1244] Modes for carrying out the invention

[1245] The present invention is a system that collects user behavioral and emotional data and automatically generates personalized content based on that data. Specific embodiments for carrying out the present invention are described below.

[1246] Collection of behavioral data

[1247] The server collects behavioral data from the user's device, such as operation logs, playback history, browsing time, search history, and likes and skips. This data is used to analyze the user's areas of interest. For example, if a user frequently listens to music in the "rock" genre, that information is stored by the server.

[1248] Collection and recognition of emotional data

[1249] The server collects user emotion data using the camera and microphone connected to the user's device. The emotion engine uses specific software (e.g., EmotionRecognizer) to analyze the user's facial expressions and voice in real time and recognize the user's emotional state. For example, if a user is smiling while listening to a particular song, that information is recorded as emotion data.

[1250] Analysis of areas of interest and emotional state

[1251] The collected behavioral and emotional data is stored in a database on the server. The server's AI engine analyzes this data to identify the user's areas of interest and emotional state. This includes what themes the user is interested in and the emotional state in which they enjoy those themes.

[1252] Content selection and automatic generation

[1253] The server retrieves relevant information from news APIs and music libraries based on the analysis results. Using this information, it automatically generates articles and playlists using natural language generation (NLG) technology. Specifically, it uses software like MusicRecommender to create playlists that recommend songs best suited to the user's emotional state. For example, if a user is feeling energetic and exercising, it will generate a playlist of upbeat music.

[1254] Output and preview display

[1255] The generated articles and playlists are displayed on the user's device in preview format. The user can review the generated content and make corrections or changes as needed. For example, if the user is satisfied with the generated playlist, they can play it directly.

[1256] Print and download options available.

[1257] User-created content is output in a printable format. The server generates the content in PDF format and provides a download link to the user's device. It is also possible to initiate direct printing by connecting to the user's printer.

[1258] Example of a prompt

[1259] "Please describe a program that analyzes a user's emotions while they are listening to music and generates the optimal playlist for them."

[1260] Through the above processing, the present invention realizes the provision of personalized content based on user behavioral data and emotional data. By using this system, users can easily obtain content that matches their interests and emotions.

[1261] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1262] Step 1:

[1263] The server collects behavioral data from the user's device. Specifically, it collects data on songs played by the user on the application, playback time, skipped songs, search history, and actions such as liking and sharing, and stores this data in a database. Input data includes operation logs and playback history, while output is behavioral data that serves as the basis for analysis.

[1264] Step 2:

[1265] The server collects and recognizes user emotion data using the terminal's camera and microphone. Specifically, it uses EmotionRecognizer software to analyze the user's facial expressions and voice in real time while they are playing music, and recognizes their emotional state (joy, sadness, excitement, etc.). The input data is real-time facial expressions and voice, and the output is the recognized emotion data.

[1266] Step 3:

[1267] The server analyzes collected behavioral and emotional data to identify the user's areas of interest and emotional state. Specifically, it uses an AI engine to analyze what music genres the user is interested in and under what emotional state they enjoy them. The input data consists of behavioral and emotional data, and the output is data related to the user's areas of interest and emotional state.

[1268] Step 4:

[1269] The server selects and automatically generates relevant content based on the analysis results. Specifically, it uses MusicRecommender software to generate a playlist containing songs best suited to the user's emotional state. The input data includes areas of interest and emotional state data, and the output is the generated playlist.

[1270] Step 5:

[1271] The server displays a preview of the generated playlist on the user's device. Specifically, it sends the generated playlist to the device in real time and displays it within the application. The input data is the generated playlist, and the output is the previewed playlist.

[1272] Step 6:

[1273] Users can review a previewed playlist and make corrections or changes as needed. Specifically, they can add, delete, or rearrange songs within the playlist. The input data is the previewed playlist, and the output is the playlist modified by the user.

[1274] Step 7:

[1275] The server outputs the user-selected playlist in a printable format. Specifically, it generates the playlist in PDF format and provides a download link to the user's device. The input data is a playlist modified or changed by the user, and the output is a PDF playlist file.

[1276] Through these processing steps, users can easily enjoy personalized content based on their interests and emotions.

[1277] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1278] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1279] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1280] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1281] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1282] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1283] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1284] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1285] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1286] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1287] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1288] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1289] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1290] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1291] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1292] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1293] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1294] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1295] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1296] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1297] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1298] The following is further disclosed regarding the embodiments described above.

[1299] (Claim 1)

[1300] Means for collecting user behavior data,

[1301] A means of analyzing user areas of interest based on collected behavioral data,

[1302] A means of automatically selecting relevant information based on the analysis results,

[1303] A method for automatically generating articles based on selected information,

[1304] A means of laying out the generated articles and outputting them as a newspaper,

[1305] A system that includes this.

[1306] (Claim 2)

[1307] The system according to claim 1, further comprising means for analyzing images from a photo folder and generating an article based on those images.

[1308] (Claim 3)

[1309] The system according to claim 1, further comprising means for outputting a newspaper selected by the user in a printable format.

[1310] "Example 1"

[1311] (Claim 1)

[1312] A means of collecting operation data performed on the user's device,

[1313] A means of analyzing collected operational data to identify the user's areas of interest,

[1314] Means for obtaining relevant information from external data sources based on identified areas of interest,

[1315] A means of generating text content based on acquired information,

[1316] A method for rearranging the generated text content in order of user interest and laying it out in a newspaper format,

[1317] A means for previewing a laid-out newspaper on an electronic device,

[1318] A system that includes this.

[1319] (Claim 2)

[1320] The system according to claim 1, further comprising means for analyzing an image from the user's photo storage location and generating text content based on the image.

[1321] (Claim 3)

[1322] The system according to claim 1, further comprising means for outputting a newspaper selected by the user in a printable format.

[1323] "Application Example 1"

[1324] (Claim 1)

[1325] Means for collecting user behavior data,

[1326] A means of analyzing user areas of interest based on collected behavioral data,

[1327] A means of automatically selecting relevant information based on the analysis results,

[1328] A method for automatically generating articles based on selected information,

[1329] A means of laying out the generated articles and outputting them as a newspaper,

[1330] On an e-commerce site, a method for recommending products based on user behavior data,

[1331] A means of outputting articles generated based on the user's areas of interest in PDF format,

[1332] A system that includes this.

[1333] (Claim 2)

[1334] The system according to claim 1, further comprising means for analyzing images from a photo folder and generating an article based on those images.

[1335] (Claim 3)

[1336] The system according to claim 1, further comprising means for outputting a newspaper selected by the user in a printable format.

[1337] "Example 2 of combining an emotion engine"

[1338] (Claim 1)

[1339] Means for collecting user behavior data,

[1340] A means of analyzing user areas of interest based on collected behavioral data,

[1341] A means of automatically selecting relevant information based on the analysis results,

[1342] A means of collecting and analyzing user sentiment data,

[1343] A method for automatically generating articles based on selected information and sentiment data,

[1344] A means of laying out the generated articles and outputting them as a newspaper,

[1345] A system that includes this.

[1346] (Claim 2)

[1347] The system according to claim 1, further comprising means for analyzing images from a photo folder and generating an article based on those images.

[1348] (Claim 3)

[1349] The system according to claim 1, further comprising means for outputting a newspaper selected by the user in a printable format.

[1350] "Application example 2 when combining with an emotional engine"

[1351] (Claim 1)

[1352] Means for collecting user behavior data,

[1353] A means of analyzing user areas of interest based on collected behavioral data,

[1354] A means of automatically selecting relevant information based on the analysis results,

[1355] A method for automatically generating articles based on selected information,

[1356] A means of laying out the generated articles and outputting them as a newspaper,

[1357] Means for collecting and recognizing user sentiment data,

[1358] Means for selecting and generating information based on collected and recognized sentiment data,

[1359] A system that includes this.

[1360] (Claim 2)

[1361] The system according to claim 1, further comprising means for analyzing images from a photo folder and generating an article based on those images.

[1362] (Claim 3)

[1363] The system according to claim 1, further comprising means for outputting a newspaper selected by the user in a printable format. [Explanation of Symbols]

[1364] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for collecting user behavior data, A means of analyzing user areas of interest based on collected behavioral data, A means of automatically selecting relevant information based on the analysis results, A method for automatically generating articles based on selected information, A means of laying out the generated articles and outputting them as a newspaper, A system that includes this.

2. The system according to claim 1, further comprising means for analyzing images from a photo folder and generating articles based on those images.

3. The system according to claim 1, further comprising means for outputting a newspaper selected by the user in a printable format.

Images