System

Abstract

A system is provided.SOLUTION: A system comprising: means for selecting a relevant lecture video based on user input interests and challenges; means for summarizing content of the selected lecture video; means for generating a concrete action plan based on the summarized content; and means for providing the generated action plan to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Today, many people watch lecture videos and read books to seek self-improvement and business knowledge. However, many users are unable to effectively utilize this information and put it into practice. In particular, it is difficult to find the most useful content from the vast amount of information available and apply it to real life. There is a need for a system that can solve this problem and help users efficiently absorb knowledge and put it into concrete action. [Means for solving the problem]

[0005] The present invention provides a system that selects relevant lecture videos based on user-entered interests and challenges, summarizes the content of the videos, and generates a specific action plan. Specifically, the system includes the following means:

[0006] A way for users to input their interests and concerns

[0007] A means of selecting relevant lecture videos based on user-entered information

[0008] A means of summarizing the content of selected lecture videos

[0009] A means of generating concrete action plans from the summary

[0010] A means of providing the generated action plan to the user

[0011] Furthermore, the system uses natural language processing technology to summarize the content of the lecture video and generate a concrete action plan based on the main points contained in the summary, allowing users to efficiently obtain information and create a concrete action plan.

[0012] A "user" is an individual or corporation that uses the system and is responsible for inputting their interests and challenges.

[0013] "Interests" refers to information that indicates the topics or areas in which a user is particularly interested.

[0014] "Issues" are information that indicates the problems the user is currently facing or the situation they would like to improve.

[0015] "Lecture videos" are video content in which experts or celebrities speak on a specific topic, with the aim of sharing their knowledge and experiences.

[0016] "Means" is a general term for methods, techniques, and devices used to achieve a specific purpose.

[0017] "Selection" is the act of choosing the option that best suits a given condition from among many options.

[0018] A summary is a concise summary of the important parts of a longer piece of text or lecture.

[0019] An "action plan" is a specific plan of action to achieve a specific goal.

[0020] "Natural language processing technology" is a computer science technique for analyzing, understanding, and generating human language. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] The present invention is a system that selects lecture videos based on a user's interests and challenges, and provides summaries and specific action plans. Specific embodiments of this system are described in detail below.

[0043] System Overview

[0044] The system includes the following major components:

[0045] 1. User Interface

[0046] 2. Server

[0047] 3. Natural Language Processing Engine

[0048] 4. Database

[0049] Program processing

[0050] 1. User Interface

[0051] Users log in using a device and enter their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"). This information is collected by the device and sent to the server.

[0052] 2. Server-side processing

[0053] The server analyzes the received information and selects relevant lecture videos. The database stores a large number of lecture videos, each of which has associated themes and keywords as metadata. Based on this metadata, the server searches for and selects the video that best suits the user's request.

[0054] 3. Natural Language Processing Engine

[0055] The server sends the transcripts of the selected lecture videos to a natural language processing engine. The engine extracts key points from the transcripts and generates a concise summary. At the same time, it generates a specific action plan based on these key points. For example, if the user's interest is "leadership" and the challenge is "improving team motivation," the engine will generate the following action plan:

[0056] Have one-on-one meetings with the whole team every week

[0057] Set specific goals for your team and regularly review progress

[0058] Host workshops to foster open communication

[0059] 4. Notification of Results

[0060] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[0061] Specific examples

[0062] As a concrete example, let's say a user has an interest in "deepening their knowledge about leadership in the workplace" and a challenge in "raising team morale and improving performance." The user enters this information into the device.

[0063] The device sends this information to the server, which then searches for relevant lecture videos based on the keywords "leadership" and "team motivation." For example, "lectures by renowned leadership experts" might be selected.

[0064] The server sends the transcript of this talk to a natural language processing engine, extracts the main points, and creates a summary, along with an action plan like this:

[0065] Conduct regular one-on-one feedback sessions with team members

[0066] Plan and implement team building activities

[0067] Implement a system to visualize and quantitatively evaluate team performance

[0068] The server sends this information to the user's device, where the user can review it and take action in real life.

[0069] In this way, a system is provided that allows users to efficiently gather knowledge and create specific action plans.

[0070] The processing flow will be explained below.

[0071] Step 1:

[0072] A user logs in using a terminal. After logging in, the user enters their interests (e.g., "Leadership") and the challenges they are currently facing (e.g., "I want to improve my team's motivation").

[0073] Step 2:

[0074] The terminal collects the user's input information and prepares to send it to the server.

[0075] Step 3:

[0076] The device sends the user's input information to the server. The sent information includes the user's interests and concerns.

[0077] Step 4:

[0078] The server analyzes the received user information, and extracts keywords that are most relevant to the user's interests and issues.

[0079] Step 5:

[0080] The server searches a database for relevant lecture videos, which contain metadata such as themes and keywords related to each video.

[0081] Step 6:

[0082] The server selects the most relevant lecture videos based on the user's input interests and challenges.

[0083] Step 7:

[0084] The server retrieves the transcripts of the selected lecture videos. The transcripts are a written version of the video content.

[0085] Step 8:

[0086] The server sends the transcript to a natural language processing engine, which processes the transcript to extract key points.

[0087] Step 9:

[0088] A natural language processing engine analyzes the transcript and summarizes key points and key points, which are concise summaries of the essence of the lecture video.

[0089] Step 10:

[0090] The server generates a concrete action plan based on the summarized content, which is a concrete action plan that the user can carry out in real life.

[0091] Step 11:

[0092] The server sends the generated summary and action plan to the user's terminal.

[0093] Step 12:

[0094] The terminal stores the received summary and action plan and notifies the user that the user has received new information.

[0095] Step 13:

[0096] The user checks the notification and views the summary and action plan displayed on the device, allowing the user to create a specific action plan.

[0097] In this way, users can efficiently absorb the knowledge from the lecture videos and put it into concrete action.

[0098] Example 1

[0099] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] With conventional systems, it was difficult for users to efficiently collect information that matched their interests and challenges and to create specific action plans. In particular, there was a problem in that it took a great deal of time and effort to select highly relevant information from a large amount of media content, summarize that content, and create an action plan.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0102] In this invention, the server includes means for selecting relevant media content based on the interests and issues input by the user, means for summarizing the selected media content, and means for generating a specific action plan based on the summarized content, thereby enabling the user to quickly and efficiently obtain information that is in line with their interests and issues and to create a specific action plan based on that information.

[0103] A "user" is an individual who operates the system and inputs their interests and tasks.

[0104] "Interests" are topics or themes that users particularly want to know about, learn about, or explore further.

[0105] A "challenge" is a problem the user wants to solve or an obstacle they want to overcome.

[0106] "Media content" refers to information or materials that are primarily provided in the form of video, audio, text, etc.

[0107] "Selection tools" are functions that identify and select the most appropriate media content based on user-entered interests and challenges.

[0108] The "summarizing means" is a function that briefly summarizes the content of the selected media content and extracts the main points.

[0109] "Means for generating a concrete action plan" is a function that suggests specific steps or actions that the user should take based on the summarized points.

[0110] The "means of providing" is a function that presents the generated action plan to the user in an easy-to-understand manner, allowing them to use it as a reference for action.

[0111] A "database" is a storage device that stores media content and its associated themes and keywords and allows them to be searched as needed.

[0112] "Natural language processing technology" is a technology for analyzing language data (e.g., transcripts) and understanding their meaning.

[0113] A "transcript" is text data that converts the content of video, audio, etc. into text.

[0114] "Means of analysis" refers to the function of analyzing data such as user input and transcripts and extracting necessary information.

[0115] The present invention is a system that selects optimal media content based on a user's interests and challenges, summarizes the content, and provides a specific action plan. The system mainly consists of the following components:

[0116] 1. User Interface

[0117] 2. Server

[0118] 3. Natural Language Processing Engine

[0119] 4. Database

[0120] User Interface

[0121] Users log in to the system using a terminal and enter their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"), which are collected by the terminal and sent to the server.

[0122] server

[0123] The server analyzes the information received from the user and selects relevant media content. The server is connected to a database that stores a large number of media contents, and each content is assigned related themes and keywords as metadata. The server uses this metadata to search for and select the content that best suits the user's request.

[0124] Natural Language Processing Engine

[0125] The server sends the transcript of the selected media content to a natural language processing engine, which extracts key points from the transcript and generates a concise summary, along with a concrete plan of action based on those key points.

[0126] For example, if the user's interest is "leadership" and the challenge is "improving team motivation," the engine will generate an action plan like this:

[0127] Have one-on-one meetings with the whole team every week

[0128] Set specific goals for your team and regularly review progress

[0129] Host workshops to foster open communication

[0130] Notification of results

[0131] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[0132] Specific examples

[0133] As a concrete example, let's say a user has an interest in "deepening their knowledge about leadership in the workplace" and a challenge in "raising team morale and improving performance." The user enters this information into the device.

[0134] The device sends this information to a server, which then searches for relevant media content based on the keywords "leadership" and "team motivation." For example, it might select "video lectures by renowned leadership experts" from a database.

[0135] The server sends the transcript of this talk to a natural language processing engine, extracts the main points, and generates a summary, along with an action plan like this:

[0136] Conduct regular one-on-one feedback sessions with team members

[0137] Plan and implement team building activities

[0138] Implement a system to visualize and quantitatively evaluate team performance

[0139] Prompt Sentence Examples

[0140] "Based on the leadership presentation video, what is your specific action plan for improving team motivation?"

[0141] "Find a video of a talk that will help motivate your team and provide a summary and actionable plan of action."

[0142] This system allows users to efficiently gather knowledge and create specific action plans, thereby providing effective support for achieving individual goals.

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1:

[0145] User Login

[0146] A user logs into the system using a terminal. The user accesses the login screen and enters their username and password. The terminal sends this information to the server, and if authentication is successful, the home screen is displayed.

[0147] Input: The username and password entered by the user

[0148] Output: Show user's home screen upon successful authentication

[0149] Step 2:

[0150] Input of user interests and issues

[0151] The user enters their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"). The user enters these into the form on the home screen and presses the submit button. The input information is sent from the device to the server.

[0152] Input: User-entered interests and challenges

[0153] Output: User interests and challenges sent to the server

[0154] Step 3:

[0155] Sending user information

[0156] The device sends the entered information to the server. The device summarizes the user's interests and challenges and sends them to the server in the form of an HTTP request. The server receives this request and begins processing.

[0157] Input: User interests and challenges sent from the device

[0158] Output: User information in a format that can be parsed by the server

[0159] Step 4:

[0160] Server information analysis

[0161] The server analyzes the received user information and extracts relevant keywords. The server uses an analysis module to identify themes related to interests and issues. Based on the extracted keywords, the next step is to search for related media content.

[0162] Input: User information (interests and issues)

[0163] Output: Extracted related keywords

[0164] Step 5:

[0165] Selection of relevant media content

[0166] The server searches the database and selects relevant media content based on the extracted keywords. The server sends a keyword search query to the database and retrieves a list of the most relevant media content.

[0167] Input: Extracted keywords

[0168] Output: A list of related media content

[0169] Step 6:

[0170] Sending a transcript

[0171] The server sends the most relevant media content transcripts to the natural language processing engine, which then parses the transcripts, which are a transcription of the audio from the video.

[0172] Input: Transcript of associated media content

[0173] Output: Transcript sent to the natural language processing engine

[0174] Step 7:

[0175] Generate summaries and action plans

[0176] A natural language processing engine analyzes the transcript and generates a summary and a concrete plan of action. The engine extracts key points from the transcript, creates a concise summary, and suggests a concrete plan of action based on these points.

[0177] Input: Transcript

[0178] Output: Summary and action plan

[0179] Step 8:

[0180] Notification of results

[0181] The server sends the generated summary and action plan to the user's device, where the user can review the information and plan a specific course of action. The server sends this information in the form of an HTTP response.

[0182] Input: Summary and Action Plan

[0183] Output: Summary and action plan sent to the user's device

[0184] (Application example 1)

[0185] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0186] Conventional lecture video delivery systems are limited to providing video summaries and specific action plans, and have the problem of not directly contributing to improving users' actual work or performance. Furthermore, the observation of users' work and suggestions for improvement themselves are often done manually, which is inefficient. The present invention aims to improve work efficiency and effectiveness by observing users' work and providing specific action plans in real time based on the observations.

[0187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0188] In this invention, the server includes means for selecting relevant lecture videos based on the interests and challenges input by the user, means for summarizing the content of the selected lecture videos, means for generating a specific action plan based on the summarized content, means for observing the user's work with a robot and analyzing the observation data, and means for providing the generated action plan to the user via the robot's display and voice notification. This makes it possible to monitor the user's work status in real time and instantly provide optimal improvement suggestions.

[0189] "Means for selecting relevant lecture videos based on the interests and issues entered by the user" refers to a function that analyzes the interest and issue information entered by the user using an input device and automatically selects the video content that is most relevant to that information from a database.

[0190] "Means for summarizing the content of selected lecture videos" refers to a function that analyzes the content of selected videos using natural language processing technology and provides important points in a shortened format.

[0191] "Means for generating specific action plans based on the summarized content" refers to the function of generating specific action plans that can actually be implemented based on the main points extracted from the content of the summarized lecture video.

[0192] "Means for the robot to observe the user's work and analyze the observed data" refers to the function of the robot to monitor the user's work status using cameras and sensors, and to collect and analyze the data in real time.

[0193] "Means for providing the generated action plan to the user through the robot's display and voice notification" refers to the function of displaying the generated specific action plan on the robot's display and communicating it to the user through voice notification.

[0194] "Means for receiving input of interests and challenges from users" refers to a function that provides an interface for users to input their interests and challenges they are facing into the system.

[0195] "Natural language processing technology" is a technology that enables machines to understand and process human language, and refers to technology that analyzes text, generates summaries, extracts meaning, etc.

[0196] This invention is a system that selects relevant lecture videos based on a user's interests and challenges, and provides summaries and specific action plans. Furthermore, the system aims to support factory robot operations by observing the user's work and making suggestions for work improvement in real time. A specific embodiment of this system is described in detail below.

[0197] System Overview

[0198] The system includes the following major components:

[0199] 1. User Interface

[0200] 2. Server

[0201] 3. Natural Language Processing Engine

[0202] 4. Database

[0203] 5. Factory robots

[0204] Program processing

[0205] 1. User Interface

[0206] Users log in using their devices and enter their interests (e.g., "quality control") and challenges (e.g., "reducing lead time"). This information is collected by the device and sent to the server, which then obtains basic data to understand the user's needs and select appropriate lecture videos.

[0207] 2. Server-side processing

[0208] The server analyzes the received information and selects relevant lecture videos. The database stores a large number of lecture videos, each of which is assigned metadata such as related themes and keywords. Based on this metadata, the server searches for and selects the video that best suits the user's request. This process uses advanced data analysis algorithms.

[0209] 3. Natural Language Processing Engine

[0210] The server sends the transcripts of the selected lecture videos to a natural language processing engine. This engine uses the transformers library to extract key points from the transcripts and generate summaries. At the same time, it generates specific action plans based on these key points. For example, if the user's interest is "quality control" and the challenge is "reducing lead time," the engine will generate the following action plan:

[0211] Increase the frequency of quality inspections

[0212] Review of standard operating procedures

[0213] Introducing a system to visualize work efficiency

[0214] 4. Observing factory robots at work

[0215] Factory robots use cameras and sensors to observe the user's work in real time. This observation data is sent to a server as needed and analyzed. The work data observed by the robot is associated with themes and issues entered by the user and used to generate more accurate action plans.

[0216] 5. Notification of Results

[0217] The generated summary and action plan are sent from the server to the user's device and the factory robot, which then displays the information on the display and notifies the user via voice notification. The results can also be viewed via a smartphone app.

[0218] Specific examples

[0219] As a concrete example, let's say a user has the interests and challenges of "I want to deepen my knowledge about quality control" and "I want to shorten lead time." The user enters this information into their device, which then sends it to the server. The server searches for lecture videos on "quality control" and selects appropriate videos. The transcripts of the selected videos are summarized using a natural language processing engine, and specific action plans such as "increase the frequency of quality inspections" and "review standard operating procedures" are generated.

[0220] Prompt Sentence Examples

[0221] The user's interests and challenges are "quality control" and "shortening lead time." Please select a lecture video that meets these criteria and generate a summary of it. Also, please provide a specific action plan for improvement based on these criteria.

[0222] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0223] Step 1:

[0224] A user logs in using a terminal and inputs their interests and challenges. The inputted interest is "quality control" and the challenge is "shortening lead time." This information is collected by the terminal and sent to the server. The input here is the user's text data, and the output is a data packet that sends this to the server.

[0225] Step 2:

[0226] The server analyzes the received data on the user's interests and challenges using a keyword extraction algorithm. The server matches the metadata of the lecture videos in the database with the keywords contained in the user's interests and challenges to select relevant lecture videos. The input is keywords and metadata, and the output is a list of the most suitable lecture videos.

[0227] Step 3:

[0228] We take the transcripts of selected talk videos and send them to a natural language processing engine, which uses the transformers library to extract key points from the transcripts and generate concise summaries. The input is the talk video transcript, and the output is the summary text.

[0229] Step 4:

[0230] The same natural language processing engine is used to generate specific action plans from the summary. The engine further analyzes the summary text and generates actionable plans, such as "increase the frequency of quality inspections" or "review standard operating procedures." The input is the summary text, and the output is a list of action plans.

[0231] Step 5:

[0232] Factory robots use cameras and sensors to observe the user's work. This observation data is sent to a server in real time. The server analyzes the observation data and evaluates the user's work status. The input is the observation data, and the output is the analysis result.

[0233] Step 6:

[0234] The server adjusts and optimizes the generated action plan based on the analysis results. The observation data and the action plan are collated to provide specific suggestions tailored to the user's current situation. The input is the observation data and the existing action plan, and the output is an optimized action plan.

[0235] Step 7:

[0236] The generated summary and action plan are sent from the server to the user's device and the factory robot. The factory robot displays the information on a display and communicates it to the user through a voice notification. The same information can also be viewed on the user's device. The input is the optimized action plan, and the output is the display and voice notification.

[0237] An example prompt is:

[0238] "The user's interests and challenges are 'quality control' and 'shortening lead time.' Please select a lecture video that meets these criteria and generate a summary of it. Also, please provide a specific action plan for improvement based on these criteria."

[0239] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0240] The present invention combines a system that selects relevant lecture videos based on a user's interests and challenges, summarizes the content, and provides a specific action plan with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[0241] System Overview

[0242] The system includes the following major components:

[0243] 1. User Interface

[0244] 2. Server

[0245] 3. Natural Language Processing Engine

[0246] 4. Emotion Engine

[0247] 5. Database

[0248] Program processing

[0249] 1. User Interface

[0250] Users log in using their devices and input their interests and the challenges they are currently facing. This information is collected by the device and sent to the server. The emotion engine then recognizes the user's emotions based on the information they input and their interactions.

[0251] 2. Server-side processing

[0252] The server analyzes the received information and selects relevant lecture videos, taking into account the user's recognized emotions. The database stores a large number of lecture videos, each of which has associated themes and keywords as metadata. The server combines this metadata with emotional information to search for and select the most appropriate video.

[0253] 3. Emotion Engine

[0254] The emotion engine analyzes user input and interactions (e.g., keyboard input speed, facial expression analysis, etc.) to recognize the user's emotional state. This information is sent to the server and used to select videos and generate action plans.

[0255] 4. Natural Language Processing Engine

[0256] The server sends the transcripts of the selected lecture videos to a natural language processing engine, which extracts key points from the transcripts and generates a concise summary. At the same time, a specific action plan is generated based on these key points. Taking into account the results of the emotion engine, an appropriate action plan is created according to the user's emotional state.

[0257] 5. Notification of Results

[0258] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[0259] Specific examples

[0260] For example, if a user has an interest in "I want to deepen my knowledge about leadership in the workplace" and a goal in "I want to improve team morale and performance," if the emotion engine recognizes the user's emotional state as "high stress," it will also take this information into account.

[0261] The user enters this information into the device, which then sends it to the server. The server searches for relevant lecture videos based on the keywords "leadership" and "team motivation," while also taking into account the user's current emotional state of "high stress." For example, a "lecture on the psychological aspects of leadership" might be selected.

[0262] The server sends the transcript of this talk to a natural language processing engine, which summarizes the main points and generates an action plan like this:

[0263] Use relaxation techniques regularly

[0264] Facilitate open stress management dialogue within your team

[0265] Introduce flexible scheduling

[0266] The server sends this information to the user's device, where the user can review it and take action in real life.

[0267] In this way, a system is provided that allows the user to efficiently gather knowledge and create a specific action plan according to their emotional state.

[0268] The processing flow will be explained below.

[0269] Step 1:

[0270] A user logs in using a terminal. After logging in, the user enters their interests (e.g., "Leadership") and the challenges they are currently facing (e.g., "I want to improve my team's motivation").

[0271] Step 2:

[0272] The emotion engine analyzes the user's input information and interactions (e.g., keyboard input speed, facial expression analysis, etc.) and recognizes the user's emotional state (e.g., "high stress").

[0273] Step 3:

[0274] The terminal collects the user's input information and emotional state and prepares to send it to the server.

[0275] Step 4:

[0276] The terminal transmits the user's input information and emotional state to the server, and the transmitted information includes the user's interests, concerns, and emotional state.

[0277] Step 5:

[0278] The server analyzes the received user information, extracting keywords that are most relevant to the user's interests and issues, and also taking into account the user's emotional state.

[0279] Step 6:

[0280] The server searches the database for relevant lecture videos. The lecture video database contains metadata such as themes and keywords related to each video. The server combines this metadata with emotional information to search for and select the most appropriate video.

[0281] Step 7:

[0282] The server selects the most relevant lecture videos based on the user's input interests and challenges, as well as their perceived emotional state.

[0283] Step 8:

[0284] The server retrieves the transcripts of the selected lecture videos. The transcripts are a written version of the video content.

[0285] Step 9:

[0286] The server sends the transcript to a natural language processing engine, which processes the transcript to extract key points.

[0287] Step 10:

[0288] A natural language processing engine analyzes the transcript and summarizes key points and key points, which are concise summaries of the essence of the lecture video.

[0289] Step 11:

[0290] The server generates a specific action plan based on the summary, taking into account the user's emotional state. For example, if the user is feeling stressed, the action plan may include stress management and relaxation.

[0291] Step 12:

[0292] The server sends the generated summary and action plan to the user's terminal.

[0293] Step 13:

[0294] The terminal stores the received summary and action plan and notifies the user that the user has received new information.

[0295] Step 14:

[0296] The user checks the notification and views the summary and action plan displayed on the device, allowing the user to create a specific action plan.

[0297] In this way, a system is provided that allows users to efficiently absorb knowledge from lecture videos and create specific action plans according to their emotional state.

[0298] Example 2

[0299] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0300] Previous systems were unable to consider the user's emotional state when providing relevant lecture videos based on the user's interests and challenges. As a result, they were unable to provide optimal information for the user or generate specific and effective action plans. Furthermore, they were unable to appropriately change feedback according to the user's individual emotional state, which limited the user experience.

[0301] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0302] In this invention, the server includes a means for selecting relevant lecture videos based on the interests and issues input by the user, a means for recognizing emotions from the user's input information and interactions, and a means for selecting relevant videos in consideration of the recognized emotional information. This makes it possible to select the most appropriate lecture video taking into consideration the user's emotional state and to provide a specific action plan according to the emotion.

[0303] "User" means any individual or entity that uses the System.

[0304] "Interests" refers to the themes and topics that interest users.

[0305] "Challenge" refers to the problem the user is currently facing or the issue they want to solve.

[0306] A "lecture video" is video content in which a specialist or expert explains a specific topic.

[0307] "Means" refers to methods or techniques for achieving a specific purpose.

[0308] A "summary" is a concise summary of the main points extracted from the content of a lecture video.

[0309] An "action plan" is a set of specific recommended actions that can be taken based on the summarized content.

[0310] "Means for recognizing emotions" refers to technology that analyzes a user's emotional state based on their behavior, expressions, etc. Examples include keyboard input speed and facial expression analysis.

[0311] "Natural language processing technology" is a technology that allows machines to analyze text data, understand its meaning, and process human language.

[0312] "Metadata" is additional information that accompanies the data, and includes, for example, the theme and keywords of the video.

[0313] A "server" is a computer system that receives requests from users and processes them in cooperation with a database.

[0314] A "database" is a system that organizes, manages, and stores a variety of data.

[0315] "Interaction" refers to a series of operations, actions, and inputs that a user makes with a system.

[0316] The present invention combines a system that selects relevant lecture videos based on a user's interests and challenges, summarizes the content, and provides a specific action plan with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[0317] User Interface

[0318] Users log in to the system using a terminal. After logging in, they input their interests (e.g., strengthening leadership) and challenges (e.g., improving team morale). The emotion engine also analyzes the user's emotional state based on facial expressions and keyboard typing speed. This allows the system to take the user's emotions into account.

[0319] Sending information to the server

[0320] The device sends the collected user input information and emotion information to the server, which then performs subsequent processing based on this information.

[0321] Selection of lecture videos

[0322] The server analyzes the received information and searches for relevant lecture videos in its database. The database stores multiple lecture videos, each of which is assigned metadata such as theme, keywords, and emotion tags. The server compares this metadata with user information to select the most suitable video.

[0323] Processing the transcript

[0324] The server sends the transcripts (written transcripts of what was said) of the selected lecture videos to a natural language processing engine, which analyzes the transcripts and extracts key points.

[0325] Generate a summary and action plan

[0326] The natural language processing engine generates a summary based on the extracted key points, and then takes into account the emotional information from the emotion engine to create a specific action plan suited to the user's situation, such as "introducing relaxation techniques" for users experiencing high stress.

[0327] Notifying users of the results

[0328] The server sends the generated summary and action plan to the user's device, where the user can review the information and create a workable action plan for real life.

[0329] Specific examples

[0330] For example, consider a user who has an interest in "I want to deepen my knowledge about leadership in the workplace" and a goal to "improve team morale and performance." If the emotion engine recognizes the user's emotional state as "high stress," this information will also be taken into account.

[0331] The user enters this information into the device, which then sends it to the server. The server then searches for relevant lecture videos based on keywords like "leadership" and "team motivation," taking into account the user's emotional state of "high stress." For example, a video titled "Lectures on the psychological aspects of leadership" might be selected.

[0332] The server sends the transcript of this talk to a natural language processing engine, which summarizes the main points and generates an action plan like this:

[0333] Use relaxation techniques regularly

[0334] Facilitate open stress management dialogue within your team

[0335] Introduce flexible scheduling

[0336] The server sends this information to the user's device, where the user can review it and take action in real life.

[0337] Prompt Sentence Examples

[0338] "I would like to deepen my knowledge of leadership in the workplace. I would also like to motivate my team and improve their performance. Based on this information, please suggest appropriate talk videos, provide summaries, and concrete action plans."

[0339] In this way, this system not only provides information based on the user's interests and challenges, but also provides optimal content and specific action plans that take into account the user's emotional state.

[0340] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0341] Step 1:

[0342] A user logs into the system using a terminal. After logging in, the user inputs their interests (e.g., strengthening leadership) and challenges (e.g., improving team morale). The terminal collects the input information and prepares it. The input at this time is text data related to the user's interests and challenges.

[0343] Step 2:

[0344] The device sends the collected input information to the server. Specifically, it sends data using an HTTP POST request. This request includes text data of the user's input interests and tasks. The output is the input information data received by the server.

[0345] Step 3:

[0346] The server analyzes the received input information and uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's input information and interactions (e.g., keyboard input speed, facial expression analysis, etc.) to extract the user's emotional state. For example, emotional data such as "high stress" or "low motivation" can be obtained.

[0347] Step 4:

[0348] The server searches for relevant lecture videos in the database, taking into account the emotional information from the emotion engine and user input information. The database stores metadata of lecture videos (e.g., themes, keywords, emotional tags), and matches this with user information to select the most suitable video. The input is user information and emotional information, and the output is the selected lecture video.

[0349] Step 5:

[0350] The server sends the transcripts of the selected lecture videos to a natural language processing engine, which analyzes the transcripts and extracts key points. The input is the lecture video transcript, and the output is a summarized version of the key points.

[0351] Step 6:

[0352] The natural language processing engine generates a summary based on the extracted key points, and then generates a specific action plan taking into account the emotional information obtained from the emotion engine. For example, if the user is "highly stressed," the following action plan will be generated:

[0353] Use relaxation techniques regularly

[0354] Facilitate open stress management dialogue within your team

[0355] Introduce flexible scheduling

[0356] The input is key points and emotional information, and the output is a summary and an action plan.

[0357] Step 7:

[0358] The server sends the generated summary and action plan to the user's device. The user can check this information on the device and create a specific action plan. The input is the summary and action plan, and the output is information provided to the user. A specific example of operation would be when the user clicks the "View Results" button on the device to check the provided summary and action plan.

[0359] (Application example 2)

[0360] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0361] While users need to properly learn about home security knowledge and techniques and take concrete measures, it is difficult to select appropriate content from the vast amount of information available. Furthermore, there are few systems that provide appropriate content and measures based on the user's emotional state and stress level. Therefore, there is a need for a system that provides information optimized for each user's individual situation and provides a concrete action plan.

[0362] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0363] In this invention, the server includes means for selecting relevant video content based on interests and issues input by the user, means for summarizing the content of the selected video content, means for generating a specific action plan based on the summarized content, means for recognizing the emotional state of the user using emotion recognition technology, and means for selecting appropriate video content based on the emotional state of the user, thereby enabling the user to efficiently obtain appropriate security content and an action plan.

[0364] "User" refers to any individual or organization that uses this system.

[0365] "Interests" refers to the information users want to know and the topics they want to learn about.

[0366] A "challenge" refers to a problem that a user is currently facing or wants to solve.

[0367] "Video content" refers to information in video format stored on the Internet or in a database.

[0368] A "summary" refers to information that concisely summarizes the main points of the video content.

[0369] An "action plan" is a plan that indicates specific actions that a user should take based on the summarized information.

[0370] "Emotion recognition technology" refers to technology that analyzes a user's facial expressions, behavior, etc. to identify their emotional state.

[0371] "Emotional state" refers to a user's current psychological state or mood.

[0372] "Means for selection" refers to a method or system for selecting relevant video content based on a user's interests and challenges.

[0373] "Means for providing" refers to a method or system for notifying the user of the generated action plan or summary information.

[0374] "Means for summarizing" refers to a method or system that uses natural language processing technology to concisely summarize the content of video content.

[0375] "Means for generating" refers to a method or system that generates an action plan based on the summarized content.

[0376] "Means for recognizing" refers to a method or system that uses emotion recognition technology to analyze and identify a user's emotional state.

[0377] MODE FOR CARRYING OUT THE INVENTION

[0378] System Overview

[0379] This invention combines emotion recognition technology with a system that selects appropriate video content based on the user's interests and challenges, and provides a summary and a concrete action plan. The system includes the following main components:

[0380] User interface: Users input their interests and concerns using their smartphones.

[0381] Server: Analyzes the input information and selects relevant video content.

[0382] Emotion Recognition Engine: Recognizes the user's emotional state.

[0383] Natural language processing engine: Summarizes the content of selected videos and generates specific action plans.

[0384] Database: Stores video content and makes it searchable.

[0385] Hardware and Software Configuration

[0386] Smartphones: Used as a user interface, they use a camera to capture the user's facial expressions.

[0387] Emotion Recognition Engine: Uses Affectiva SDK or equivalent emotion recognition technology.

[0388] Natural language processing engine: Uses OpenAI GPT or a similar generative model.

[0389] Database: Use AWS RDS or Google Firebase to store video content and its metadata.

[0390] Processing flow

[0391] User Interface

[0392] Users input their interests and current challenges through a smartphone application, and the smartphone camera captures the user's facial expressions and sends them to an emotion recognition engine.

[0393] emotion recognition technology

[0394] The emotion recognition engine (Affectiva SDK) analyzes the captured facial images of the user and recognizes their emotional state. This emotional information is sent to the server along with the user's interests and concerns.

[0395] Video content selection

[0396] The server analyzes the received information (interests, issues, emotional state) and searches for relevant video content from a database, using video metadata and tags.

[0397] Generate summaries and action plans

[0398] The server then sends the transcripts of the selected video content to a natural language processing engine (OpenAI GPT), which extracts key points from the transcript and generates a concise summary. At the same time, it generates a specific action plan based on these key points. Taking into account the user's emotional state, an appropriate action plan is created based on the user's psychological situation.

[0399] Notification of results

[0400] The generated summary and action plan are sent from the server to the user's smartphone, where the user can review the information and implement specific action plans.

[0401] Specific examples

[0402] As a concrete example, let's say a user has an interest in "deepening their knowledge about home security" and a goal of "strengthening measures against suspicious individuals." Furthermore, if the emotion recognition technology recognizes the user's emotional state as "high stress," this information is also taken into consideration. The user enters this information into their smartphone, captures their facial expressions with the camera, and sends them to the emotion recognition engine. The server searches for videos related to "home security" and "measures against suspicious individuals," taking their emotional state into consideration.

[0403] For example, if the selected video is a "Lecture on Home Security," the server sends the transcript of this lecture to a natural language processing engine to generate a summary and action plan, which could look something like this:

[0404] Summary: Locking gates and installing security cameras are important ways to improve home security.

[0405] Action plan: Select the type of security camera that will be effective in preventing suspicious individuals and determine where to install it.

[0406] Prompt Sentence Examples

[0407] User Input: I want to solve home security issues

[0408] Emotion: High stress

[0409] Prompt for the generative AI model:

[0410] Create a summary of your home security measures and provide a specific action plan for high-stress situations.

[0411] This allows users to efficiently obtain appropriate security content and action plans.

[0412] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0413] Step 1:

[0414] Users use their smartphones to input their interests and current challenges. This information is sent from the device to the server. Input can be done using text boxes or selection lists.

[0415] Input: Interests (e.g., "Home security"), Issues (e.g., "Countermeasures against suspicious persons")

[0416] Output: User interest and issue data sent to the server

[0417] Step 2:

[0418] The user captures their facial expression using the smartphone camera. The emotion recognition engine analyzes the facial image and recognizes the user's emotional state. The recognition results are sent to the server.

[0419] Input: facial expression image

[0420] Output: User's emotional state (e.g. "high stress")

[0421] Step 3:

[0422] The server analyzes the received interest, task, and emotional state data and searches a database for relevant video content, using video metadata and tags.

[0423] Input: Interests, tasks, and emotional state data

[0424] Output: Selected video content (e.g., "Lecture video on home security measures")

[0425] Step 4:

[0426] The server sends the transcript of the selected video content to a natural language processing engine, which generates a summary and a concrete action plan, automatically generated using a generative AI model.

[0427] Input: Transcript of video content

[0428] Output: Summary and action plan (e.g., "Lock the gates and install security cameras")

[0429] Step 5:

[0430] The server sends the generated summary and action plan to the user's device, where the user can review it and take specific action.

[0431] Input: Summary and Action Plan

[0432] Output: Summary and action plan displayed on user terminal

[0433] This allows users to select appropriate video content based on their interests and challenges, summarize the content in an easy-to-understand format, and put a concrete action plan into action.

[0434] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0435] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0436] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0437] [Second embodiment]

[0438] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0439] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0440] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0441] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0442] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0443] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0444] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0445] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0446] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0447] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0448] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0449] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0450] The present invention is a system that selects lecture videos based on a user's interests and challenges, and provides summaries and specific action plans. Specific embodiments of this system are described in detail below.

[0451] System Overview

[0452] The system includes the following major components:

[0453] 1. User Interface

[0454] 2. Server

[0455] 3. Natural Language Processing Engine

[0456] 4. Database

[0457] Program processing

[0458] 1. User Interface

[0459] Users log in using a device and enter their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"). This information is collected by the device and sent to the server.

[0460] 2. Server-side processing

[0461] The server analyzes the received information and selects relevant lecture videos. The database stores a large number of lecture videos, each of which has associated themes and keywords as metadata. Based on this metadata, the server searches for and selects the video that best suits the user's request.

[0462] 3. Natural Language Processing Engine

[0463] The server sends the transcripts of the selected lecture videos to a natural language processing engine. The engine extracts key points from the transcripts and generates a concise summary. At the same time, it generates a specific action plan based on these key points. For example, if the user's interest is "leadership" and the challenge is "improving team motivation," the engine will generate the following action plan:

[0464] Have one-on-one meetings with the whole team every week

[0465] Set specific goals for your team and regularly review progress

[0466] Host workshops to foster open communication

[0467] 4. Notification of Results

[0468] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[0469] Specific examples

[0470] As a concrete example, let's say a user has an interest in "deepening their knowledge about leadership in the workplace" and a challenge in "raising team morale and improving performance." The user enters this information into the device.

[0471] The device sends this information to the server, which then searches for relevant lecture videos based on the keywords "leadership" and "team motivation." For example, "lectures by renowned leadership experts" might be selected.

[0472] The server sends the transcript of this talk to a natural language processing engine, extracts the main points, and creates a summary, along with an action plan like this:

[0473] Conduct regular one-on-one feedback sessions with team members

[0474] Plan and implement team building activities

[0475] Implement a system to visualize and quantitatively evaluate team performance

[0476] The server sends this information to the user's device, where the user can review it and take action in real life.

[0477] In this way, a system is provided that allows users to efficiently gather knowledge and create specific action plans.

[0478] The processing flow will be explained below.

[0479] Step 1:

[0480] A user logs in using a terminal. After logging in, the user enters their interests (e.g., "Leadership") and the challenges they are currently facing (e.g., "I want to improve my team's motivation").

[0481] Step 2:

[0482] The terminal collects the user's input information and prepares to send it to the server.

[0483] Step 3:

[0484] The device sends the user's input information to the server. The sent information includes the user's interests and concerns.

[0485] Step 4:

[0486] The server analyzes the received user information, and extracts keywords that are most relevant to the user's interests and issues.

[0487] Step 5:

[0488] The server searches a database for relevant lecture videos, which contain metadata such as themes and keywords related to each video.

[0489] Step 6:

[0490] The server selects the most relevant lecture videos based on the user's input interests and challenges.

[0491] Step 7:

[0492] The server retrieves the transcripts of the selected lecture videos. The transcripts are a written version of the video content.

[0493] Step 8:

[0494] The server sends the transcript to a natural language processing engine, which processes the transcript to extract key points.

[0495] Step 9:

[0496] A natural language processing engine analyzes the transcript and summarizes key points and key points, which are concise summaries of the essence of the lecture video.

[0497] Step 10:

[0498] The server generates a concrete action plan based on the summarized content, which is a concrete action plan that the user can carry out in real life.

[0499] Step 11:

[0500] The server sends the generated summary and action plan to the user's terminal.

[0501] Step 12:

[0502] The terminal stores the received summary and action plan and notifies the user that the user has received new information.

[0503] Step 13:

[0504] The user checks the notification and views the summary and action plan displayed on the device, allowing the user to create a specific action plan.

[0505] In this way, users can efficiently absorb the knowledge from the lecture videos and put it into concrete action.

[0506] Example 1

[0507] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0508] With conventional systems, it was difficult for users to efficiently collect information that matched their interests and challenges and to create specific action plans. In particular, there was a problem in that it took a great deal of time and effort to select highly relevant information from a large amount of media content, summarize that content, and create an action plan.

[0509] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0510] In this invention, the server includes means for selecting relevant media content based on the interests and issues input by the user, means for summarizing the selected media content, and means for generating a specific action plan based on the summarized content, thereby enabling the user to quickly and efficiently obtain information that is in line with their interests and issues and to create a specific action plan based on that information.

[0511] A "user" is an individual who operates the system and inputs their interests and tasks.

[0512] "Interests" are topics or themes that users particularly want to know about, learn about, or explore further.

[0513] A "challenge" is a problem the user wants to solve or an obstacle they want to overcome.

[0514] "Media content" refers to information or materials that are primarily provided in the form of video, audio, text, etc.

[0515] "Selection tools" are functions that identify and select the most appropriate media content based on user-entered interests and challenges.

[0516] The "summarizing means" is a function that briefly summarizes the content of the selected media content and extracts the main points.

[0517] "Means for generating a concrete action plan" is a function that suggests specific steps or actions that the user should take based on the summarized points.

[0518] The "means of providing" is a function that presents the generated action plan to the user in an easy-to-understand manner, allowing them to use it as a reference for action.

[0519] A "database" is a storage device that stores media content and its associated themes and keywords and allows them to be searched as needed.

[0520] "Natural language processing technology" is a technology for analyzing language data (e.g., transcripts) and understanding their meaning.

[0521] A "transcript" is text data that converts the content of video, audio, etc. into text.

[0522] "Means of analysis" refers to the function of analyzing data such as user input and transcripts and extracting necessary information.

[0523] The present invention is a system that selects optimal media content based on a user's interests and challenges, summarizes the content, and provides a specific action plan. The system mainly consists of the following components:

[0524] 1. User Interface

[0525] 2. Server

[0526] 3. Natural Language Processing Engine

[0527] 4. Database

[0528] User Interface

[0529] Users log in to the system using a terminal and enter their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"), which are collected by the terminal and sent to the server.

[0530] server

[0531] The server analyzes the information received from the user and selects relevant media content. The server is connected to a database that stores a large number of media contents, and each content is assigned related themes and keywords as metadata. The server uses this metadata to search for and select the content that best suits the user's request.

[0532] Natural Language Processing Engine

[0533] The server sends the transcript of the selected media content to a natural language processing engine, which extracts key points from the transcript and generates a concise summary, along with a concrete plan of action based on those key points.

[0534] For example, if the user's interest is "leadership" and the challenge is "improving team motivation," the engine will generate an action plan like this:

[0535] Have one-on-one meetings with the whole team every week

[0536] Set specific goals for your team and regularly review progress

[0537] Host workshops to foster open communication

[0538] Notification of results

[0539] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[0540] Specific examples

[0541] As a concrete example, let's say a user has an interest in "deepening their knowledge about leadership in the workplace" and a challenge in "raising team morale and improving performance." The user enters this information into the device.

[0542] The device sends this information to a server, which then searches for relevant media content based on the keywords "leadership" and "team motivation." For example, it might select "video lectures by renowned leadership experts" from a database.

[0543] The server sends the transcript of this talk to a natural language processing engine, extracts the main points, and generates a summary, along with an action plan like this:

[0544] Conduct regular one-on-one feedback sessions with team members

[0545] Plan and implement team building activities

[0546] Implement a system to visualize and quantitatively evaluate team performance

[0547] Prompt Sentence Examples

[0548] "Based on the leadership presentation video, what is your specific action plan for improving team motivation?"

[0549] "Find a video of a talk that will help motivate your team and provide a summary and actionable plan of action."

[0550] This system allows users to efficiently gather knowledge and create specific action plans, thereby providing effective support for achieving individual goals.

[0551] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0552] Step 1:

[0553] User Login

[0554] A user logs into the system using a terminal. The user accesses the login screen and enters their username and password. The terminal sends this information to the server, and if authentication is successful, the home screen is displayed.

[0555] Input: The username and password entered by the user

[0556] Output: Show user's home screen upon successful authentication

[0557] Step 2:

[0558] Input of user interests and issues

[0559] The user enters their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"). The user enters these into the form on the home screen and presses the submit button. The input information is sent from the device to the server.

[0560] Input: User-entered interests and challenges

[0561] Output: User interests and challenges sent to the server

[0562] Step 3:

[0563] Sending user information

[0564] The device sends the entered information to the server. The device summarizes the user's interests and challenges and sends them to the server in the form of an HTTP request. The server receives this request and begins processing.

[0565] Input: User interests and challenges sent from the device

[0566] Output: User information in a format that can be parsed by the server

[0567] Step 4:

[0568] Server information analysis

[0569] The server analyzes the received user information and extracts relevant keywords. The server uses an analysis module to identify themes related to interests and issues. Based on the extracted keywords, the next step is to search for related media content.

[0570] Input: User information (interests and issues)

[0571] Output: Extracted related keywords

[0572] Step 5:

[0573] Selection of relevant media content

[0574] The server searches the database and selects relevant media content based on the extracted keywords. The server sends a keyword search query to the database and retrieves a list of the most relevant media content.

[0575] Input: Extracted keywords

[0576] Output: A list of related media content

[0577] Step 6:

[0578] Sending a transcript

[0579] The server sends the most relevant media content transcripts to the natural language processing engine, which then parses the transcripts, which are a transcription of the audio from the video.

[0580] Input: Transcript of associated media content

[0581] Output: Transcript sent to the natural language processing engine

[0582] Step 7:

[0583] Generate summaries and action plans

[0584] A natural language processing engine analyzes the transcript and generates a summary and a concrete plan of action. The engine extracts key points from the transcript, creates a concise summary, and suggests a concrete plan of action based on these points.

[0585] Input: Transcript

[0586] Output: Summary and action plan

[0587] Step 8:

[0588] Notification of results

[0589] The server sends the generated summary and action plan to the user's device, where the user can review the information and plan a specific course of action. The server sends this information in the form of an HTTP response.

[0590] Input: Summary and Action Plan

[0591] Output: Summary and action plan sent to the user's device

[0592] (Application example 1)

[0593] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0594] Conventional lecture video delivery systems are limited to providing video summaries and specific action plans, and have the problem of not directly contributing to improving users' actual work or performance. Furthermore, the observation of users' work and suggestions for improvement themselves are often done manually, which is inefficient. The present invention aims to improve work efficiency and effectiveness by observing users' work and providing specific action plans in real time based on the observations.

[0595] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0596] In this invention, the server includes means for selecting relevant lecture videos based on the interests and challenges input by the user, means for summarizing the content of the selected lecture videos, means for generating a specific action plan based on the summarized content, means for observing the user's work with a robot and analyzing the observation data, and means for providing the generated action plan to the user via the robot's display and voice notification. This makes it possible to monitor the user's work status in real time and instantly provide optimal improvement suggestions.

[0597] "Means for selecting relevant lecture videos based on the interests and issues entered by the user" refers to a function that analyzes the interest and issue information entered by the user using an input device and automatically selects the video content that is most relevant to that information from a database.

[0598] "Means for summarizing the content of selected lecture videos" refers to a function that analyzes the content of selected videos using natural language processing technology and provides important points in a shortened format.

[0599] "Means for generating specific action plans based on the summarized content" refers to the function of generating specific action plans that can actually be implemented based on the main points extracted from the content of the summarized lecture video.

[0600] "Means for the robot to observe the user's work and analyze the observed data" refers to the function of the robot to monitor the user's work status using cameras and sensors, and to collect and analyze the data in real time.

[0601] "Means for providing the generated action plan to the user through the robot's display and voice notification" refers to the function of displaying the generated specific action plan on the robot's display and communicating it to the user through voice notification.

[0602] "Means for receiving input of interests and challenges from users" refers to a function that provides an interface for users to input their interests and challenges they are facing into the system.

[0603] "Natural language processing technology" is a technology that enables machines to understand and process human language, and refers to technology that analyzes text, generates summaries, extracts meaning, etc.

[0604] This invention is a system that selects relevant lecture videos based on a user's interests and challenges, and provides summaries and specific action plans. Furthermore, the system aims to support factory robot operations by observing the user's work and making suggestions for work improvement in real time. A specific embodiment of this system is described in detail below.

[0605] System Overview

[0606] The system includes the following major components:

[0607] 1. User Interface

[0608] 2. Server

[0609] 3. Natural Language Processing Engine

[0610] 4. Database

[0611] 5. Factory robots

[0612] Program processing

[0613] 1. User Interface

[0614] Users log in using their devices and enter their interests (e.g., "quality control") and challenges (e.g., "reducing lead time"). This information is collected by the device and sent to the server, which then obtains basic data to understand the user's needs and select appropriate lecture videos.

[0615] 2. Server-side processing

[0616] The server analyzes the received information and selects relevant lecture videos. The database stores a large number of lecture videos, each of which is assigned metadata such as related themes and keywords. Based on this metadata, the server searches for and selects the video that best suits the user's request. This process uses advanced data analysis algorithms.

[0617] 3. Natural Language Processing Engine

[0618] The server sends the transcripts of the selected lecture videos to a natural language processing engine. This engine uses the transformers library to extract key points from the transcripts and generate summaries. At the same time, it generates specific action plans based on these key points. For example, if the user's interest is "quality control" and the challenge is "reducing lead time," the engine will generate the following action plan:

[0619] Increase the frequency of quality inspections

[0620] Review of standard operating procedures

[0621] Introducing a system to visualize work efficiency

[0622] 4. Observing factory robots at work

[0623] Factory robots use cameras and sensors to observe the user's work in real time. This observation data is sent to a server as needed and analyzed. The work data observed by the robot is associated with themes and issues entered by the user and used to generate more accurate action plans.

[0624] 5. Notification of Results

[0625] The generated summary and action plan are sent from the server to the user's device and the factory robot, which then displays the information on the display and notifies the user via voice notification. The results can also be viewed via a smartphone app.

[0626] Specific examples

[0627] As a concrete example, let's say a user has the interests and challenges of "I want to deepen my knowledge about quality control" and "I want to shorten lead time." The user enters this information into their device, which then sends it to the server. The server searches for lecture videos on "quality control" and selects appropriate videos. The transcripts of the selected videos are summarized using a natural language processing engine, and specific action plans such as "increase the frequency of quality inspections" and "review standard operating procedures" are generated.

[0628] Prompt Sentence Examples

[0629] The user's interests and challenges are "quality control" and "shortening lead time." Please select a lecture video that meets these criteria and generate a summary of it. Also, please provide a specific action plan for improvement based on these criteria.

[0630] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0631] Step 1:

[0632] A user logs in using a terminal and inputs their interests and challenges. The inputted interest is "quality control" and the challenge is "shortening lead time." This information is collected by the terminal and sent to the server. The input here is the user's text data, and the output is a data packet that sends this to the server.

[0633] Step 2:

[0634] The server analyzes the received data on the user's interests and challenges using a keyword extraction algorithm. The server matches the metadata of the lecture videos in the database with the keywords contained in the user's interests and challenges to select relevant lecture videos. The input is keywords and metadata, and the output is a list of the most suitable lecture videos.

[0635] Step 3:

[0636] We take the transcripts of selected talk videos and send them to a natural language processing engine, which uses the transformers library to extract key points from the transcripts and generate concise summaries. The input is the talk video transcript, and the output is the summary text.

[0637] Step 4:

[0638] The same natural language processing engine is used to generate specific action plans from the summary. The engine further analyzes the summary text and generates actionable plans, such as "increase the frequency of quality inspections" or "review standard operating procedures." The input is the summary text, and the output is a list of action plans.

[0639] Step 5:

[0640] Factory robots use cameras and sensors to observe the user's work. This observation data is sent to a server in real time. The server analyzes the observation data and evaluates the user's work status. The input is the observation data, and the output is the analysis result.

[0641] Step 6:

[0642] The server adjusts and optimizes the generated action plan based on the analysis results. The observation data and the action plan are collated to provide specific suggestions tailored to the user's current situation. The input is the observation data and the existing action plan, and the output is an optimized action plan.

[0643] Step 7:

[0644] The generated summary and action plan are sent from the server to the user's device and the factory robot. The factory robot displays the information on a display and communicates it to the user through a voice notification. The same information can also be viewed on the user's device. The input is the optimized action plan, and the output is the display and voice notification.

[0645] An example prompt is:

[0646] "The user's interests and challenges are 'quality control' and 'shortening lead time.' Please select a lecture video that meets these criteria and generate a summary of it. Also, please provide a specific action plan for improvement based on these criteria."

[0647] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0648] The present invention combines a system that selects relevant lecture videos based on a user's interests and challenges, summarizes the content, and provides a specific action plan with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[0649] System Overview

[0650] The system includes the following major components:

[0651] 1. User Interface

[0652] 2. Server

[0653] 3. Natural Language Processing Engine

[0654] 4. Emotion Engine

[0655] 5. Database

[0656] Program processing

[0657] 1. User Interface

[0658] Users log in using their devices and input their interests and the challenges they are currently facing. This information is collected by the device and sent to the server. The emotion engine then recognizes the user's emotions based on the information they input and their interactions.

[0659] 2. Server-side processing

[0660] The server analyzes the received information and selects relevant lecture videos, taking into account the user's recognized emotions. The database stores a large number of lecture videos, each of which has associated themes and keywords as metadata. The server combines this metadata with emotional information to search for and select the most appropriate video.

[0661] 3. Emotion Engine

[0662] The emotion engine analyzes user input and interactions (e.g., keyboard input speed, facial expression analysis, etc.) to recognize the user's emotional state. This information is sent to the server and used to select videos and generate action plans.

[0663] 4. Natural Language Processing Engine

[0664] The server sends the transcripts of the selected lecture videos to a natural language processing engine, which extracts key points from the transcripts and generates a concise summary. At the same time, a specific action plan is generated based on these key points. Taking into account the results of the emotion engine, an appropriate action plan is created according to the user's emotional state.

[0665] 5. Notification of Results

[0666] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[0667] Specific examples

[0668] For example, if a user has an interest in "I want to deepen my knowledge about leadership in the workplace" and a goal in "I want to improve team morale and performance," if the emotion engine recognizes the user's emotional state as "high stress," it will also take this information into account.

[0669] The user enters this information into the device, which then sends it to the server. The server searches for relevant lecture videos based on the keywords "leadership" and "team motivation," while also taking into account the user's current emotional state of "high stress." For example, a "lecture on the psychological aspects of leadership" might be selected.

[0670] The server sends the transcript of this talk to a natural language processing engine, which summarizes the main points and generates an action plan like this:

[0671] Use relaxation techniques regularly

[0672] Facilitate open stress management dialogue within your team

[0673] Introduce flexible scheduling

[0674] The server sends this information to the user's device, where the user can review it and take action in real life.

[0675] In this way, a system is provided that allows the user to efficiently gather knowledge and create a specific action plan according to their emotional state.

[0676] The processing flow will be explained below.

[0677] Step 1:

[0678] A user logs in using a terminal. After logging in, the user enters their interests (e.g., "Leadership") and the challenges they are currently facing (e.g., "I want to improve my team's motivation").

[0679] Step 2:

[0680] The emotion engine analyzes the user's input information and interactions (e.g., keyboard input speed, facial expression analysis, etc.) and recognizes the user's emotional state (e.g., "high stress").

[0681] Step 3:

[0682] The terminal collects the user's input information and emotional state and prepares to send it to the server.

[0683] Step 4:

[0684] The terminal transmits the user's input information and emotional state to the server, and the transmitted information includes the user's interests, concerns, and emotional state.

[0685] Step 5:

[0686] The server analyzes the received user information, extracting keywords that are most relevant to the user's interests and issues, and also taking into account the user's emotional state.

[0687] Step 6:

[0688] The server searches the database for relevant lecture videos. The lecture video database contains metadata such as themes and keywords related to each video. The server combines this metadata with emotional information to search for and select the most appropriate video.

[0689] Step 7:

[0690] The server selects the most relevant lecture videos based on the user's input interests and challenges, as well as their perceived emotional state.

[0691] Step 8:

[0692] The server retrieves the transcripts of the selected lecture videos. The transcripts are a written version of the video content.

[0693] Step 9:

[0694] The server sends the transcript to a natural language processing engine, which processes the transcript to extract key points.

[0695] Step 10:

[0696] A natural language processing engine analyzes the transcript and summarizes key points and key points, which are concise summaries of the essence of the lecture video.

[0697] Step 11:

[0698] The server generates a specific action plan based on the summary, taking into account the user's emotional state. For example, if the user is feeling stressed, the action plan may include stress management and relaxation.

[0699] Step 12:

[0700] The server sends the generated summary and action plan to the user's terminal.

[0701] Step 13:

[0702] The terminal stores the received summary and action plan and notifies the user that the user has received new information.

[0703] Step 14:

[0704] The user checks the notification and views the summary and action plan displayed on the device, allowing the user to create a specific action plan.

[0705] In this way, a system is provided that allows users to efficiently absorb knowledge from lecture videos and create specific action plans according to their emotional state.

[0706] Example 2

[0707] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0708] Previous systems were unable to consider the user's emotional state when providing relevant lecture videos based on the user's interests and challenges. As a result, they were unable to provide optimal information for the user or generate specific and effective action plans. Furthermore, they were unable to appropriately change feedback according to the user's individual emotional state, which limited the user experience.

[0709] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0710] In this invention, the server includes a means for selecting relevant lecture videos based on the interests and issues input by the user, a means for recognizing emotions from the user's input information and interactions, and a means for selecting relevant videos in consideration of the recognized emotional information. This makes it possible to select the most appropriate lecture video taking into consideration the user's emotional state and to provide a specific action plan according to the emotion.

[0711] "User" means any individual or entity that uses the System.

[0712] "Interests" refers to the themes and topics that interest users.

[0713] "Challenge" refers to the problem the user is currently facing or the issue they want to solve.

[0714] A "lecture video" is video content in which a specialist or expert explains a specific topic.

[0715] "Means" refers to methods or techniques for achieving a specific purpose.

[0716] A "summary" is a concise summary of the main points extracted from the content of a lecture video.

[0717] An "action plan" is a set of specific recommended actions that can be taken based on the summarized content.

[0718] "Means for recognizing emotions" refers to technology that analyzes a user's emotional state based on their behavior, expressions, etc. Examples include keyboard input speed and facial expression analysis.

[0719] "Natural language processing technology" is a technology that allows machines to analyze text data, understand its meaning, and process human language.

[0720] "Metadata" is additional information that accompanies the data, and includes, for example, the theme and keywords of the video.

[0721] A "server" is a computer system that receives requests from users and processes them in cooperation with a database.

[0722] A "database" is a system that organizes, manages, and stores a variety of data.

[0723] "Interaction" refers to a series of operations, actions, and inputs that a user makes with a system.

[0724] The present invention combines a system that selects relevant lecture videos based on a user's interests and challenges, summarizes the content, and provides a specific action plan with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[0725] User Interface

[0726] Users log in to the system using a terminal. After logging in, they input their interests (e.g., strengthening leadership) and challenges (e.g., improving team morale). The emotion engine also analyzes the user's emotional state based on facial expressions and keyboard typing speed. This allows the system to take the user's emotions into account.

[0727] Sending information to the server

[0728] The device sends the collected user input information and emotion information to the server, which then performs subsequent processing based on this information.

[0729] Selection of lecture videos

[0730] The server analyzes the received information and searches for relevant lecture videos in its database. The database stores multiple lecture videos, each of which is assigned metadata such as theme, keywords, and emotion tags. The server compares this metadata with user information to select the most suitable video.

[0731] Processing the transcript

[0732] The server sends the transcripts (written transcripts of what was said) of the selected lecture videos to a natural language processing engine, which analyzes the transcripts and extracts key points.

[0733] Generate a summary and action plan

[0734] The natural language processing engine generates a summary based on the extracted key points, and then takes into account the emotional information from the emotion engine to create a specific action plan suited to the user's situation, such as "introducing relaxation techniques" for users experiencing high stress.

[0735] Notifying users of the results

[0736] The server sends the generated summary and action plan to the user's device, where the user can review the information and create a workable action plan for real life.

[0737] Specific examples

[0738] For example, consider a user who has an interest in "I want to deepen my knowledge about leadership in the workplace" and a goal to "improve team morale and performance." If the emotion engine recognizes the user's emotional state as "high stress," this information will also be taken into account.

[0739] The user enters this information into the device, which then sends it to the server. The server then searches for relevant lecture videos based on keywords like "leadership" and "team motivation," taking into account the user's emotional state of "high stress." For example, a video titled "Lectures on the psychological aspects of leadership" might be selected.

[0740] The server sends the transcript of this talk to a natural language processing engine, which summarizes the main points and generates an action plan like this:

[0741] Use relaxation techniques regularly

[0742] Facilitate open stress management dialogue within your team

[0743] Introduce flexible scheduling

[0744] The server sends this information to the user's device, where the user can review it and take action in real life.

[0745] Prompt Sentence Examples

[0746] "I would like to deepen my knowledge of leadership in the workplace. I would also like to motivate my team and improve their performance. Based on this information, please suggest appropriate talk videos, provide summaries, and concrete action plans."

[0747] In this way, this system not only provides information based on the user's interests and challenges, but also provides optimal content and specific action plans that take into account the user's emotional state.

[0748] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0749] Step 1:

[0750] A user logs into the system using a terminal. After logging in, the user inputs their interests (e.g., strengthening leadership) and challenges (e.g., improving team morale). The terminal collects the input information and prepares it. The input at this time is text data related to the user's interests and challenges.

[0751] Step 2:

[0752] The device sends the collected input information to the server. Specifically, it sends data using an HTTP POST request. This request includes text data of the user's input interests and tasks. The output is the input information data received by the server.

[0753] Step 3:

[0754] The server analyzes the received input information and uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's input information and interactions (e.g., keyboard input speed, facial expression analysis, etc.) to extract the user's emotional state. For example, emotional data such as "high stress" or "low motivation" can be obtained.

[0755] Step 4:

[0756] The server searches for relevant lecture videos in the database, taking into account the emotional information from the emotion engine and user input information. The database stores metadata of lecture videos (e.g., themes, keywords, emotional tags), and matches this with user information to select the most suitable video. The input is user information and emotional information, and the output is the selected lecture video.

[0757] Step 5:

[0758] The server sends the transcripts of the selected lecture videos to a natural language processing engine, which analyzes the transcripts and extracts key points. The input is the lecture video transcript, and the output is a summarized version of the key points.

[0759] Step 6:

[0760] The natural language processing engine generates a summary based on the extracted key points, and then generates a specific action plan taking into account the emotional information obtained from the emotion engine. For example, if the user is "highly stressed," the following action plan will be generated:

[0761] Use relaxation techniques regularly

[0762] Facilitate open stress management dialogue within your team

[0763] Introduce flexible scheduling

[0764] The input is key points and emotional information, and the output is a summary and an action plan.

[0765] Step 7:

[0766] The server sends the generated summary and action plan to the user's device. The user can check this information on the device and create a specific action plan. The input is the summary and action plan, and the output is information provided to the user. A specific example of operation would be when the user clicks the "View Results" button on the device to check the provided summary and action plan.

[0767] (Application example 2)

[0768] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0769] While users need to properly learn about home security knowledge and techniques and take concrete measures, it is difficult to select appropriate content from the vast amount of information available. Furthermore, there are few systems that provide appropriate content and measures based on the user's emotional state and stress level. Therefore, there is a need for a system that provides information optimized for each user's individual situation and provides a concrete action plan.

[0770] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0771] In this invention, the server includes means for selecting relevant video content based on interests and issues input by the user, means for summarizing the content of the selected video content, means for generating a specific action plan based on the summarized content, means for recognizing the emotional state of the user using emotion recognition technology, and means for selecting appropriate video content based on the emotional state of the user, thereby enabling the user to efficiently obtain appropriate security content and an action plan.

[0772] "User" refers to any individual or organization that uses this system.

[0773] "Interests" refers to the information users want to know and the topics they want to learn about.

[0774] A "challenge" refers to a problem that a user is currently facing or wants to solve.

[0775] "Video content" refers to information in video format stored on the Internet or in a database.

[0776] A "summary" refers to information that concisely summarizes the main points of the video content.

[0777] An "action plan" is a plan that indicates specific actions that a user should take based on the summarized information.

[0778] "Emotion recognition technology" refers to technology that analyzes a user's facial expressions, behavior, etc. to identify their emotional state.

[0779] "Emotional state" refers to a user's current psychological state or mood.

[0780] "Means for selection" refers to a method or system for selecting relevant video content based on a user's interests and challenges.

[0781] "Means for providing" refers to a method or system for notifying the user of the generated action plan or summary information.

[0782] "Means for summarizing" refers to a method or system that uses natural language processing technology to concisely summarize the content of video content.

[0783] "Means for generating" refers to a method or system that generates an action plan based on the summarized content.

[0784] "Means for recognizing" refers to a method or system that uses emotion recognition technology to analyze and identify a user's emotional state.

[0785] MODE FOR CARRYING OUT THE INVENTION

[0786] System Overview

[0787] This invention combines emotion recognition technology with a system that selects appropriate video content based on the user's interests and challenges, and provides a summary and a concrete action plan. The system includes the following main components:

[0788] User interface: Users input their interests and concerns using their smartphones.

[0789] Server: Analyzes the input information and selects relevant video content.

[0790] Emotion Recognition Engine: Recognizes the user's emotional state.

[0791] Natural language processing engine: Summarizes the content of selected videos and generates specific action plans.

[0792] Database: Stores video content and makes it searchable.

[0793] Hardware and Software Configuration

[0794] Smartphones: Used as a user interface, they use a camera to capture the user's facial expressions.

[0795] Emotion Recognition Engine: Uses Affectiva SDK or equivalent emotion recognition technology.

[0796] Natural language processing engine: Uses OpenAI GPT or a similar generative model.

[0797] Database: Use AWS RDS or Google Firebase to store video content and its metadata.

[0798] Processing flow

[0799] User Interface

[0800] Users input their interests and current challenges through a smartphone application, and the smartphone camera captures the user's facial expressions and sends them to an emotion recognition engine.

[0801] emotion recognition technology

[0802] The emotion recognition engine (Affectiva SDK) analyzes the captured facial images of the user and recognizes their emotional state. This emotional information is sent to the server along with the user's interests and concerns.

[0803] Video content selection

[0804] The server analyzes the received information (interests, issues, emotional state) and searches for relevant video content from a database, using video metadata and tags.

[0805] Generate summaries and action plans

[0806] The server then sends the transcripts of the selected video content to a natural language processing engine (OpenAI GPT), which extracts key points from the transcript and generates a concise summary. At the same time, it generates a specific action plan based on these key points. Taking into account the user's emotional state, an appropriate action plan is created based on the user's psychological situation.

[0807] Notification of results

[0808] The generated summary and action plan are sent from the server to the user's smartphone, where the user can review the information and implement specific action plans.

[0809] Specific examples

[0810] As a concrete example, let's say a user has an interest in "deepening their knowledge about home security" and a goal of "strengthening measures against suspicious individuals." Furthermore, if the emotion recognition technology recognizes the user's emotional state as "high stress," this information is also taken into consideration. The user enters this information into their smartphone, captures their facial expressions with the camera, and sends them to the emotion recognition engine. The server searches for videos related to "home security" and "measures against suspicious individuals," taking their emotional state into consideration.

[0811] For example, if the selected video is a "Lecture on Home Security," the server sends the transcript of this lecture to a natural language processing engine to generate a summary and action plan, which could look something like this:

[0812] Summary: Locking gates and installing security cameras are important ways to improve home security.

[0813] Action plan: Select the type of security camera that will be effective in preventing suspicious individuals and determine where to install it.

[0814] Prompt Sentence Examples

[0815] User Input: I want to solve home security issues

[0816] Emotion: High stress

[0817] Prompt for the generative AI model:

[0818] Create a summary of your home security measures and provide a specific action plan for high-stress situations.

[0819] This allows users to efficiently obtain appropriate security content and action plans.

[0820] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0821] Step 1:

[0822] Users use their smartphones to input their interests and current challenges. This information is sent from the device to the server. Input can be done using text boxes or selection lists.

[0823] Input: Interests (e.g., "Home security"), Issues (e.g., "Countermeasures against suspicious persons")

[0824] Output: User interest and issue data sent to the server

[0825] Step 2:

[0826] The user captures their facial expression using the smartphone camera. The emotion recognition engine analyzes the facial image and recognizes the user's emotional state. The recognition results are sent to the server.

[0827] Input: facial expression image

[0828] Output: User's emotional state (e.g. "high stress")

[0829] Step 3:

[0830] The server analyzes the received interest, task, and emotional state data and searches a database for relevant video content, using video metadata and tags.

[0831] Input: Interests, tasks, and emotional state data

[0832] Output: Selected video content (e.g., "Lecture video on home security measures")

[0833] Step 4:

[0834] The server sends the transcript of the selected video content to a natural language processing engine, which generates a summary and a concrete action plan, automatically generated using a generative AI model.

[0835] Input: Transcript of video content

[0836] Output: Summary and action plan (e.g., "Lock the gates and install security cameras")

[0837] Step 5:

[0838] The server sends the generated summary and action plan to the user's device, where the user can review it and take specific action.

[0839] Input: Summary and Action Plan

[0840] Output: Summary and action plan displayed on user terminal

[0841] This allows users to select appropriate video content based on their interests and challenges, summarize the content in an easy-to-understand format, and put a concrete action plan into action.

[0842] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0843] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0844] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0845] [Third embodiment]

[0846] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0847] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0848] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0849] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0850] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0851] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0852] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0853] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0854] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0855] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0856] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0857] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0858] The present invention is a system that selects lecture videos based on a user's interests and challenges, and provides summaries and specific action plans. Specific embodiments of this system are described in detail below.

[0859] System Overview

[0860] The system includes the following major components:

[0861] 1. User Interface

[0862] 2. Server

[0863] 3. Natural Language Processing Engine

[0864] 4. Database

[0865] Program processing

[0866] 1. User Interface

[0867] Users log in using a device and enter their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"). This information is collected by the device and sent to the server.

[0868] 2. Server-side processing

[0869] The server analyzes the received information and selects relevant lecture videos. The database stores a large number of lecture videos, each of which has associated themes and keywords as metadata. Based on this metadata, the server searches for and selects the video that best suits the user's request.

[0870] 3. Natural Language Processing Engine

[0871] The server sends the transcripts of the selected lecture videos to a natural language processing engine. The engine extracts key points from the transcripts and generates a concise summary. At the same time, it generates a specific action plan based on these key points. For example, if the user's interest is "leadership" and the challenge is "improving team motivation," the engine will generate the following action plan:

[0872] Have one-on-one meetings with the whole team every week

[0873] Set specific goals for your team and regularly review progress

[0874] Host workshops to foster open communication

[0875] 4. Notification of Results

[0876] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[0877] Specific examples

[0878] As a concrete example, let's say a user has an interest in "deepening their knowledge about leadership in the workplace" and a challenge in "raising team morale and improving performance." The user enters this information into the device.

[0879] The device sends this information to the server, which then searches for relevant lecture videos based on the keywords "leadership" and "team motivation." For example, "lectures by renowned leadership experts" might be selected.

[0880] The server sends the transcript of this talk to a natural language processing engine, extracts the main points, and creates a summary, along with an action plan like this:

[0881] Conduct regular one-on-one feedback sessions with team members

[0882] Plan and implement team building activities

[0883] Implement a system to visualize and quantitatively evaluate team performance

[0884] The server sends this information to the user's device, where the user can review it and take action in real life.

[0885] In this way, a system is provided that allows users to efficiently gather knowledge and create specific action plans.

[0886] The processing flow will be explained below.

[0887] Step 1:

[0888] A user logs in using a terminal. After logging in, the user enters their interests (e.g., "Leadership") and the challenges they are currently facing (e.g., "I want to improve my team's motivation").

[0889] Step 2:

[0890] The terminal collects the user's input information and prepares to send it to the server.

[0891] Step 3:

[0892] The device sends the user's input information to the server. The sent information includes the user's interests and concerns.

[0893] Step 4:

[0894] The server analyzes the received user information, and extracts keywords that are most relevant to the user's interests and issues.

[0895] Step 5:

[0896] The server searches a database for relevant lecture videos, which contain metadata such as themes and keywords related to each video.

[0897] Step 6:

[0898] The server selects the most relevant lecture videos based on the user's input interests and challenges.

[0899] Step 7:

[0900] The server retrieves the transcripts of the selected lecture videos. The transcripts are a written version of the video content.

[0901] Step 8:

[0902] The server sends the transcript to a natural language processing engine, which processes the transcript to extract key points.

[0903] Step 9:

[0904] A natural language processing engine analyzes the transcript and summarizes key points and key points, which are concise summaries of the essence of the lecture video.

[0905] Step 10:

[0906] The server generates a concrete action plan based on the summarized content, which is a concrete action plan that the user can carry out in real life.

[0907] Step 11:

[0908] The server sends the generated summary and action plan to the user's terminal.

[0909] Step 12:

[0910] The terminal stores the received summary and action plan and notifies the user that the user has received new information.

[0911] Step 13:

[0912] The user checks the notification and views the summary and action plan displayed on the device, allowing the user to create a specific action plan.

[0913] In this way, users can efficiently absorb the knowledge from the lecture videos and put it into concrete action.

[0914] Example 1

[0915] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0916] With conventional systems, it was difficult for users to efficiently collect information that matched their interests and challenges and to create specific action plans. In particular, there was a problem in that it took a great deal of time and effort to select highly relevant information from a large amount of media content, summarize that content, and create an action plan.

[0917] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0918] In this invention, the server includes means for selecting relevant media content based on the interests and issues input by the user, means for summarizing the selected media content, and means for generating a specific action plan based on the summarized content, thereby enabling the user to quickly and efficiently obtain information that is in line with their interests and issues and to create a specific action plan based on that information.

[0919] A "user" is an individual who operates the system and inputs their interests and tasks.

[0920] "Interests" are topics or themes that users particularly want to know about, learn about, or explore further.

[0921] A "challenge" is a problem the user wants to solve or an obstacle they want to overcome.

[0922] "Media content" refers to information or materials that are primarily provided in the form of video, audio, text, etc.

[0923] "Selection tools" are functions that identify and select the most appropriate media content based on user-entered interests and challenges.

[0924] The "summarizing means" is a function that briefly summarizes the content of the selected media content and extracts the main points.

[0925] "Means for generating a concrete action plan" is a function that suggests specific steps or actions that the user should take based on the summarized points.

[0926] The "means of providing" is a function that presents the generated action plan to the user in an easy-to-understand manner, allowing them to use it as a reference for action.

[0927] A "database" is a storage device that stores media content and its associated themes and keywords and allows them to be searched as needed.

[0928] "Natural language processing technology" is a technology for analyzing language data (e.g., transcripts) and understanding their meaning.

[0929] A "transcript" is text data that converts the content of video, audio, etc. into text.

[0930] "Means of analysis" refers to the function of analyzing data such as user input and transcripts and extracting necessary information.

[0931] The present invention is a system that selects optimal media content based on a user's interests and challenges, summarizes the content, and provides a specific action plan. The system mainly consists of the following components:

[0932] 1. User Interface

[0933] 2. Server

[0934] 3. Natural Language Processing Engine

[0935] 4. Database

[0936] User Interface

[0937] Users log in to the system using a terminal and enter their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"), which are collected by the terminal and sent to the server.

[0938] server

[0939] The server analyzes the information received from the user and selects relevant media content. The server is connected to a database that stores a large number of media contents, and each content is assigned related themes and keywords as metadata. The server uses this metadata to search for and select the content that best suits the user's request.

[0940] Natural Language Processing Engine

[0941] The server sends the transcript of the selected media content to a natural language processing engine, which extracts key points from the transcript and generates a concise summary, along with a concrete plan of action based on those key points.

[0942] For example, if the user's interest is "leadership" and the challenge is "improving team motivation," the engine will generate an action plan like this:

[0943] Have one-on-one meetings with the whole team every week

[0944] Set specific goals for your team and regularly review progress

[0945] Host workshops to foster open communication

[0946] Notification of results

[0947] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[0948] Specific examples

[0949] As a concrete example, let's say a user has an interest in "deepening their knowledge about leadership in the workplace" and a challenge in "raising team morale and improving performance." The user enters this information into the device.

[0950] The device sends this information to a server, which then searches for relevant media content based on the keywords "leadership" and "team motivation." For example, it might select "video lectures by renowned leadership experts" from a database.

[0951] The server sends the transcript of this talk to a natural language processing engine, extracts the main points, and generates a summary, along with an action plan like this:

[0952] Conduct regular one-on-one feedback sessions with team members

[0953] Plan and implement team building activities

[0954] Implement a system to visualize and quantitatively evaluate team performance

[0955] Prompt Sentence Examples

[0956] "Based on the leadership presentation video, what is your specific action plan for improving team motivation?"

[0957] "Find a video of a talk that will help motivate your team and provide a summary and actionable plan of action."

[0958] This system allows users to efficiently gather knowledge and create specific action plans, thereby providing effective support for achieving individual goals.

[0959] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0960] Step 1:

[0961] User Login

[0962] A user logs into the system using a terminal. The user accesses the login screen and enters their username and password. The terminal sends this information to the server, and if authentication is successful, the home screen is displayed.

[0963] Input: The username and password entered by the user

[0964] Output: Show user's home screen upon successful authentication

[0965] Step 2:

[0966] Input of user interests and issues

[0967] The user enters their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"). The user enters these into the form on the home screen and presses the submit button. The input information is sent from the device to the server.

[0968] Input: User-entered interests and challenges

[0969] Output: User interests and challenges sent to the server

[0970] Step 3:

[0971] Sending user information

[0972] The device sends the entered information to the server. The device summarizes the user's interests and challenges and sends them to the server in the form of an HTTP request. The server receives this request and begins processing.

[0973] Input: User interests and challenges sent from the device

[0974] Output: User information in a format that can be parsed by the server

[0975] Step 4:

[0976] Server information analysis

[0977] The server analyzes the received user information and extracts relevant keywords. The server uses an analysis module to identify themes related to interests and issues. Based on the extracted keywords, the next step is to search for related media content.

[0978] Input: User information (interests and issues)

[0979] Output: Extracted related keywords

[0980] Step 5:

[0981] Selection of relevant media content

[0982] The server searches the database and selects relevant media content based on the extracted keywords. The server sends a keyword search query to the database and retrieves a list of the most relevant media content.

[0983] Input: Extracted keywords

[0984] Output: A list of related media content

[0985] Step 6:

[0986] Sending a transcript

[0987] The server sends the most relevant media content transcripts to the natural language processing engine, which then parses the transcripts, which are a transcription of the audio from the video.

[0988] Input: Transcript of associated media content

[0989] Output: Transcript sent to the natural language processing engine

[0990] Step 7:

[0991] Generate summaries and action plans

[0992] A natural language processing engine analyzes the transcript and generates a summary and a concrete plan of action. The engine extracts key points from the transcript, creates a concise summary, and suggests a concrete plan of action based on these points.

[0993] Input: Transcript

[0994] Output: Summary and action plan

[0995] Step 8:

[0996] Notification of results

[0997] The server sends the generated summary and action plan to the user's device, where the user can review the information and plan a specific course of action. The server sends this information in the form of an HTTP response.

[0998] Input: Summary and Action Plan

[0999] Output: Summary and action plan sent to the user's device

[1000] (Application example 1)

[1001] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1002] Conventional lecture video delivery systems are limited to providing video summaries and specific action plans, and have the problem of not directly contributing to improving users' actual work or performance. Furthermore, the observation of users' work and suggestions for improvement themselves are often done manually, which is inefficient. The present invention aims to improve work efficiency and effectiveness by observing users' work and providing specific action plans in real time based on the observations.

[1003] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1004] In this invention, the server includes means for selecting relevant lecture videos based on the interests and challenges input by the user, means for summarizing the content of the selected lecture videos, means for generating a specific action plan based on the summarized content, means for observing the user's work with a robot and analyzing the observation data, and means for providing the generated action plan to the user via the robot's display and voice notification. This makes it possible to monitor the user's work status in real time and instantly provide optimal improvement suggestions.

[1005] "Means for selecting relevant lecture videos based on the interests and issues entered by the user" refers to a function that analyzes the interest and issue information entered by the user using an input device and automatically selects the video content that is most relevant to that information from a database.

[1006] "Means for summarizing the content of selected lecture videos" refers to a function that analyzes the content of selected videos using natural language processing technology and provides important points in a shortened format.

[1007] "Means for generating specific action plans based on the summarized content" refers to the function of generating specific action plans that can actually be implemented based on the main points extracted from the content of the summarized lecture video.

[1008] "Means for the robot to observe the user's work and analyze the observed data" refers to the function of the robot to monitor the user's work status using cameras and sensors, and to collect and analyze the data in real time.

[1009] "Means for providing the generated action plan to the user through the robot's display and voice notification" refers to the function of displaying the generated specific action plan on the robot's display and communicating it to the user through voice notification.

[1010] "Means for receiving input of interests and challenges from users" refers to a function that provides an interface for users to input their interests and challenges they are facing into the system.

[1011] "Natural language processing technology" is a technology that enables machines to understand and process human language, and refers to technology that analyzes text, generates summaries, extracts meaning, etc.

[1012] This invention is a system that selects relevant lecture videos based on a user's interests and challenges, and provides summaries and specific action plans. Furthermore, the system aims to support factory robot operations by observing the user's work and making suggestions for work improvement in real time. A specific embodiment of this system is described in detail below.

[1013] System Overview

[1014] The system includes the following major components:

[1015] 1. User Interface

[1016] 2. Server

[1017] 3. Natural Language Processing Engine

[1018] 4. Database

[1019] 5. Factory robots

[1020] Program processing

[1021] 1. User Interface

[1022] Users log in using their devices and enter their interests (e.g., "quality control") and challenges (e.g., "reducing lead time"). This information is collected by the device and sent to the server, which then obtains basic data to understand the user's needs and select appropriate lecture videos.

[1023] 2. Server-side processing

[1024] The server analyzes the received information and selects relevant lecture videos. The database stores a large number of lecture videos, each of which is assigned metadata such as related themes and keywords. Based on this metadata, the server searches for and selects the video that best suits the user's request. This process uses advanced data analysis algorithms.

[1025] 3. Natural Language Processing Engine

[1026] The server sends the transcripts of the selected lecture videos to a natural language processing engine. This engine uses the transformers library to extract key points from the transcripts and generate summaries. At the same time, it generates specific action plans based on these key points. For example, if the user's interest is "quality control" and the challenge is "reducing lead time," the engine will generate the following action plan:

[1027] Increase the frequency of quality inspections

[1028] Review of standard operating procedures

[1029] Introducing a system to visualize work efficiency

[1030] 4. Observing factory robots at work

[1031] Factory robots use cameras and sensors to observe the user's work in real time. This observation data is sent to a server as needed and analyzed. The work data observed by the robot is associated with themes and issues entered by the user and used to generate more accurate action plans.

[1032] 5. Notification of Results

[1033] The generated summary and action plan are sent from the server to the user's device and the factory robot, which then displays the information on the display and notifies the user via voice notification. The results can also be viewed via a smartphone app.

[1034] Specific examples

[1035] As a concrete example, let's say a user has the interests and challenges of "I want to deepen my knowledge about quality control" and "I want to shorten lead time." The user enters this information into their device, which then sends it to the server. The server searches for lecture videos on "quality control" and selects appropriate videos. The transcripts of the selected videos are summarized using a natural language processing engine, and specific action plans such as "increase the frequency of quality inspections" and "review standard operating procedures" are generated.

[1036] Prompt Sentence Examples

[1037] The user's interests and challenges are "quality control" and "shortening lead time." Please select a lecture video that meets these criteria and generate a summary of it. Also, please provide a specific action plan for improvement based on these criteria.

[1038] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1039] Step 1:

[1040] A user logs in using a terminal and inputs their interests and challenges. The inputted interest is "quality control" and the challenge is "shortening lead time." This information is collected by the terminal and sent to the server. The input here is the user's text data, and the output is a data packet that sends this to the server.

[1041] Step 2:

[1042] The server analyzes the received data on the user's interests and challenges using a keyword extraction algorithm. The server matches the metadata of the lecture videos in the database with the keywords contained in the user's interests and challenges to select relevant lecture videos. The input is keywords and metadata, and the output is a list of the most suitable lecture videos.

[1043] Step 3:

[1044] We take the transcripts of selected talk videos and send them to a natural language processing engine, which uses the transformers library to extract key points from the transcripts and generate concise summaries. The input is the talk video transcript, and the output is the summary text.

[1045] Step 4:

[1046] The same natural language processing engine is used to generate specific action plans from the summary. The engine further analyzes the summary text and generates actionable plans, such as "increase the frequency of quality inspections" or "review standard operating procedures." The input is the summary text, and the output is a list of action plans.

[1047] Step 5:

[1048] Factory robots use cameras and sensors to observe the user's work. This observation data is sent to a server in real time. The server analyzes the observation data and evaluates the user's work status. The input is the observation data, and the output is the analysis result.

[1049] Step 6:

[1050] The server adjusts and optimizes the generated action plan based on the analysis results. The observation data and the action plan are collated to provide specific suggestions tailored to the user's current situation. The input is the observation data and the existing action plan, and the output is an optimized action plan.

[1051] Step 7:

[1052] The generated summary and action plan are sent from the server to the user's device and the factory robot. The factory robot displays the information on a display and communicates it to the user through a voice notification. The same information can also be viewed on the user's device. The input is the optimized action plan, and the output is the display and voice notification.

[1053] An example prompt is:

[1054] "The user's interests and challenges are 'quality control' and 'shortening lead time.' Please select a lecture video that meets these criteria and generate a summary of it. Also, please provide a specific action plan for improvement based on these criteria."

[1055] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1056] The present invention combines a system that selects relevant lecture videos based on a user's interests and challenges, summarizes the content, and provides a specific action plan with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[1057] System Overview

[1058] The system includes the following major components:

[1059] 1. User Interface

[1060] 2. Server

[1061] 3. Natural Language Processing Engine

[1062] 4. Emotion Engine

[1063] 5. Database

[1064] Program processing

[1065] 1. User Interface

[1066] Users log in using their devices and input their interests and the challenges they are currently facing. This information is collected by the device and sent to the server. The emotion engine then recognizes the user's emotions based on the information they input and their interactions.

[1067] 2. Server-side processing

[1068] The server analyzes the received information and selects relevant lecture videos, taking into account the user's recognized emotions. The database stores a large number of lecture videos, each of which has associated themes and keywords as metadata. The server combines this metadata with emotional information to search for and select the most appropriate video.

[1069] 3. Emotion Engine

[1070] The emotion engine analyzes user input and interactions (e.g., keyboard input speed, facial expression analysis, etc.) to recognize the user's emotional state. This information is sent to the server and used to select videos and generate action plans.

[1071] 4. Natural Language Processing Engine

[1072] The server sends the transcripts of the selected lecture videos to a natural language processing engine, which extracts key points from the transcripts and generates a concise summary. At the same time, a specific action plan is generated based on these key points. Taking into account the results of the emotion engine, an appropriate action plan is created according to the user's emotional state.

[1073] 5. Notification of Results

[1074] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[1075] Specific examples

[1076] For example, if a user has an interest in "I want to deepen my knowledge about leadership in the workplace" and a goal in "I want to improve team morale and performance," if the emotion engine recognizes the user's emotional state as "high stress," it will also take this information into account.

[1077] The user enters this information into the device, which then sends it to the server. The server searches for relevant lecture videos based on the keywords "leadership" and "team motivation," while also taking into account the user's current emotional state of "high stress." For example, a "lecture on the psychological aspects of leadership" might be selected.

[1078] The server sends the transcript of this talk to a natural language processing engine, which summarizes the main points and generates an action plan like this:

[1079] Use relaxation techniques regularly

[1080] Facilitate open stress management dialogue within your team

[1081] Introduce flexible scheduling

[1082] The server sends this information to the user's device, where the user can review it and take action in real life.

[1083] In this way, a system is provided that allows the user to efficiently gather knowledge and create a specific action plan according to their emotional state.

[1084] The processing flow will be explained below.

[1085] Step 1:

[1086] A user logs in using a terminal. After logging in, the user enters their interests (e.g., "Leadership") and the challenges they are currently facing (e.g., "I want to improve my team's motivation").

[1087] Step 2:

[1088] The emotion engine analyzes the user's input information and interactions (e.g., keyboard input speed, facial expression analysis, etc.) and recognizes the user's emotional state (e.g., "high stress").

[1089] Step 3:

[1090] The terminal collects the user's input information and emotional state and prepares to send it to the server.

[1091] Step 4:

[1092] The terminal transmits the user's input information and emotional state to the server, and the transmitted information includes the user's interests, concerns, and emotional state.

[1093] Step 5:

[1094] The server analyzes the received user information, extracting keywords that are most relevant to the user's interests and issues, and also taking into account the user's emotional state.

[1095] Step 6:

[1096] The server searches the database for relevant lecture videos. The lecture video database contains metadata such as themes and keywords related to each video. The server combines this metadata with emotional information to search for and select the most appropriate video.

[1097] Step 7:

[1098] The server selects the most relevant lecture videos based on the user's input interests and challenges, as well as their perceived emotional state.

[1099] Step 8:

[1100] The server retrieves the transcripts of the selected lecture videos. The transcripts are a written version of the video content.

[1101] Step 9:

[1102] The server sends the transcript to a natural language processing engine, which processes the transcript to extract key points.

[1103] Step 10:

[1104] A natural language processing engine analyzes the transcript and summarizes key points and key points, which are concise summaries of the essence of the lecture video.

[1105] Step 11:

[1106] The server generates a specific action plan based on the summary, taking into account the user's emotional state. For example, if the user is feeling stressed, the action plan may include stress management and relaxation.

[1107] Step 12:

[1108] The server sends the generated summary and action plan to the user's terminal.

[1109] Step 13:

[1110] The terminal stores the received summary and action plan and notifies the user that the user has received new information.

[1111] Step 14:

[1112] The user checks the notification and views the summary and action plan displayed on the device, allowing the user to create a specific action plan.

[1113] In this way, a system is provided that allows users to efficiently absorb knowledge from lecture videos and create specific action plans according to their emotional state.

[1114] Example 2

[1115] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1116] Previous systems were unable to consider the user's emotional state when providing relevant lecture videos based on the user's interests and challenges. As a result, they were unable to provide optimal information for the user or generate specific and effective action plans. Furthermore, they were unable to appropriately change feedback according to the user's individual emotional state, which limited the user experience.

[1117] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1118] In this invention, the server includes a means for selecting relevant lecture videos based on the interests and issues input by the user, a means for recognizing emotions from the user's input information and interactions, and a means for selecting relevant videos in consideration of the recognized emotional information. This makes it possible to select the most appropriate lecture video taking into consideration the user's emotional state and to provide a specific action plan according to the emotion.

[1119] "User" means any individual or entity that uses the System.

[1120] "Interests" refers to the themes and topics that interest users.

[1121] "Challenge" refers to the problem the user is currently facing or the issue they want to solve.

[1122] A "lecture video" is video content in which a specialist or expert explains a specific topic.

[1123] "Means" refers to methods or techniques for achieving a specific purpose.

[1124] A "summary" is a concise summary of the main points extracted from the content of a lecture video.

[1125] An "action plan" is a set of specific recommended actions that can be taken based on the summarized content.

[1126] "Means for recognizing emotions" refers to technology that analyzes a user's emotional state based on their behavior, expressions, etc. Examples include keyboard input speed and facial expression analysis.

[1127] "Natural language processing technology" is a technology that allows machines to analyze text data, understand its meaning, and process human language.

[1128] "Metadata" is additional information that accompanies the data, and includes, for example, the theme and keywords of the video.

[1129] A "server" is a computer system that receives requests from users and processes them in cooperation with a database.

[1130] A "database" is a system that organizes, manages, and stores a variety of data.

[1131] "Interaction" refers to a series of operations, actions, and inputs that a user makes with a system.

[1132] The present invention combines a system that selects relevant lecture videos based on a user's interests and challenges, summarizes the content, and provides a specific action plan with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[1133] User Interface

[1134] Users log in to the system using a terminal. After logging in, they input their interests (e.g., strengthening leadership) and challenges (e.g., improving team morale). The emotion engine also analyzes the user's emotional state based on facial expressions and keyboard typing speed. This allows the system to take the user's emotions into account.

[1135] Sending information to the server

[1136] The device sends the collected user input information and emotion information to the server, which then performs subsequent processing based on this information.

[1137] Selection of lecture videos

[1138] The server analyzes the received information and searches for relevant lecture videos in its database. The database stores multiple lecture videos, each of which is assigned metadata such as theme, keywords, and emotion tags. The server compares this metadata with user information to select the most suitable video.

[1139] Processing the transcript

[1140] The server sends the transcripts (written transcripts of what was said) of the selected lecture videos to a natural language processing engine, which analyzes the transcripts and extracts key points.

[1141] Generate a summary and action plan

[1142] The natural language processing engine generates a summary based on the extracted key points, and then takes into account the emotional information from the emotion engine to create a specific action plan suited to the user's situation, such as "introducing relaxation techniques" for users experiencing high stress.

[1143] Notifying users of the results

[1144] The server sends the generated summary and action plan to the user's device, where the user can review the information and create a workable action plan for real life.

[1145] Specific examples

[1146] For example, consider a user who has an interest in "I want to deepen my knowledge about leadership in the workplace" and a goal to "improve team morale and performance." If the emotion engine recognizes the user's emotional state as "high stress," this information will also be taken into account.

[1147] The user enters this information into the device, which then sends it to the server. The server then searches for relevant lecture videos based on keywords like "leadership" and "team motivation," taking into account the user's emotional state of "high stress." For example, a video titled "Lectures on the psychological aspects of leadership" might be selected.

[1148] The server sends the transcript of this talk to a natural language processing engine, which summarizes the main points and generates an action plan like this:

[1149] Use relaxation techniques regularly

[1150] Facilitate open stress management dialogue within your team

[1151] Introduce flexible scheduling

[1152] The server sends this information to the user's device, where the user can review it and take action in real life.

[1153] Prompt Sentence Examples

[1154] "I would like to deepen my knowledge of leadership in the workplace. I would also like to motivate my team and improve their performance. Based on this information, please suggest appropriate talk videos, provide summaries, and concrete action plans."

[1155] In this way, this system not only provides information based on the user's interests and challenges, but also provides optimal content and specific action plans that take into account the user's emotional state.

[1156] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1157] Step 1:

[1158] A user logs into the system using a terminal. After logging in, the user inputs their interests (e.g., strengthening leadership) and challenges (e.g., improving team morale). The terminal collects the input information and prepares it. The input at this time is text data related to the user's interests and challenges.

[1159] Step 2:

[1160] The device sends the collected input information to the server. Specifically, it sends data using an HTTP POST request. This request includes text data of the user's input interests and tasks. The output is the input information data received by the server.

[1161] Step 3:

[1162] The server analyzes the received input information and uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's input information and interactions (e.g., keyboard input speed, facial expression analysis, etc.) to extract the user's emotional state. For example, emotional data such as "high stress" or "low motivation" can be obtained.

[1163] Step 4:

[1164] The server searches for relevant lecture videos in the database, taking into account the emotional information from the emotion engine and user input information. The database stores metadata of lecture videos (e.g., themes, keywords, emotional tags), and matches this with user information to select the most suitable video. The input is user information and emotional information, and the output is the selected lecture video.

[1165] Step 5:

[1166] The server sends the transcripts of the selected lecture videos to a natural language processing engine, which analyzes the transcripts and extracts key points. The input is the lecture video transcript, and the output is a summarized version of the key points.

[1167] Step 6:

[1168] The natural language processing engine generates a summary based on the extracted key points, and then generates a specific action plan taking into account the emotional information obtained from the emotion engine. For example, if the user is "highly stressed," the following action plan will be generated:

[1169] Use relaxation techniques regularly

[1170] Facilitate open stress management dialogue within your team

[1171] Introduce flexible scheduling

[1172] The input is key points and emotional information, and the output is a summary and an action plan.

[1173] Step 7:

[1174] The server sends the generated summary and action plan to the user's device. The user can check this information on the device and create a specific action plan. The input is the summary and action plan, and the output is information provided to the user. A specific example of operation would be when the user clicks the "View Results" button on the device to check the provided summary and action plan.

[1175] (Application example 2)

[1176] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1177] While users need to properly learn about home security knowledge and techniques and take concrete measures, it is difficult to select appropriate content from the vast amount of information available. Furthermore, there are few systems that provide appropriate content and measures based on the user's emotional state and stress level. Therefore, there is a need for a system that provides information optimized for each user's individual situation and provides a concrete action plan.

[1178] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1179] In this invention, the server includes means for selecting relevant video content based on interests and issues input by the user, means for summarizing the content of the selected video content, means for generating a specific action plan based on the summarized content, means for recognizing the emotional state of the user using emotion recognition technology, and means for selecting appropriate video content based on the emotional state of the user, thereby enabling the user to efficiently obtain appropriate security content and an action plan.

[1180] "User" refers to any individual or organization that uses this system.

[1181] "Interests" refers to the information users want to know and the topics they want to learn about.

[1182] A "challenge" refers to a problem that a user is currently facing or wants to solve.

[1183] "Video content" refers to information in video format stored on the Internet or in a database.

[1184] A "summary" refers to information that concisely summarizes the main points of the video content.

[1185] An "action plan" is a plan that indicates specific actions that a user should take based on the summarized information.

[1186] "Emotion recognition technology" refers to technology that analyzes a user's facial expressions, behavior, etc. to identify their emotional state.

[1187] "Emotional state" refers to a user's current psychological state or mood.

[1188] "Means for selection" refers to a method or system for selecting relevant video content based on a user's interests and challenges.

[1189] "Means for providing" refers to a method or system for notifying the user of the generated action plan or summary information.

[1190] "Means for summarizing" refers to a method or system that uses natural language processing technology to concisely summarize the content of video content.

[1191] "Means for generating" refers to a method or system that generates an action plan based on the summarized content.

[1192] "Means for recognizing" refers to a method or system that uses emotion recognition technology to analyze and identify a user's emotional state.

[1193] MODE FOR CARRYING OUT THE INVENTION

[1194] System Overview

[1195] This invention combines emotion recognition technology with a system that selects appropriate video content based on the user's interests and challenges, and provides a summary and a concrete action plan. The system includes the following main components:

[1196] User interface: Users input their interests and concerns using their smartphones.

[1197] Server: Analyzes the input information and selects relevant video content.

[1198] Emotion Recognition Engine: Recognizes the user's emotional state.

[1199] Natural language processing engine: Summarizes the content of selected videos and generates specific action plans.

[1200] Database: Stores video content and makes it searchable.

[1201] Hardware and Software Configuration

[1202] Smartphones: Used as a user interface, they use a camera to capture the user's facial expressions.

[1203] Emotion Recognition Engine: Uses Affectiva SDK or equivalent emotion recognition technology.

[1204] Natural language processing engine: Uses OpenAI GPT or a similar generative model.

[1205] Database: Use AWS RDS or Google Firebase to store video content and its metadata.

[1206] Processing flow

[1207] User Interface

[1208] Users input their interests and current challenges through a smartphone application, and the smartphone camera captures the user's facial expressions and sends them to an emotion recognition engine.

[1209] emotion recognition technology

[1210] The emotion recognition engine (Affectiva SDK) analyzes the captured facial images of the user and recognizes their emotional state. This emotional information is sent to the server along with the user's interests and concerns.

[1211] Video content selection

[1212] The server analyzes the received information (interests, issues, emotional state) and searches for relevant video content from a database, using video metadata and tags.

[1213] Generate summaries and action plans

[1214] The server then sends the transcripts of the selected video content to a natural language processing engine (OpenAI GPT), which extracts key points from the transcript and generates a concise summary. At the same time, it generates a specific action plan based on these key points. Taking into account the user's emotional state, an appropriate action plan is created based on the user's psychological situation.

[1215] Notification of results

[1216] The generated summary and action plan are sent from the server to the user's smartphone, where the user can review the information and implement specific action plans.

[1217] Specific examples

[1218] As a concrete example, let's say a user has an interest in "deepening their knowledge about home security" and a goal of "strengthening measures against suspicious individuals." Furthermore, if the emotion recognition technology recognizes the user's emotional state as "high stress," this information is also taken into consideration. The user enters this information into their smartphone, captures their facial expressions with the camera, and sends them to the emotion recognition engine. The server searches for videos related to "home security" and "measures against suspicious individuals," taking their emotional state into consideration.

[1219] For example, if the selected video is a "Lecture on Home Security," the server sends the transcript of this lecture to a natural language processing engine to generate a summary and action plan, which could look something like this:

[1220] Summary: Locking gates and installing security cameras are important ways to improve home security.

[1221] Action plan: Select the type of security camera that will be effective in preventing suspicious individuals and determine where to install it.

[1222] Prompt Sentence Examples

[1223] User Input: I want to solve home security issues

[1224] Emotion: High stress

[1225] Prompt for the generative AI model:

[1226] Create a summary of your home security measures and provide a specific action plan for high-stress situations.

[1227] This allows users to efficiently obtain appropriate security content and action plans.

[1228] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1229] Step 1:

[1230] Users use their smartphones to input their interests and current challenges. This information is sent from the device to the server. Input can be done using text boxes or selection lists.

[1231] Input: Interests (e.g., "Home security"), Issues (e.g., "Countermeasures against suspicious persons")

[1232] Output: User interest and issue data sent to the server

[1233] Step 2:

[1234] The user captures their facial expression using the smartphone camera. The emotion recognition engine analyzes the facial image and recognizes the user's emotional state. The recognition results are sent to the server.

[1235] Input: facial expression image

[1236] Output: User's emotional state (e.g. "high stress")

[1237] Step 3:

[1238] The server analyzes the received interest, task, and emotional state data and searches a database for relevant video content, using video metadata and tags.

[1239] Input: Interests, tasks, and emotional state data

[1240] Output: Selected video content (e.g., "Lecture video on home security measures")

[1241] Step 4:

[1242] The server sends the transcript of the selected video content to a natural language processing engine, which generates a summary and a concrete action plan, automatically generated using a generative AI model.

[1243] Input: Transcript of video content

[1244] Output: Summary and action plan (e.g., "Lock the gates and install security cameras")

[1245] Step 5:

[1246] The server sends the generated summary and action plan to the user's device, where the user can review it and take specific action.

[1247] Input: Summary and Action Plan

[1248] Output: Summary and action plan displayed on user terminal

[1249] This allows users to select appropriate video content based on their interests and challenges, summarize the content in an easy-to-understand format, and put a concrete action plan into action.

[1250] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1251] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1252] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1253] [Fourth embodiment]

[1254] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1255] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1256] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1257] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1258] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1259] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1260] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1261] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1262] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1263] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1264] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1265] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1266] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1267] The present invention is a system that selects lecture videos based on a user's interests and challenges, and provides summaries and specific action plans. Specific embodiments of this system are described in detail below.

[1268] System Overview

[1269] The system includes the following major components:

[1270] 1. User Interface

[1271] 2. Server

[1272] 3. Natural Language Processing Engine

[1273] 4. Database

[1274] Program processing

[1275] 1. User Interface

[1276] Users log in using a device and enter their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"). This information is collected by the device and sent to the server.

[1277] 2. Server-side processing

[1278] The server analyzes the received information and selects relevant lecture videos. The database stores a large number of lecture videos, each of which has associated themes and keywords as metadata. Based on this metadata, the server searches for and selects the video that best suits the user's request.

[1279] 3. Natural Language Processing Engine

[1280] The server sends the transcripts of the selected lecture videos to a natural language processing engine. The engine extracts key points from the transcripts and generates a concise summary. At the same time, it generates a specific action plan based on these key points. For example, if the user's interest is "leadership" and the challenge is "improving team motivation," the engine will generate the following action plan:

[1281] Have one-on-one meetings with the whole team every week

[1282] Set specific goals for your team and regularly review progress

[1283] Host workshops to foster open communication

[1284] 4. Notification of Results

[1285] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[1286] Specific examples

[1287] As a concrete example, let's say a user has an interest in "deepening their knowledge about leadership in the workplace" and a challenge in "raising team morale and improving performance." The user enters this information into the device.

[1288] The device sends this information to the server, which then searches for relevant lecture videos based on the keywords "leadership" and "team motivation." For example, "lectures by renowned leadership experts" might be selected.

[1289] The server sends the transcript of this talk to a natural language processing engine, extracts the main points, and creates a summary, along with an action plan like this:

[1290] Conduct regular one-on-one feedback sessions with team members

[1291] Plan and implement team building activities

[1292] Implement a system to visualize and quantitatively evaluate team performance

[1293] The server sends this information to the user's device, where the user can review it and take action in real life.

[1294] In this way, a system is provided that allows users to efficiently gather knowledge and create specific action plans.

[1295] The processing flow will be explained below.

[1296] Step 1:

[1297] A user logs in using a terminal. After logging in, the user enters their interests (e.g., "Leadership") and the challenges they are currently facing (e.g., "I want to improve my team's motivation").

[1298] Step 2:

[1299] The terminal collects the user's input information and prepares to send it to the server.

[1300] Step 3:

[1301] The device sends the user's input information to the server. The sent information includes the user's interests and concerns.

[1302] Step 4:

[1303] The server analyzes the received user information, and extracts keywords that are most relevant to the user's interests and issues.

[1304] Step 5:

[1305] The server searches a database for relevant lecture videos, which contain metadata such as themes and keywords related to each video.

[1306] Step 6:

[1307] The server selects the most relevant lecture videos based on the user's input interests and challenges.

[1308] Step 7:

[1309] The server retrieves the transcripts of the selected lecture videos. The transcripts are a written version of the video content.

[1310] Step 8:

[1311] The server sends the transcript to a natural language processing engine, which processes the transcript to extract key points.

[1312] Step 9:

[1313] A natural language processing engine analyzes the transcript and summarizes key points and key points, which are concise summaries of the essence of the lecture video.

[1314] Step 10:

[1315] The server generates a concrete action plan based on the summarized content, which is a concrete action plan that the user can carry out in real life.

[1316] Step 11:

[1317] The server sends the generated summary and action plan to the user's terminal.

[1318] Step 12:

[1319] The terminal stores the received summary and action plan and notifies the user that the user has received new information.

[1320] Step 13:

[1321] The user checks the notification and views the summary and action plan displayed on the device, allowing the user to create a specific action plan.

[1322] In this way, users can efficiently absorb the knowledge from the lecture videos and put it into concrete action.

[1323] Example 1

[1324] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1325] With conventional systems, it was difficult for users to efficiently collect information that matched their interests and challenges and to create specific action plans. In particular, there was a problem in that it took a great deal of time and effort to select highly relevant information from a large amount of media content, summarize that content, and create an action plan.

[1326] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1327] In this invention, the server includes means for selecting relevant media content based on the interests and issues input by the user, means for summarizing the selected media content, and means for generating a specific action plan based on the summarized content, thereby enabling the user to quickly and efficiently obtain information that is in line with their interests and issues and to create a specific action plan based on that information.

[1328] A "user" is an individual who operates the system and inputs their interests and tasks.

[1329] "Interests" are topics or themes that users particularly want to know about, learn about, or explore further.

[1330] A "challenge" is a problem the user wants to solve or an obstacle they want to overcome.

[1331] "Media content" refers to information or materials that are primarily provided in the form of video, audio, text, etc.

[1332] "Selection tools" are functions that identify and select the most appropriate media content based on user-entered interests and challenges.

[1333] The "summarizing means" is a function that briefly summarizes the content of the selected media content and extracts the main points.

[1334] "Means for generating a concrete action plan" is a function that suggests specific steps or actions that the user should take based on the summarized points.

[1335] The "means of providing" is a function that presents the generated action plan to the user in an easy-to-understand manner, allowing them to use it as a reference for action.

[1336] A "database" is a storage device that stores media content and its associated themes and keywords and allows them to be searched as needed.

[1337] "Natural language processing technology" is a technology for analyzing language data (e.g., transcripts) and understanding their meaning.

[1338] A "transcript" is text data that converts the content of video, audio, etc. into text.

[1339] "Means of analysis" refers to the function of analyzing data such as user input and transcripts and extracting necessary information.

[1340] The present invention is a system that selects optimal media content based on a user's interests and challenges, summarizes the content, and provides a specific action plan. The system mainly consists of the following components:

[1341] 1. User Interface

[1342] 2. Server

[1343] 3. Natural Language Processing Engine

[1344] 4. Database

[1345] User Interface

[1346] Users log in to the system using a terminal and enter their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"), which are collected by the terminal and sent to the server.

[1347] server

[1348] The server analyzes the information received from the user and selects relevant media content. The server is connected to a database that stores a large number of media contents, and each content is assigned related themes and keywords as metadata. The server uses this metadata to search for and select the content that best suits the user's request.

[1349] Natural Language Processing Engine

[1350] The server sends the transcript of the selected media content to a natural language processing engine, which extracts key points from the transcript and generates a concise summary, along with a concrete plan of action based on those key points.

[1351] For example, if the user's interest is "leadership" and the challenge is "improving team motivation," the engine will generate an action plan like this:

[1352] Have one-on-one meetings with the whole team every week

[1353] Set specific goals for your team and regularly review progress

[1354] Host workshops to foster open communication

[1355] Notification of results

[1356] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[1357] Specific examples

[1358] As a concrete example, let's say a user has an interest in "deepening their knowledge about leadership in the workplace" and a challenge in "raising team morale and improving performance." The user enters this information into the device.

[1359] The device sends this information to a server, which then searches for relevant media content based on the keywords "leadership" and "team motivation." For example, it might select "video lectures by renowned leadership experts" from a database.

[1360] The server sends the transcript of this talk to a natural language processing engine, extracts the main points, and generates a summary, along with an action plan like this:

[1361] Conduct regular one-on-one feedback sessions with team members

[1362] Plan and implement team building activities

[1363] Implement a system to visualize and quantitatively evaluate team performance

[1364] Prompt Sentence Examples

[1365] "Based on the leadership presentation video, what is your specific action plan for improving team motivation?"

[1366] "Find a video of a talk that will help motivate your team and provide a summary and actionable plan of action."

[1367] This system allows users to efficiently gather knowledge and create specific action plans, thereby providing effective support for achieving individual goals.

[1368] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1369] Step 1:

[1370] User Login

[1371] A user logs into the system using a terminal. The user accesses the login screen and enters their username and password. The terminal sends this information to the server, and if authentication is successful, the home screen is displayed.

[1372] Input: The username and password entered by the user

[1373] Output: Show user's home screen upon successful authentication

[1374] Step 2:

[1375] Input of user interests and issues

[1376] The user enters their interests (e.g., "Leadership") and challenges (e.g., "I want to improve my team's motivation"). The user enters these into the form on the home screen and presses the submit button. The input information is sent from the device to the server.

[1377] Input: User-entered interests and challenges

[1378] Output: User interests and challenges sent to the server

[1379] Step 3:

[1380] Sending user information

[1381] The device sends the entered information to the server. The device summarizes the user's interests and challenges and sends them to the server in the form of an HTTP request. The server receives this request and begins processing.

[1382] Input: User interests and challenges sent from the device

[1383] Output: User information in a format that can be parsed by the server

[1384] Step 4:

[1385] Server information analysis

[1386] The server analyzes the received user information and extracts relevant keywords. The server uses an analysis module to identify themes related to interests and issues. Based on the extracted keywords, the next step is to search for related media content.

[1387] Input: User information (interests and issues)

[1388] Output: Extracted related keywords

[1389] Step 5:

[1390] Selection of relevant media content

[1391] The server searches the database and selects relevant media content based on the extracted keywords. The server sends a keyword search query to the database and retrieves a list of the most relevant media content.

[1392] Input: Extracted keywords

[1393] Output: A list of related media content

[1394] Step 6:

[1395] Sending a transcript

[1396] The server sends the most relevant media content transcripts to the natural language processing engine, which then parses the transcripts, which are a transcription of the audio from the video.

[1397] Input: Transcript of associated media content

[1398] Output: Transcript sent to the natural language processing engine

[1399] Step 7:

[1400] Generate summaries and action plans

[1401] A natural language processing engine analyzes the transcript and generates a summary and a concrete plan of action. The engine extracts key points from the transcript, creates a concise summary, and suggests a concrete plan of action based on these points.

[1402] Input: Transcript

[1403] Output: Summary and action plan

[1404] Step 8:

[1405] Notification of results

[1406] The server sends the generated summary and action plan to the user's device, where the user can review the information and plan a specific course of action. The server sends this information in the form of an HTTP response.

[1407] Input: Summary and Action Plan

[1408] Output: Summary and action plan sent to the user's device

[1409] (Application example 1)

[1410] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1411] Conventional lecture video delivery systems are limited to providing video summaries and specific action plans, and have the problem of not directly contributing to improving users' actual work or performance. Furthermore, the observation of users' work and suggestions for improvement themselves are often done manually, which is inefficient. The present invention aims to improve work efficiency and effectiveness by observing users' work and providing specific action plans in real time based on the observations.

[1412] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1413] In this invention, the server includes means for selecting relevant lecture videos based on the interests and challenges input by the user, means for summarizing the content of the selected lecture videos, means for generating a specific action plan based on the summarized content, means for observing the user's work with a robot and analyzing the observation data, and means for providing the generated action plan to the user via the robot's display and voice notification. This makes it possible to monitor the user's work status in real time and instantly provide optimal improvement suggestions.

[1414] "Means for selecting relevant lecture videos based on the interests and issues entered by the user" refers to a function that analyzes the interest and issue information entered by the user using an input device and automatically selects the video content that is most relevant to that information from a database.

[1415] "Means for summarizing the content of selected lecture videos" refers to a function that analyzes the content of selected videos using natural language processing technology and provides important points in a shortened format.

[1416] "Means for generating specific action plans based on the summarized content" refers to the function of generating specific action plans that can actually be implemented based on the main points extracted from the content of the summarized lecture video.

[1417] "Means for the robot to observe the user's work and analyze the observed data" refers to the function of the robot to monitor the user's work status using cameras and sensors, and to collect and analyze the data in real time.

[1418] "Means for providing the generated action plan to the user through the robot's display and voice notification" refers to the function of displaying the generated specific action plan on the robot's display and communicating it to the user through voice notification.

[1419] "Means for receiving input of interests and challenges from users" refers to a function that provides an interface for users to input their interests and challenges they are facing into the system.

[1420] "Natural language processing technology" is a technology that enables machines to understand and process human language, and refers to technology that analyzes text, generates summaries, extracts meaning, etc.

[1421] This invention is a system that selects relevant lecture videos based on a user's interests and challenges, and provides summaries and specific action plans. Furthermore, the system aims to support factory robot operations by observing the user's work and making suggestions for work improvement in real time. A specific embodiment of this system is described in detail below.

[1422] System Overview

[1423] The system includes the following major components:

[1424] 1. User Interface

[1425] 2. Server

[1426] 3. Natural Language Processing Engine

[1427] 4. Database

[1428] 5. Factory robots

[1429] Program processing

[1430] 1. User Interface

[1431] Users log in using their devices and enter their interests (e.g., "quality control") and challenges (e.g., "reducing lead time"). This information is collected by the device and sent to the server, which then obtains basic data to understand the user's needs and select appropriate lecture videos.

[1432] 2. Server-side processing

[1433] The server analyzes the received information and selects relevant lecture videos. The database stores a large number of lecture videos, each of which is assigned metadata such as related themes and keywords. Based on this metadata, the server searches for and selects the video that best suits the user's request. This process uses advanced data analysis algorithms.

[1434] 3. Natural Language Processing Engine

[1435] The server sends the transcripts of the selected lecture videos to a natural language processing engine. This engine uses the transformers library to extract key points from the transcripts and generate summaries. At the same time, it generates specific action plans based on these key points. For example, if the user's interest is "quality control" and the challenge is "reducing lead time," the engine will generate the following action plan:

[1436] Increase the frequency of quality inspections

[1437] Review of standard operating procedures

[1438] Introducing a system to visualize work efficiency

[1439] 4. Observing factory robots at work

[1440] Factory robots use cameras and sensors to observe the user's work in real time. This observation data is sent to a server as needed and analyzed. The work data observed by the robot is associated with themes and issues entered by the user and used to generate more accurate action plans.

[1441] 5. Notification of Results

[1442] The generated summary and action plan are sent from the server to the user's device and the factory robot, which then displays the information on the display and notifies the user via voice notification. The results can also be viewed via a smartphone app.

[1443] Specific examples

[1444] As a concrete example, let's say a user has the interests and challenges of "I want to deepen my knowledge about quality control" and "I want to shorten lead time." The user enters this information into their device, which then sends it to the server. The server searches for lecture videos on "quality control" and selects appropriate videos. The transcripts of the selected videos are summarized using a natural language processing engine, and specific action plans such as "increase the frequency of quality inspections" and "review standard operating procedures" are generated.

[1445] Prompt Sentence Examples

[1446] The user's interests and challenges are "quality control" and "shortening lead time." Please select a lecture video that meets these criteria and generate a summary of it. Also, please provide a specific action plan for improvement based on these criteria.

[1447] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1448] Step 1:

[1449] A user logs in using a terminal and inputs their interests and challenges. The inputted interest is "quality control" and the challenge is "shortening lead time." This information is collected by the terminal and sent to the server. The input here is the user's text data, and the output is a data packet that sends this to the server.

[1450] Step 2:

[1451] The server analyzes the received data on the user's interests and challenges using a keyword extraction algorithm. The server matches the metadata of the lecture videos in the database with the keywords contained in the user's interests and challenges to select relevant lecture videos. The input is keywords and metadata, and the output is a list of the most suitable lecture videos.

[1452] Step 3:

[1453] We take the transcripts of selected talk videos and send them to a natural language processing engine, which uses the transformers library to extract key points from the transcripts and generate concise summaries. The input is the talk video transcript, and the output is the summary text.

[1454] Step 4:

[1455] The same natural language processing engine is used to generate specific action plans from the summary. The engine further analyzes the summary text and generates actionable plans, such as "increase the frequency of quality inspections" or "review standard operating procedures." The input is the summary text, and the output is a list of action plans.

[1456] Step 5:

[1457] Factory robots use cameras and sensors to observe the user's work. This observation data is sent to a server in real time. The server analyzes the observation data and evaluates the user's work status. The input is the observation data, and the output is the analysis result.

[1458] Step 6:

[1459] The server adjusts and optimizes the generated action plan based on the analysis results. The observation data and the action plan are collated to provide specific suggestions tailored to the user's current situation. The input is the observation data and the existing action plan, and the output is an optimized action plan.

[1460] Step 7:

[1461] The generated summary and action plan are sent from the server to the user's device and the factory robot. The factory robot displays the information on a display and communicates it to the user through a voice notification. The same information can also be viewed on the user's device. The input is the optimized action plan, and the output is the display and voice notification.

[1462] An example prompt is:

[1463] "The user's interests and challenges are 'quality control' and 'shortening lead time.' Please select a lecture video that meets these criteria and generate a summary of it. Also, please provide a specific action plan for improvement based on these criteria."

[1464] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1465] The present invention combines a system that selects relevant lecture videos based on a user's interests and challenges, summarizes the content, and provides a specific action plan with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[1466] System Overview

[1467] The system includes the following major components:

[1468] 1. User Interface

[1469] 2. Server

[1470] 3. Natural Language Processing Engine

[1471] 4. Emotion Engine

[1472] 5. Database

[1473] Program processing

[1474] 1. User Interface

[1475] Users log in using their devices and input their interests and the challenges they are currently facing. This information is collected by the device and sent to the server. The emotion engine then recognizes the user's emotions based on the information they input and their interactions.

[1476] 2. Server-side processing

[1477] The server analyzes the received information and selects relevant lecture videos, taking into account the user's recognized emotions. The database stores a large number of lecture videos, each of which has associated themes and keywords as metadata. The server combines this metadata with emotional information to search for and select the most appropriate video.

[1478] 3. Emotion Engine

[1479] The emotion engine analyzes user input and interactions (e.g., keyboard input speed, facial expression analysis, etc.) to recognize the user's emotional state. This information is sent to the server and used to select videos and generate action plans.

[1480] 4. Natural Language Processing Engine

[1481] The server sends the transcripts of the selected lecture videos to a natural language processing engine, which extracts key points from the transcripts and generates a concise summary. At the same time, a specific action plan is generated based on these key points. Taking into account the results of the emotion engine, an appropriate action plan is created according to the user's emotional state.

[1482] 5. Notification of Results

[1483] The generated summary and action plan are sent from the server to the user's device, where the user can review the information and create a specific action plan.

[1484] Specific examples

[1485] For example, if a user has an interest in "I want to deepen my knowledge about leadership in the workplace" and a goal in "I want to improve team morale and performance," if the emotion engine recognizes the user's emotional state as "high stress," it will also take this information into account.

[1486] The user enters this information into the device, which then sends it to the server. The server searches for relevant lecture videos based on the keywords "leadership" and "team motivation," while also taking into account the user's current emotional state of "high stress." For example, a "lecture on the psychological aspects of leadership" might be selected.

[1487] The server sends the transcript of this talk to a natural language processing engine, which summarizes the main points and generates an action plan like this:

[1488] Use relaxation techniques regularly

[1489] Facilitate open stress management dialogue within your team

[1490] Introduce flexible scheduling

[1491] The server sends this information to the user's device, where the user can review it and take action in real life.

[1492] In this way, a system is provided that allows the user to efficiently gather knowledge and create a specific action plan according to their emotional state.

[1493] The processing flow will be explained below.

[1494] Step 1:

[1495] A user logs in using a terminal. After logging in, the user enters their interests (e.g., "Leadership") and the challenges they are currently facing (e.g., "I want to improve my team's motivation").

[1496] Step 2:

[1497] The emotion engine analyzes the user's input information and interactions (e.g., keyboard input speed, facial expression analysis, etc.) and recognizes the user's emotional state (e.g., "high stress").

[1498] Step 3:

[1499] The terminal collects the user's input information and emotional state and prepares to send it to the server.

[1500] Step 4:

[1501] The terminal transmits the user's input information and emotional state to the server, and the transmitted information includes the user's interests, concerns, and emotional state.

[1502] Step 5:

[1503] The server analyzes the received user information, extracting keywords that are most relevant to the user's interests and issues, and also taking into account the user's emotional state.

[1504] Step 6:

[1505] The server searches the database for relevant lecture videos. The lecture video database contains metadata such as themes and keywords related to each video. The server combines this metadata with emotional information to search for and select the most appropriate video.

[1506] Step 7:

[1507] The server selects the most relevant lecture videos based on the user's input interests and challenges, as well as their perceived emotional state.

[1508] Step 8:

[1509] The server retrieves the transcripts of the selected lecture videos. The transcripts are a written version of the video content.

[1510] Step 9:

[1511] The server sends the transcript to a natural language processing engine, which processes the transcript to extract key points.

[1512] Step 10:

[1513] A natural language processing engine analyzes the transcript and summarizes key points and key points, which are concise summaries of the essence of the lecture video.

[1514] Step 11:

[1515] The server generates a specific action plan based on the summary, taking into account the user's emotional state. For example, if the user is feeling stressed, the action plan may include stress management and relaxation.

[1516] Step 12:

[1517] The server sends the generated summary and action plan to the user's terminal.

[1518] Step 13:

[1519] The terminal stores the received summary and action plan and notifies the user that the user has received new information.

[1520] Step 14:

[1521] The user checks the notification and views the summary and action plan displayed on the device, allowing the user to create a specific action plan.

[1522] In this way, a system is provided that allows users to efficiently absorb knowledge from lecture videos and create specific action plans according to their emotional state.

[1523] Example 2

[1524] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1525] Previous systems were unable to consider the user's emotional state when providing relevant lecture videos based on the user's interests and challenges. As a result, they were unable to provide optimal information for the user or generate specific and effective action plans. Furthermore, they were unable to appropriately change feedback according to the user's individual emotional state, which limited the user experience.

[1526] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1527] In this invention, the server includes a means for selecting relevant lecture videos based on the interests and issues input by the user, a means for recognizing emotions from the user's input information and interactions, and a means for selecting relevant videos in consideration of the recognized emotional information. This makes it possible to select the most appropriate lecture video taking into consideration the user's emotional state and to provide a specific action plan according to the emotion.

[1528] "User" means any individual or entity that uses the System.

[1529] "Interests" refers to the themes and topics that interest users.

[1530] "Challenge" refers to the problem the user is currently facing or the issue they want to solve.

[1531] A "lecture video" is video content in which a specialist or expert explains a specific topic.

[1532] "Means" refers to methods or techniques for achieving a specific purpose.

[1533] A "summary" is a concise summary of the main points extracted from the content of a lecture video.

[1534] An "action plan" is a set of specific recommended actions that can be taken based on the summarized content.

[1535] "Means for recognizing emotions" refers to technology that analyzes a user's emotional state based on their behavior, expressions, etc. Examples include keyboard input speed and facial expression analysis.

[1536] "Natural language processing technology" is a technology that allows machines to analyze text data, understand its meaning, and process human language.

[1537] "Metadata" is additional information that accompanies the data, and includes, for example, the theme and keywords of the video.

[1538] A "server" is a computer system that receives requests from users and processes them in cooperation with a database.

[1539] A "database" is a system that organizes, manages, and stores a variety of data.

[1540] "Interaction" refers to a series of operations, actions, and inputs that a user makes with a system.

[1541] The present invention combines a system that selects relevant lecture videos based on a user's interests and challenges, summarizes the content, and provides a specific action plan with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[1542] User Interface

[1543] Users log in to the system using a terminal. After logging in, they input their interests (e.g., strengthening leadership) and challenges (e.g., improving team morale). The emotion engine also analyzes the user's emotional state based on facial expressions and keyboard typing speed. This allows the system to take the user's emotions into account.

[1544] Sending information to the server

[1545] The device sends the collected user input information and emotion information to the server, which then performs subsequent processing based on this information.

[1546] Selection of lecture videos

[1547] The server analyzes the received information and searches for relevant lecture videos in its database. The database stores multiple lecture videos, each of which is assigned metadata such as theme, keywords, and emotion tags. The server compares this metadata with user information to select the most suitable video.

[1548] Processing the transcript

[1549] The server sends the transcripts (written transcripts of what was said) of the selected lecture videos to a natural language processing engine, which analyzes the transcripts and extracts key points.

[1550] Generate a summary and action plan

[1551] The natural language processing engine generates a summary based on the extracted key points, and then takes into account the emotional information from the emotion engine to create a specific action plan suited to the user's situation, such as "introducing relaxation techniques" for users experiencing high stress.

[1552] Notifying users of the results

[1553] The server sends the generated summary and action plan to the user's device, where the user can review the information and create a workable action plan for real life.

[1554] Specific examples

[1555] For example, consider a user who has an interest in "I want to deepen my knowledge about leadership in the workplace" and a goal to "improve team morale and performance." If the emotion engine recognizes the user's emotional state as "high stress," this information will also be taken into account.

[1556] The user enters this information into the device, which then sends it to the server. The server then searches for relevant lecture videos based on keywords like "leadership" and "team motivation," taking into account the user's emotional state of "high stress." For example, a video titled "Lectures on the psychological aspects of leadership" might be selected.

[1557] The server sends the transcript of this talk to a natural language processing engine, which summarizes the main points and generates an action plan like this:

[1558] Use relaxation techniques regularly

[1559] Facilitate open stress management dialogue within your team

[1560] Introduce flexible scheduling

[1561] The server sends this information to the user's device, where the user can review it and take action in real life.

[1562] Prompt Sentence Examples

[1563] "I would like to deepen my knowledge of leadership in the workplace. I would also like to motivate my team and improve their performance. Based on this information, please suggest appropriate talk videos, provide summaries, and concrete action plans."

[1564] In this way, this system not only provides information based on the user's interests and challenges, but also provides optimal content and specific action plans that take into account the user's emotional state.

[1565] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1566] Step 1:

[1567] A user logs into the system using a terminal. After logging in, the user inputs their interests (e.g., strengthening leadership) and challenges (e.g., improving team morale). The terminal collects the input information and prepares it. The input at this time is text data related to the user's interests and challenges.

[1568] Step 2:

[1569] The device sends the collected input information to the server. Specifically, it sends data using an HTTP POST request. This request includes text data of the user's input interests and tasks. The output is the input information data received by the server.

[1570] Step 3:

[1571] The server analyzes the received input information and uses an emotion engine to recognize the user's emotional state. The emotion engine analyzes the user's input information and interactions (e.g., keyboard input speed, facial expression analysis, etc.) to extract the user's emotional state. For example, emotional data such as "high stress" or "low motivation" can be obtained.

[1572] Step 4:

[1573] The server searches for relevant lecture videos in the database, taking into account the emotional information from the emotion engine and user input information. The database stores metadata of lecture videos (e.g., themes, keywords, emotional tags), and matches this with user information to select the most suitable video. The input is user information and emotional information, and the output is the selected lecture video.

[1574] Step 5:

[1575] The server sends the transcripts of the selected lecture videos to a natural language processing engine, which analyzes the transcripts and extracts key points. The input is the lecture video transcript, and the output is a summarized version of the key points.

[1576] Step 6:

[1577] The natural language processing engine generates a summary based on the extracted key points, and then generates a specific action plan taking into account the emotional information obtained from the emotion engine. For example, if the user is "highly stressed," the following action plan will be generated:

[1578] Use relaxation techniques regularly

[1579] Facilitate open stress management dialogue within your team

[1580] Introduce flexible scheduling

[1581] The input is key points and emotional information, and the output is a summary and an action plan.

[1582] Step 7:

[1583] The server sends the generated summary and action plan to the user's device. The user can check this information on the device and create a specific action plan. The input is the summary and action plan, and the output is information provided to the user. A specific example of operation would be when the user clicks the "View Results" button on the device to check the provided summary and action plan.

[1584] (Application example 2)

[1585] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1586] While users need to properly learn about home security knowledge and techniques and take concrete measures, it is difficult to select appropriate content from the vast amount of information available. Furthermore, there are few systems that provide appropriate content and measures based on the user's emotional state and stress level. Therefore, there is a need for a system that provides information optimized for each user's individual situation and provides a concrete action plan.

[1587] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1588] In this invention, the server includes means for selecting relevant video content based on interests and issues input by the user, means for summarizing the content of the selected video content, means for generating a specific action plan based on the summarized content, means for recognizing the emotional state of the user using emotion recognition technology, and means for selecting appropriate video content based on the emotional state of the user, thereby enabling the user to efficiently obtain appropriate security content and an action plan.

[1589] "User" refers to any individual or organization that uses this system.

[1590] "Interests" refers to the information users want to know and the topics they want to learn about.

[1591] A "challenge" refers to a problem that a user is currently facing or wants to solve.

[1592] "Video content" refers to information in video format stored on the Internet or in a database.

[1593] A "summary" refers to information that concisely summarizes the main points of the video content.

[1594] An "action plan" is a plan that indicates specific actions that a user should take based on the summarized information.

[1595] "Emotion recognition technology" refers to technology that analyzes a user's facial expressions, behavior, etc. to identify their emotional state.

[1596] "Emotional state" refers to a user's current psychological state or mood.

[1597] "Means for selection" refers to a method or system for selecting relevant video content based on a user's interests and challenges.

[1598] "Means for providing" refers to a method or system for notifying the user of the generated action plan or summary information.

[1599] "Means for summarizing" refers to a method or system that uses natural language processing technology to concisely summarize the content of video content.

[1600] "Means for generating" refers to a method or system that generates an action plan based on the summarized content.

[1601] "Means for recognizing" refers to a method or system that uses emotion recognition technology to analyze and identify a user's emotional state.

[1602] MODE FOR CARRYING OUT THE INVENTION

[1603] System Overview

[1604] This invention combines emotion recognition technology with a system that selects appropriate video content based on the user's interests and challenges, and provides a summary and a concrete action plan. The system includes the following main components:

[1605] User interface: Users input their interests and concerns using their smartphones.

[1606] Server: Analyzes the input information and selects relevant video content.

[1607] Emotion Recognition Engine: Recognizes the user's emotional state.

[1608] Natural language processing engine: Summarizes the content of selected videos and generates specific action plans.

[1609] Database: Stores video content and makes it searchable.

[1610] Hardware and Software Configuration

[1611] Smartphones: Used as a user interface, they use a camera to capture the user's facial expressions.

[1612] Emotion Recognition Engine: Uses Affectiva SDK or equivalent emotion recognition technology.

[1613] Natural language processing engine: Uses OpenAI GPT or a similar generative model.

[1614] Database: Use AWS RDS or Google Firebase to store video content and its metadata.

[1615] Processing flow

[1616] User Interface

[1617] Users input their interests and current challenges through a smartphone application, and the smartphone camera captures the user's facial expressions and sends them to an emotion recognition engine.

[1618] emotion recognition technology

[1619] The emotion recognition engine (Affectiva SDK) analyzes the captured facial images of the user and recognizes their emotional state. This emotional information is sent to the server along with the user's interests and concerns.

[1620] Video content selection

[1621] The server analyzes the received information (interests, issues, emotional state) and searches for relevant video content from a database, using video metadata and tags.

[1622] Generate summaries and action plans

[1623] The server then sends the transcripts of the selected video content to a natural language processing engine (OpenAI GPT), which extracts key points from the transcript and generates a concise summary. At the same time, it generates a specific action plan based on these key points. Taking into account the user's emotional state, an appropriate action plan is created based on the user's psychological situation.

[1624] Notification of results

[1625] The generated summary and action plan are sent from the server to the user's smartphone, where the user can review the information and implement specific action plans.

[1626] Specific examples

[1627] As a concrete example, let's say a user has an interest in "deepening their knowledge about home security" and a goal of "strengthening measures against suspicious individuals." Furthermore, if the emotion recognition technology recognizes the user's emotional state as "high stress," this information is also taken into consideration. The user enters this information into their smartphone, captures their facial expressions with the camera, and sends them to the emotion recognition engine. The server searches for videos related to "home security" and "measures against suspicious individuals," taking their emotional state into consideration.

[1628] For example, if the selected video is a "Lecture on Home Security," the server sends the transcript of this lecture to a natural language processing engine to generate a summary and action plan, which could look something like this:

[1629] Summary: Locking gates and installing security cameras are important ways to improve home security.

[1630] Action plan: Select the type of security camera that will be effective in preventing suspicious individuals and determine where to install it.

[1631] Prompt Sentence Examples

[1632] User Input: I want to solve home security issues

[1633] Emotion: High stress

[1634] Prompt for the generative AI model:

[1635] Create a summary of your home security measures and provide a specific action plan for high-stress situations.

[1636] This allows users to efficiently obtain appropriate security content and action plans.

[1637] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1638] Step 1:

[1639] Users use their smartphones to input their interests and current challenges. This information is sent from the device to the server. Input can be done using text boxes or selection lists.

[1640] Input: Interests (e.g., "Home security"), Issues (e.g., "Countermeasures against suspicious persons")

[1641] Output: User interest and issue data sent to the server

[1642] Step 2:

[1643] The user captures their facial expression using the smartphone camera. The emotion recognition engine analyzes the facial image and recognizes the user's emotional state. The recognition results are sent to the server.

[1644] Input: facial expression image

[1645] Output: User's emotional state (e.g. "high stress")

[1646] Step 3:

[1647] The server analyzes the received interest, task, and emotional state data and searches a database for relevant video content, using video metadata and tags.

[1648] Input: Interests, tasks, and emotional state data

[1649] Output: Selected video content (e.g., "Lecture video on home security measures")

[1650] Step 4:

[1651] The server sends the transcript of the selected video content to a natural language processing engine, which generates a summary and a concrete action plan, automatically generated using a generative AI model.

[1652] Input: Transcript of video content

[1653] Output: Summary and action plan (e.g., "Lock the gates and install security cameras")

[1654] Step 5:

[1655] The server sends the generated summary and action plan to the user's device, where the user can review it and take specific action.

[1656] Input: Summary and Action Plan

[1657] Output: Summary and action plan displayed on user terminal

[1658] This allows users to select appropriate video content based on their interests and challenges, summarize the content in an easy-to-understand format, and put a concrete action plan into action.

[1659] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1660] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1661] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1662] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1663] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1664] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1665] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1666] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1667] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1668] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1669] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1670] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1671] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1672] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1673] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1674] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1675] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1676] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1677] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1678] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1679] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1680] The following is further disclosed regarding the above embodiment.

[1681] (Claim 1)

[1682] A means for selecting relevant lecture videos based on the user's inputted interests and challenges;

[1683] A means for summarizing the content of selected lecture videos;

[1684] A means for generating a concrete action plan based on the summarized content;

[1685] a means for providing the generated action plan to the user;

[1686] A system including:

[1687] (Claim 2)

[1688] 10. The system of claim 1, further comprising means for receiving input of interests and challenges from a user.

[1689] (Claim 3)

[1690] 10. The system of claim 1, which uses natural language processing technology to summarize the content of a lecture video and generate a specific action plan based on the main points contained in the summary.

[1691] "Example 1"

[1692] (Claim 1)

[1693] means for selecting relevant media content based on user-entered interests and challenges;

[1694] means for summarizing the content of the selected media content;

[1695] a means for generating a specific action plan based on the summarized content;

[1696] a means for providing the generated action plan to a user;

[1697] means for receiving input of interests and challenges from a user;

[1698] using a database of themes and keywords related to each piece of media content;

[1699] means for analyzing a transcript of the media content using natural language processing techniques to generate a summary and a plan of action;

[1700] A system including:

[1701] (Claim 2)

[1702] 10. The system of claim 1, further comprising means for analyzing the user's input and selecting relevant media content based on the analysis.

[1703] (Claim 3)

[1704] 10. The system of claim 1, further comprising means for notifying a user terminal of the generated summary and specific action plan.

[1705] "Application Example 1"

[1706] (Claim 1)

[1707] A means for selecting relevant lecture videos based on the user's inputted interests and challenges;

[1708] A means for summarizing the content of selected lecture videos;

[1709] A means for generating a concrete action plan based on the summarized content;

[1710] a means for observing a user's work by the robot and analyzing the observed data;

[1711] a means for providing the generated action plan to the user via a display and voice notification on the robot;

[1712] A system including:

[1713] (Claim 2)

[1714] 10. The system of claim 1, further comprising means for receiving input of interests and challenges from a user.

[1715] (Claim 3)

[1716] 10. The system of claim 1, which uses natural language processing technology to summarize the content of a lecture video and generate a specific action plan based on the main points contained in the summary.

[1717] "Example 2: Combining Emotion Engines"

[1718] (Claim 1)

[1719] A means for selecting relevant lecture videos based on the user's inputted interests and challenges;

[1720] A means for summarizing the content of selected lecture videos;

[1721] A means for generating a concrete action plan based on the summarized content;

[1722] a means for providing the generated action plan to the user;

[1723] A means for recognizing emotions from user input and interactions;

[1724] A means for selecting related videos taking into consideration the recognized emotion information;

[1725] A means for generating an appropriate action plan based on the emotional information;

[1726] A system including:

[1727] (Claim 2)

[1728] 10. The system of claim 1, further comprising means for receiving input of interests and challenges from a user.

[1729] (Claim 3)

[1730] 10. The system of claim 1, which uses natural language processing technology to summarize the content of a lecture video and generate a specific action plan based on the main points contained in the summary.

[1731] "Application example 2 when combining emotion engines"

[1732] (Claim 1)

[1733] a means for selecting relevant video content based on user-entered interests and challenges;

[1734] means for summarizing the content of the selected video content;

[1735] a means for generating a specific action plan based on the summarized content;

[1736] a means for providing the generated action plan to a user;

[1737] means for recognizing the emotional state of a user using emotion recognition technology;

[1738] A means for selecting appropriate video content based on the emotional state of a user;

[1739] A system including:

[1740] (Claim 2)

[1741] 10. The system of claim 1, further comprising means for receiving input of interests and challenges from a user.

[1742] (Claim 3)

[1743] 10. The system of claim 1, wherein natural language processing techniques are used to summarize the content of the video content and generate a specific plan of action based on key points contained in the summary. [Explanation of symbols]

[1744] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for selecting relevant lecture videos based on the user's inputted interests and challenges; A means for summarizing the content of selected lecture videos; A means for generating a concrete action plan based on the summarized content; a means for providing the generated action plan to the user; A system including:

2. The system of claim 1 further comprising means for receiving input of interests and challenges from a user.

3. The system of claim 1 , which uses natural language processing technology to summarize the content of a lecture video and generate a specific action plan based on the main points contained in the summary.

Images