system

Abstract

Provide a system. A means for receiving game video and audio data; an artificial intelligence engine means for analyzing the received data; a means for generating personalized advice based on the analysis results; means for providing the generated advice to a user; A chatbot means that always accepts inquiries from users, Provide self-assessment sheets and questionnaires and analyze the results; A means for generating a message to improve self-esteem based on the analysis result and providing the message to the user; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Mental health has a significant impact on performance in sports, business, academics, and other areas, but a lack of experts and awareness remains a challenge. In particular, it is currently difficult to receive mental support regardless of time or place. It is also difficult to provide personalized advice tailored to individual needs, and there is a lack of means to sustainably improve users' self-evaluation and self-esteem. To address these issues, a system that can provide efficient and continuous mental support is needed. [Means for solving the problem]

[0005] The present invention is a system that generates personalized advice using an artificial intelligence engine that receives and analyzes game video and audio data. Specifically, the system analyzes a user's performance based on the received data and provides advice tailored to the user's needs based on past data and current analysis results. It also includes a chatbot that is always available to answer questions from users, enabling 24 / 7 support. Furthermore, the system provides self-evaluation sheets and questionnaires, analyzes the results, generates messages to improve self-esteem, and provides these to users, thereby building mental toughness. This system allows users to receive efficient and continuous mental support regardless of time or location, and is expected to improve performance.

[0006] "Game footage" is video data that a user captures during a competition or game.

[0007] "Audio data" refers to audio data recorded by a user during a competition or game.

[0008] The "receiving means" is a part that has the function of receiving game video and audio data sent from the user.

[0009] The "artificial intelligence engine" is the part that has the function of analyzing the received data and extracting performance metrics and operational details.

[0010] "Personalized advice" means advice or suggestions that are individually generated based on a user's specific circumstances and performance data.

[0011] A "user" is an individual who uses the system, such as an athlete, business person, or student.

[0012] A "chatbot" is a program that uses artificial intelligence to respond in real time to users' questions and inquiries about mental health.

[0013] A "self-evaluation sheet" is a questionnaire-style document that allows users to evaluate their own performance and mental state.

[0014] A "message to improve self-esteem" is positive feedback or advice provided to a user to improve their self-esteem based on the results of their self-evaluation.

[0015] The "internal database" is a data storage system for recording and saving received data, analysis results, chat history, etc. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention relates to an AI mental trainer system aimed at improving mental health. It receives and analyzes game video and audio data and provides personalized advice. This system allows users to receive efficient and continuous mental support regardless of time or place.

[0038] Program processing explanation

[0039] 1. Data Collection

[0040] User:

[0041] Users use their smartphones or PCs to upload their game video and audio data to the application, providing the system with data that can be later analyzed.

[0042] Device:

[0043] The terminal receives video and audio data from the user and transmits it to the server.

[0044] server:

[0045] The server stores the received data and prepares it for the next analysis process.

[0046] 2. Data Analysis

[0047] server:

[0048] The received game video and audio data is passed to an AI analysis engine, which extracts the user's movements, success rate, and performance metrics, such as shot success rate and running distance.

[0049] server:

[0050] The analysis results are stored in a database and managed for each user.

[0051] 3. Generating personalized advice

[0052] server:

[0053] It generates personalized advice based on the user's past data and current analysis results, including specific practice methods and mental advice.

[0054] server:

[0055] The generated advice is transmitted to the user terminal.

[0056] Device:

[0057] Display the received advice to the user within the application.

[0058] 4. 24 / 7 support

[0059] User:

[0060] Users can access the AI ​​chatbot within the application when they feel anxious before a match or during a crucial moment.

[0061] Device:

[0062] Start the chatbot function and connect to the server.

[0063] server:

[0064] Receives user questions and generates answers by referencing past data and knowledge bases.

[0065] server:

[0066] Answers to questions and advice are sent to the device in real time.

[0067] Device:

[0068] The received response is displayed to the user.

[0069] 5. Improved self-esteem and self-confidence

[0070] server:

[0071] Self-evaluation sheets and questionnaires are generated periodically and sent to the user's terminal.

[0072] Device:

[0073] The received self-evaluation sheet is displayed to the user, and an interface is provided to prompt the user to respond.

[0074] User:

[0075] The user answers the self-evaluation sheet and sends the data from the terminal to the server.

[0076] server:

[0077] The received data is analyzed and messages and advice are generated to improve self-esteem.

[0078] server:

[0079] The generated message or advice is sent to the user terminal.

[0080] Device:

[0081] The received messages and advice are displayed to the user to improve self-evaluation and self-affirmation.

[0082] Specific examples

[0083] 1. Example of Athlete A

[0084] Data collection:

[0085] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[0086] Data Analysis:

[0087] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[0088] Providing personalized advice:

[0089] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[0090] 24 / 7 support:

[0091] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data to provide advice on "strategies that were successful in the previous match."

[0092] Increased self-esteem and self-esteem:

[0093] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[0094] In this way, the system of the present invention provides efficient and continuous mental support to users, thereby improving their performance.

[0095] The processing flow will be explained below.

[0096] Data collection and analysis

[0097] Step 1:

[0098] Users use their smartphones or PCs to record video and audio data of their own matches.

[0099] Step 2:

[0100] Users upload the captured game footage and audio data to the server via an application on their device.

[0101] Step 3:

[0102] The device receives the uploaded video and audio data and sends it to the server.

[0103] Step 4:

[0104] The server stores the received video and audio data and prepares it for the next analysis process.

[0105] Step 5:

[0106] The server passes the video data to an AI analytics engine, which analyzes the data and extracts user behavior, success rate, and performance metrics.

[0107] Step 6:

[0108] The server stores the analysis results in an internal database and manages them for each user.

[0109] Generate personalized advice

[0110] Step 1:

[0111] The server prepares data to generate personalized advice based on the user's past performance data and current analysis results.

[0112] Step 2:

[0113] The server's AI engine analyzes the collected data and generates practice methods and mental advice tailored to the user.

[0114] Step 3:

[0115] The server transmits the generated advice to the user's terminal.

[0116] Step 4:

[0117] The terminal displays the received advice to the user within the application.

[0118] Step 5:

[0119] The user checks the displayed advice and prepares to proceed to the next phase.

[0120] 24 / 7 support

[0121] Step 1:

[0122] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[0123] Step 2:

[0124] The terminal displays the chatbot screen and connects to the server.

[0125] Step 3:

[0126] Users input questions or inquiries to the chatbot.

[0127] Step 4:

[0128] The server receives questions and inquiries from users and generates answers by referencing past data and a knowledge base.

[0129] Step 5:

[0130] The server sends the generated answers and advice to the user's terminal.

[0131] Step 6:

[0132] The terminal displays the received answers and advice to the user in real time.

[0133] Step 7:

[0134] The server stores the chat history with the user in a database, which can be used for future counseling.

[0135] Improved self-esteem and self-esteem

[0136] Step 1:

[0137] The server periodically generates self-evaluation sheets and questionnaires and sends them to the user's terminal.

[0138] Step 2:

[0139] The terminal displays the received self-evaluation sheet to the user and provides an interface that prompts the user to answer.

[0140] Step 3:

[0141] The user answers the self-evaluation sheet and transmits the data from the terminal to the server.

[0142] Step 4:

[0143] The server receives the data from the self-assessment sheet, and the AI ​​engine performs the analysis.

[0144] Step 5:

[0145] Based on the analysis results, the server generates positive messages and advice to improve self-esteem.

[0146] Step 6:

[0147] The server sends the generated message or advice to the user's terminal.

[0148] Step 7:

[0149] The device displays the received messages and advice to the user, encouraging them to improve their self-esteem.

[0150] Example 1

[0151] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0152] Conventional mental training systems have had issues with insufficient collection and analysis of user performance data, making it difficult to provide personalized advice. Furthermore, they lacked the functionality to accept consultations from users in real time 24 hours a day, 365 days a year, leaving insufficient means for improving self-evaluation and self-affirmation.

[0153] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0154] In this invention, the server includes: means for receiving game video and audio data; artificial intelligence engine means for analyzing the received data; means for generating personalized advice based on the analysis results; means for providing the generated advice to the user; chatbot means for always accepting consultations from users; means for providing self-assessment sheets and questionnaires and analyzing the results; means for generating messages to improve self-esteem based on the analysis results and providing them to the user; means for the user to upload game video and audio data using the terminal; means for saving the data and preparing for the next analysis process; means for passing the data to the AI ​​analysis engine and extracting performance metrics; means for saving the performance metrics in a database and managing them for each user; means for sending the generated advice to the user terminal in real time; means for displaying the received advice on the user terminal; means for activating the AI ​​chatbot function and connecting to the server; means for using an artificial intelligence model to generate optimal answers to the user's questions and sending them to the terminal; means for the user to access the chatbot when they feel anxious before a game; and means for generating self-assessment sheets and questionnaires for the user and sending them to the terminal. This enables detailed analysis of users' performance data and the provision of efficient and personalized advice, providing real-time mental support and improving self-evaluation 24 hours a day, 365 days a year.

[0155] "Game video and audio data" refers to video and audio data recorded by a user during a sport or activity.

[0156] The "receiving means" refers to a function or module for acquiring data transmitted from a user terminal.

[0157] The "analytical artificial intelligence engine means" is a system that analyzes received data using deep learning or machine learning to extract important performance metrics.

[0158] A "means for generating personalized advice" is an algorithm or module for providing advice tailored to the individual needs of each user.

[0159] The "means for providing advice to the user" is a system for displaying and notifying the generated advice on the user's terminal.

[0160] A "chatbot" is a conversation system that uses artificial intelligence to accept questions and inquiries from users and automatically respond.

[0161] "Means for providing self-evaluation sheets and questionnaires" refers to a system that provides questions and forms for users to request self-evaluation and feedback.

[0162] The "means for analyzing the results" refers to algorithms and modules for analyzing the user's self-evaluation sheets and questionnaire responses.

[0163] The "means for generating messages to improve self-esteem" is a system that generates messages to improve a user's self-esteem and motivation based on the results of self-evaluation.

[0164] "Means for uploading" refers to the means by which a user sends game video and audio data to the system, typically via an application or web portal.

[0165] The "means for storing data" is a storage system for safely and efficiently storing received game video and audio data.

[0166] "Means for preparing for analysis processing" refers to a system that performs the necessary data preprocessing and formatting before analyzing the data.

[0167] "Means for extracting performance metrics" refers to a function that extracts important indicators of athletic performance (e.g., shooting success rate, distance traveled, etc.) from the analyzed data.

[0168] "Means for managing performance metrics" refers to a system that organizes, stores, and manages extracted indicators for each user.

[0169] The "means for transmitting in real time" is a communication system for instantly transmitting the generated advice or message to the user's terminal.

[0170] "Means for activating AI chatbot functions" refers to a system that activates chatbot functions in response to user requests.

[0171] "Means for generating optimal answers using an artificial intelligence model" refers to an AI module that generates optimal answers to user questions by referencing past data and knowledge bases.

[0172] The "means for transmitting to the terminal" is a system that transmits the generated answers and advice to the user's terminal.

[0173] "Means to access a chatbot when feeling anxious" is a function that allows users to access a chatbot and consult with it when they feel psychological anxiety.

[0174] "Means for generating self-assessment sheets and questionnaires" refers to a system that creates questions and forms for users to periodically conduct self-assessments.

[0175] The program in this system is an AI mental trainer system designed to improve the user's mental health. This system receives and analyzes game video and audio data and provides personalized advice. Specific embodiments of this system are described below.

[0176] Data collection

[0177] User:

[0178] Users can upload their game video and audio data to the application using their smartphone or PC. For example, they can send video files saved on their smartphone via the app's upload function.

[0179] Device:

[0180] The device (user's smartphone or PC) receives the uploaded video and audio data and sends it to the server. The data is designed to be sent to the server via a network connection.

[0181] server:

[0182] The server stores the received video and audio data in a database and prepares it for the next analysis process. The stored data is used for analysis by the AI ​​analysis engine, which will be described later.

[0183] Data analysis

[0184] server:

[0185] The server passes the received data to an AI analysis engine, such as Tensorflow (registered trademark) or PyTorch, which utilizes deep learning technology. This extracts performance metrics (e.g., shooting percentage, distance traveled), and stores this data in a database.

[0186] server:

[0187] The analysis results are managed for each user and used for subsequent processing as needed.

[0188] Generate personalized advice

[0189] server:

[0190] The server uses a generative AI model to generate personalized advice based on the user's past data and analysis results. For example, it uses a machine learning model to generate "five practice methods to improve shooting accuracy."

[0191] server:

[0192] The generated advice is sent to the user terminal in real time via network communication.

[0193] Device:

[0194] The device then displays the received advice to the user within the application. The advice can be provided not only as text information, but also as video or audio guides.

[0195] 24 / 7 support

[0196] User:

[0197] Before a match or when feeling stressed, users can access the AI ​​chatbot within the app by opening the app's chatbot function and typing in a question.

[0198] Device:

[0199] The chatbot function is activated and connected to the server, where real-time communication takes place.

[0200] server:

[0201] When a user asks a question, it references past data and a knowledge base to generate the best answer, using natural language processing (NLP) models and sometimes generative AI models such as GPT-3 (registered trademark).

[0202] server:

[0203] The generated answer is sent to the user terminal.

[0204] Device:

[0205] The terminal displays the received answers to the user, and responds quickly to the user's concerns and questions.

[0206] Improved self-esteem and self-esteem

[0207] server:

[0208] Self-assessment sheets and questionnaires are periodically generated and sent to the user's device. This process also uses machine learning models to generate questions that are appropriate for the user.

[0209] Device:

[0210] The device displays the received self-assessment sheet to the user within the app, providing an interface that allows the user to easily respond.

[0211] User:

[0212] The user answers the self-evaluation sheet and sends the data from the terminal to the server. All the user has to do is enter the details of their self-evaluation and press the send button.

[0213] server:

[0214] The system analyzes the received evaluation data and generates messages and advice to improve the user's self-esteem. For example, it generates positive messages such as "Your performance has improved dramatically this month! Keep it up!"

[0215] server:

[0216] The generated message or advice is sent to the user terminal.

[0217] Device:

[0218] The device will display the received messages and advice to the user, aiming to improve their motivation and self-esteem. Notifications will be sent in the form of push notifications or in-app messages.

[0219] Specific examples

[0220] 1. Example of Athlete A

[0221] Data collection:

[0222] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[0223] Data Analysis:

[0224] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[0225] Providing personalized advice:

[0226] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[0227] 24 / 7 support:

[0228] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data to provide advice on "strategies that were successful in the previous match."

[0229] Increased self-esteem and self-esteem:

[0230] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[0231] Prompt Sentence Examples

[0232] "What is your shooting percentage in last month's games?"

[0233] "What do you need to improve in practice this week?"

[0234] "How can I reduce the anxiety I feel before a match?"

[0235] In this way, the system of the present invention provides efficient and continuous mental support to users, thereby improving their performance.

[0236] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0237] Step 1:

[0238] User:

[0239] Users use their smartphones or PCs to upload their own game video and audio data to the application by using the app's upload function to select the data and then pressing the send button.

[0240] Input: Game video and audio data

[0241] Output: The data file received by the application

[0242] Step 2:

[0243] Device:

[0244] The device (user's smartphone or PC) receives the uploaded video and audio data and sends it to the server via the Internet. A network connection must be established.

[0245] Input: User uploaded data

[0246] Output: Data packet sent to the server

[0247] Step 3:

[0248] server:

[0249] The server stores the received video and audio data in a database. Before storing the data, it checks it to ensure that it is not corrupted or invalid.

[0250] Input: Data sent from the terminal

[0251] Output: Data stored in the database

[0252] Step 4:

[0253] server:

[0254] The stored data is passed to an AI analytics engine for analysis, which uses deep learning techniques (such as TensorFlow or PyTorch) to extract performance metrics.

[0255] Input: Video and audio data stored in a database

[0256] Output: Extracted performance metrics (e.g., shooting percentage, distance traveled)

[0257] Step 5:

[0258] server:

[0259] The analysis results are saved in a database and managed for each user. The saved data can also be used as user history data.

[0260] Input: Performance metrics

[0261] Output: Analysis results stored in a database

[0262] Step 6:

[0263] server:

[0264] Using a generative AI model, the system generates personalized advice based on the user's past data and the latest analysis results. For example, if a user's shooting success rate is low, the system generates "practice methods to improve shooting accuracy."

[0265] Input: Historical data and latest analysis results

[0266] Output: Personalized advice

[0267] Step 7:

[0268] server:

[0269] The generated advice is sent to the user's device via the network, and notifications are also sent in real time.

[0270] Enter: personalized advice

[0271] Output: Advice sent to the user's terminal

[0272] Step 8:

[0273] Device:

[0274] The device displays the received advice to the user within the application. Advice may be provided not only as text information, but also as video or audio guides.

[0275] Input: Advice sent by the server

[0276] Output: Advice displayed to the user

[0277] Step 9:

[0278] User:

[0279] Before a match or when feeling stressed, users can access the AI ​​chatbot within the application and input their questions or concerns, and receive support in an interactive format using the chatbot's functionality.

[0280] Input: User questions and inquiries

[0281] Output: The text entered into the chatbot

[0282] Step 10:

[0283] Device:

[0284] The chatbot function is activated and the entered questions or inquiries are sent to the server. Since the connection is real-time, a low-latency network environment is recommended.

[0285] Input: User questions and inquiries

[0286] Output: Questions and queries sent to the server

[0287] Step 11:

[0288] server:

[0289] It uses past data and knowledge bases to generate optimal answers using artificial intelligence models (e.g., GPT-3). Depending on the question, it may refer to past match data or analysis results.

[0290] Input: User questions and knowledge base

[0291] Output: The generated answer

[0292] Step 12:

[0293] server:

[0294] The generated answer is sent to the user's device in real time, and the user is immediately notified via a network connection.

[0295] Input: Generated answer

[0296] Output: Answer sent to user's terminal

[0297] Step 13:

[0298] Device:

[0299] The device then displays the received answers to the user, who can use them to solve problems and reduce stress.

[0300] Input: The answer sent by the server

[0301] Output: The answer that is displayed to the user

[0302] Step 14:

[0303] server:

[0304] Self-assessment sheets and questionnaires are generated periodically and sent to the user's device. Questions appropriate for the user are selected using a machine learning model.

[0305] Input: User history and rating data

[0306] Output: Generated self-assessment sheets and questionnaires

[0307] Step 15:

[0308] Device:

[0309] It provides an interface that displays self-evaluation sheets and questionnaires to users and encourages them to respond, allowing users to input their answers intuitively.

[0310] Input: Self-assessment sheets and surveys sent from the server

[0311] Output: Questions displayed to the user

[0312] Step 16:

[0313] User:

[0314] Users answer a self-assessment form and the data is sent from their device to a server, which may be designed to maintain anonymity.

[0315] Input: User's answer

[0316] Output: Response data sent to the server

[0317] Step 17:

[0318] server:

[0319] The system analyzes the received self-evaluation data and generates messages and advice to improve the user's self-esteem, placing emphasis on positive feedback.

[0320] Input: User response data

[0321] Output: any messages or advice generated

[0322] Step 18:

[0323] server:

[0324] The generated messages and advice are sent to the user's terminal in real time, and the user is notified immediately.

[0325] Input: Generated messages and advice

[0326] Output: Messages and advice sent to the user's terminal

[0327] Step 19:

[0328] Device:

[0329] The device will display the received messages and advice to the user. Messages will be sent in the form of push notifications or in-app messages.

[0330] Input: Messages and advice sent by the server

[0331] Output: Messages and advice displayed to the user

[0332] In this way, the system of the present invention provides efficient and continuous mental support to the user through the specific actions at each step, thereby improving performance.

[0333] (Application example 1)

[0334] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0335] In today's food delivery industry, delivery workers are experiencing increasing stress and anxiety during their daily work, which can lead to a decline in work efficiency and mental health. Furthermore, the lack of personalized support tailored to each delivery worker means that improvements in mental health and performance are not being linked. Given this background, there is a need for a system that provides efficient and continuous mental support to delivery workers, balancing work efficiency and mental health.

[0336] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0337] In this invention, the server includes means for receiving game video and audio data, artificial intelligence engine means for analyzing the received data, means for generating personalized advice based on the analysis results, means for providing the generated advice to the user, chatbot means for constantly accepting inquiries from the user, means for providing self-evaluation sheets and questionnaires and analyzing the results, means for generating messages to improve self-esteem based on the analysis results and providing them to the user, means for recording guidance and voice memos for delivery personnel during deliveries, means for analyzing the recorded data to extract delivery performance and stress factors, and means for generating advice for work improvement and mental support based on the extraction results. This allows for efficient and continuous mental support to be provided to delivery personnel, enabling improved work efficiency and improved mental health.

[0338] The "means for receiving game video and audio data" refers to a device or system for receiving video and audio recorded by a user.

[0339] The "artificial intelligence engine means for analyzing received data" is an analytical engine that uses artificial intelligence to analyze received video and audio data and extract useful information.

[0340] The "means for generating personalized advice based on the analysis results" refers to a device or system for customizing and generating optimal advice for a user based on the analyzed data.

[0341] The "means for providing generated advice to a user" is a device or system for providing customized advice to a user.

[0342] The "chatbot means that always accepts inquiries from users" is a chatbot that always accepts questions and inquiries from users and automatically responds to them.

[0343] The "means for providing a self-assessment sheet or questionnaire and analyzing the results" refers to a device or system for providing a self-assessment sheet or questionnaire to users and analyzing the response results.

[0344] The "means for generating and providing a user with a message that will improve self-esteem based on the analysis results" refers to a device or system for generating and providing a user with a message that will improve self-esteem based on the analyzed evaluation data.

[0345] The "means for the delivery person to record guidance and voice memos during delivery" refers to a device or system for recording guidance and voice memos during delivery.

[0346] The "means for analyzing recorded data to extract delivery performance and stress factors" refers to a device or system for analyzing recorded guidance and voice memos to identify delivery performance and stress factors.

[0347] The "means for generating advice for business improvement and mental support based on the extraction results" refers to a device or system for generating advice for business improvement and mental support based on the analyzed data.

[0348] As an embodiment of the present invention, a system for improving the mental health of food delivery workers is constructed as follows.

[0349] 1. Data Collection:

[0350] User: Delivery personnel use their smartphones to record guidance and voice memos during deliveries, which allows data on daily work and delivery situations to be accumulated.

[0351] Device: The smartphone transmits the recorded data to the server. The smartphone is equipped with recording and data transmission functions.

[0352] 2. Data Analysis:

[0353] Server: Uses an AI analysis engine to analyze the received guidance and voice data. The analysis engine identifies delivery performance (e.g., delivery time, success rate) and stress factors (e.g., stress word analysis in the voice).

[0354] The specific software used includes voice recognition technology and machine learning models, such as the Google® Cloud Speech-to-Text API and TensorFlow models.

[0355] 3. Advice Generation:

[0356] Server: Generates personalized advice based on the analysis results, including specific suggestions for work improvement and mental support.

[0357] Device: Advice is sent to the smartphone and displayed to the user using notifications and dashboard features.

[0358] 4. Real-time support:

[0359] Users: When they feel anxious or stressed during a delivery, they can use the in-app chatbot for advice.

[0360] Terminal: The chatbot function is activated and connected to the server.

[0361] Server: The chatbot references historical data and a knowledge base to provide real-time support.

[0362] 5. Self-assessment and feedback:

[0363] Server: Periodically generates self-evaluation sheets and questionnaires and sends them to the delivery person's smartphone.

[0364] Terminal: The delivery person answers a self-evaluation sheet and sends the data to the server.

[0365] Server: Based on the analyzed data, it generates and sends a self-esteem-boosting message to the delivery person, for example, providing feedback such as "Your stress level has improved by 30% over the past month."

[0366] For example, when delivery person A drives a new route, he can check his smartphone for advice on how to relieve stress and how to deliver efficiently. Based on past data, the AI ​​chatbot will respond by saying, "Since you were able to reduce the time by 15 minutes on your last route, we'll suggest ways to reduce stress this time as well."

[0367] Example prompt sentence:

[0368] "Analyze delivery drivers' voice guidance and provide personalized advice on stress factors and how to improve their work."

[0369] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0370] Step 1:

[0371] Data collection:

[0372] Input: Guidance and voice memos recorded by delivery personnel using their smartphones during deliveries

[0373] Specific behavior:

[0374] User: While making a delivery, the delivery person uses the recording function on their smartphone to record the immediate situation, thoughts, and emotions while driving as voice memos.

[0375] Device: The smartphone temporarily stores the recorded audio data and sends it to the server using mobile data.

[0376] Output: Audio data sent to the server

[0377] Step 2:

[0378] Data received and stored on the server:

[0379] Input: Audio data from the device

[0380] Specific behavior:

[0381] Server: Receives the voice data and stores it in a database, tagged with the date and delivery person ID.

[0382] Output: Saved audio data

[0383] Step 3:

[0384] Data Analysis:

[0385] Input: Stored audio data

[0386] Specific behavior:

[0387] Server: Passes the voice data to an AI analytics engine, which uses speech recognition technology to convert it into text and extracts stress factors and performance metrics, such as detecting specific keywords and phrases that indicate stress, and calculating delivery times and delivery success rates.

[0388] Tools used: Google Cloud Speech-to-Text API and TensorFlow model

[0389] Output: Analysis results (stress factors, delivery performance metrics)

[0390] Step 4:

[0391] Generate personalized advice:

[0392] Input: Analysis results

[0393] Specific behavior:

[0394] Server: Based on the analysis results, it generates advice for business improvement and mental support. For example, it generates specific advice such as "Based on the data from the past month, it would be better to choose off-peak times to further shorten delivery times."

[0395] Output: Personalized advice

[0396] Step 5:

[0397] Providing advice:

[0398] Enter: personalized advice

[0399] Specific behavior:

[0400] Server: Sends the generated advice to the delivery person's smartphone.

[0401] Device: The smartphone displays the received advice to the user via the notification function.

[0402] Output: Advice displayed on the delivery person's smartphone

[0403] Step 6:

[0404] Real-time support:

[0405] Input: Message from delivery person

[0406] Specific behavior:

[0407] User: If they have any concerns or questions during delivery, they can send a consultation message using the chatbot function on their smartphone.

[0408] Terminal: The smartphone sends a consultation message to the server.

[0409] Server: The chatbot references historical data and a knowledge base to generate real-time responses, such as "Your last route saved you 15 minutes, so try a similar route this time."

[0410] Output: Real-time response from the chatbot

[0411] Step 7:

[0412] Self-assessment and feedback:

[0413] Input: Self-evaluation sheet from the server, delivery person's evaluation answers

[0414] Specific behavior:

[0415] Server: Periodically sends self-evaluation sheets and surveys to delivery workers' smartphones, including questions about stress levels and work performance.

[0416] User: Answer each assessment item and submit the data from their smartphone.

[0417] Server: Analyzes the received evaluation data and generates a feedback message, such as providing positive feedback like "Your stress level has improved by 30% over the past month."

[0418] Output: Feedback message displayed on the delivery person's smartphone

[0419] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0420] This invention relates to an AI mental trainer system aimed at improving mental health. It receives and analyzes game video and audio data to provide personalized advice, and also incorporates an emotion engine that recognizes the user's emotions. This makes it possible to provide more appropriate mental support based on the user's needs and emotional state.

[0421] Program processing explanation

[0422] 1. Data Collection

[0423] User:

[0424] Users use their smartphones or PCs to upload their game video and audio data to the application, providing the system with data that can be later analyzed.

[0425] Device:

[0426] The terminal receives video and audio data from the user and transmits it to the server.

[0427] server:

[0428] The server stores the received data and prepares it for the next analysis process.

[0429] 2. Data Analysis

[0430] server:

[0431] The received game video and audio data is passed to an AI analysis engine, which extracts user actions, success rates, and performance metrics.

[0432] server:

[0433] The analysis results are stored in a database and managed for each user.

[0434] 3. Emotion analysis

[0435] server:

[0436] The received video and audio data is passed to the emotion engine, which analyzes the user's facial expressions and voice to determine their emotional state.

[0437] server:

[0438] Based on the analysis results of the emotion engine, data is prepared to generate more appropriate advice.

[0439] 4. Generating personalized advice

[0440] server:

[0441] It generates personalized advice based on the user's past performance data, current analysis results, and emotion engine analysis results.

[0442] server:

[0443] The generated advice is sent to the user's terminal.

[0444] Device:

[0445] Display the received advice to the user within the application.

[0446] 5. 24 / 7 support

[0447] User:

[0448] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[0449] Device:

[0450] Start the chatbot function and connect to the server.

[0451] server:

[0452] Receives a user question and generates an answer by referencing the emotion engine, past data, and knowledge base.

[0453] server:

[0454] Answers to questions and advice are sent to the user's device.

[0455] Device:

[0456] The received answers are displayed to the user in real time.

[0457] server:

[0458] The chat history with the user is saved in a database, which can be used for future counseling.

[0459] 6. Improved self-esteem and self-confidence

[0460] server:

[0461] Self-evaluation sheets and questionnaires are generated periodically and sent to the user's device.

[0462] Device:

[0463] The received self-evaluation sheet is displayed to the user, and an interface is provided to prompt the user to respond.

[0464] User:

[0465] The user answers the self-evaluation sheet and sends the data from the terminal to the server.

[0466] server:

[0467] The received data is analyzed and messages and advice are generated to improve self-esteem.

[0468] server:

[0469] The generated message or advice is sent to the user's terminal.

[0470] Device:

[0471] The received messages and advice are displayed to the user, encouraging them to improve their self-esteem.

[0472] Specific examples

[0473] 1. Example of Athlete A

[0474] Data collection:

[0475] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[0476] Data Analysis:

[0477] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[0478] Emotion analysis:

[0479] The server passes the video data to the emotion engine, which identifies pressure or anxiety from Player A's facial expressions and voice. For example, it analyzes that Player A is nervous before a match.

[0480] Providing personalized advice:

[0481] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[0482] 24 / 7 support:

[0483] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data and sentiment analysis results to provide advice on "strategies that were successful in the previous match."

[0484] Increased self-esteem and self-esteem:

[0485] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[0486] In this way, the system of the present invention provides users with efficient and continuous mental support, improving their performance. By adding an emotion engine, it becomes possible to provide more appropriate advice and support according to the user's emotional state.

[0487] The processing flow will be explained below.

[0488] MODE FOR CARRYING OUT THE INVENTION (DETAILED PROCESSING FLOW)

[0489] Data collection

[0490] Step 1:

[0491] Users use their smartphones or PCs to record video and audio data of their own matches.

[0492] Step 2:

[0493] Users upload the captured game footage and audio data to the server via an application on their device.

[0494] Step 3:

[0495] The device receives the uploaded video and audio data and sends it to the server.

[0496] Step 4:

[0497] The server stores the received video and audio data and prepares it for the next analysis process.

[0498] Data analysis

[0499] Step 5:

[0500] The server passes the video data to an AI analytics engine, which analyzes the data and extracts user behavior, success rate, and performance metrics.

[0501] Step 6:

[0502] The server stores the analysis results in an internal database and manages them for each user.

[0503] Emotion analysis

[0504] Step 7:

[0505] The server passes the video and audio data to the emotion engine, which analyzes the user's facial expressions and voice to determine their emotional state.

[0506] Step 8:

[0507] The server prepares data to generate more appropriate advice based on the analysis results of the emotion engine.

[0508] Generate personalized advice

[0509] Step 9:

[0510] The server generates personalized advice based on the user's past performance data, current analysis results, and the analysis results of the emotion engine.

[0511] Step 10:

[0512] The server transmits the generated advice to the user's terminal.

[0513] Step 11:

[0514] The terminal displays the received advice to the user within the application.

[0515] Step 12:

[0516] The user checks the displayed advice and prepares to proceed to the next phase.

[0517] 24 / 7 support

[0518] Step 13:

[0519] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[0520] Step 14:

[0521] The terminal displays the chatbot screen and connects to the server.

[0522] Step 15:

[0523] Users input questions or inquiries to the chatbot.

[0524] Step 16:

[0525] The server receives questions and inquiries from users, references past data, emotion engine results, and a knowledge base, and generates answers.

[0526] Step 17:

[0527] The server sends the generated answers and advice to the user's terminal.

[0528] Step 18:

[0529] The terminal displays the received answers and advice to the user in real time.

[0530] Step 19:

[0531] The server stores the chat history with the user in a database, which can be used for future counseling.

[0532] Improved self-esteem and self-esteem

[0533] Step 20:

[0534] The server periodically generates self-evaluation sheets and questionnaires and sends them to the user's terminal.

[0535] Step 21:

[0536] The terminal displays the received self-evaluation sheet to the user and provides an interface that prompts the user to answer.

[0537] Step 22:

[0538] The user answers the self-evaluation sheet and transmits the data from the terminal to the server.

[0539] Step 23:

[0540] The server receives the data from the self-assessment sheet, and the AI ​​engine performs the analysis.

[0541] Step 24:

[0542] Based on the analysis results, the server generates positive messages and advice to improve self-esteem.

[0543] Step 25:

[0544] The server sends the generated message or advice to the user's terminal.

[0545] Step 26:

[0546] The device displays the received messages and advice to the user, encouraging them to improve their self-esteem.

[0547] Specific examples

[0548] Example of Athlete A

[0549] Step 1:

[0550] Athlete A uploads game footage from his smartphone to the app.

[0551] Step 2:

[0552] The terminal receives the video and transmits it to the server.

[0553] Step 3:

[0554] The server receives the video and passes it to the AI ​​analysis engine.

[0555] Step 4:

[0556] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[0557] Step 5:

[0558] The server passes the video data to the emotion engine, which identifies pressure or anxiety from Player A's facial expressions and voice. For example, it analyzes that Player A is nervous before a match.

[0559] Step 6:

[0560] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[0561] Step 7:

[0562] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data and sentiment analysis results to provide advice on "strategies that were successful in the previous match."

[0563] Step 8:

[0564] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[0565] In this way, the system of the present invention, which is combined with an emotion engine, provides efficient and continuous mental support to the user, and realizes more appropriate advice and support according to the user's emotional state.

[0566] Example 2

[0567] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0568] Conventional mental care systems have difficulty analyzing users' emotional states and performance in real time and providing appropriate, personalized advice. Furthermore, there has been a lack of systems that provide comprehensive, continuous support for 24 / 7 support and for improving self-esteem based on self-evaluation. Therefore, there is a need for an effective system that can sustainably improve users' mental health and performance.

[0569] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0570] In this invention, the server includes a device for receiving game video and audio data, an intelligence engine for analyzing the received data, a device for generating personalized advice based on the analysis results, a device for providing the generated advice to users, a conversational bot device for constantly accepting consultations from users, an engine for analyzing emotional states, a device for providing self-evaluation sheets and questionnaires and analyzing the results, and a device for generating messages to improve positivity based on the analysis results and providing them to users. This makes it possible to analyze users' emotional states and performance in real time and provide personalized, appropriate advice. It also provides comprehensive and continuous support 24 hours a day, 365 days a year, and helps improve self-esteem based on self-evaluation.

[0571] The "device for receiving game video and audio data" is a device for electronically receiving game video and audio data provided by a user.

[0572] The "intelligence engine" is an analytical device that uses artificial intelligence to analyze received game video and audio data.

[0573] The "device for generating personalized advice" is a device that generates advice optimized for each user based on the analysis results.

[0574] The "device for providing generated advice to a user" is a device for providing generated advice to a user in a format that is easy to use.

[0575] A "conversation bot device" is an automatic response device that accepts inquiries from users 24 hours a day, 365 days a year, and provides answers in an interactive format.

[0576] The "emotional state analysis engine" is a device for analyzing the user's emotional state from received video and audio data.

[0577] The "device for providing a self-assessment sheet or questionnaire and analyzing the results" is a device that provides a self-assessment sheet or questionnaire to a user and analyzes the user's responses.

[0578] The "device for generating and providing users with messages that enhance positive self-esteem" is a device for generating and providing users with messages that enhance positive self-esteem based on the analysis results of self-evaluations and questionnaires.

[0579] This invention relates to an AI mental trainer system aimed at improving mental health. It receives and analyzes various data and provides personalized advice. It also incorporates an emotion engine that analyzes the user's emotional state. Below, we explain how this system is specifically implemented.

[0580] First, the system consists of the following main components:

[0581] 1. Data Collection Equipment

[0582] User: Uses a smartphone or PC to upload video and audio data from their match to a dedicated app.

[0583] Terminal: Receives data, compresses it if necessary, and sends it to the server. This process uses software that recognizes and processes specific data formats, such as MP4 for video data and WAV for audio data.

[0584] 2. Intelligence Engine

[0585] Server: Receives game footage and audio data stored in the database and analyzes the movement and audio characteristics using analysis tools such as "OpenPose" and "Librosa." The analysis results include the user's movement success rate and level of tension.

[0586] 3. Emotional state analysis engine

[0587] Server: Analyzes video and audio data using emotion analysis tools such as "FER2013" and "OpenSmile," and identifies emotions from the user's facial expressions and tone of voice.

[0588] 4. Device for generating personalized advice

[0589] Server: Based on the analysis results, the generative AI model "GPT-4 (registered trademark)" is used to generate advice tailored to each user. For example, it provides practice methods to improve the user's shooting success rate or breathing techniques to help them relax.

[0590] 5. Device for Providing Generated Advice to Users

[0591] Server: Generates advice and sends it to the user's device.

[0592] Terminal: Receives the advice and displays it to the user within the application.

[0593] 6. Conversational Bot Device

[0594] Users: Use the chatbot feature within the application to ask questions before a game, after practice, etc.

[0595] Terminal: Receives the query and sends it to the server.

[0596] Server: Refers to the emotion engine and knowledge base to generate appropriate answers and send them to the user.

[0597] Terminal: Shows the answer to the user in real time.

[0598] 7. Self-evaluation sheets and questionnaires providing device

[0599] Server: Periodically generates self-assessment sheets and questionnaires and sends them to the user's device.

[0600] Terminal: Displays the received self-assessment sheet or questionnaire to the user and prompts them to respond.

[0601] User: Answers and sends the data from the device to the server.

[0602] Server: Analyzes the received data, generates messages and advice to improve positive feelings, and provides them to the user.

[0603] Specific examples

[0604] The following explanation will be given using athlete A as an example.

[0605] 1. Data collection: Athlete A uploads game footage from his smartphone to the app. The device receives the footage and sends it to the server. The server then passes the footage to the AI ​​analysis engine.

[0606] 2. Data analysis: The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. This analysis result is stored in a database.

[0607] 3. Emotion analysis: The server passes the video data to the emotion engine, which identifies pressure and anxiety from Player A's facial expressions and voice. For example, it analyzes that Player A is nervous before a match.

[0608] 4. Providing advice: Based on the analysis results, the server generates "Five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[0609] 5. 24-hour support: Player A feels anxious before a match and consults the chatbot within the app. The server uses past data and sentiment analysis results to provide advice on "strategies that were successful in the previous match."

[0610] 6. Self-evaluation: At the end of the month, the server sends a self-evaluation sheet to Player A, who answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[0611] This allows the system of the present invention to provide users with efficient and continuous mental support, improving their performance. By adding an emotion engine, it becomes possible to provide advice and support according to the user's emotional state.

[0612] (Example of a prompt)

[0613] "What are some specific ways to ease pre-game tension?"

[0614] "Please tell me some effective practice methods to improve my shooting percentage."

[0615] "What are the sentiment analysis results for recent games?"

[0616] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0617] Step 1: Upload your data

[0618] Description:

[0619] Users: Use their smartphones or PCs to upload their own game video and audio data to a dedicated application.

[0620] Input: Game video (file format: MP4) and audio data (file format: WAV)

[0621] Output: Data transmission request for receiving

[0622] Device: Validate the file format of the received video and audio data to ensure it is in the correct format.

[0623] Specific operation: Verify that the video data is in MP4 format and the audio data is in WAV format.

[0624] Input: Game video and audio data

[0625] Output: Validated data

[0626] Terminal: Compresses the verified data and transfers it to the server.

[0627] What it does: Improves transfer speeds by compressing data.

[0628] Input: Validated data

[0629] Output: Compressed data and transfer request

[0630] Step 2: Save your data

[0631] Description:

[0632] Server: Receives compressed data sent from the device and stores it in a database.

[0633] Input: Compressed data

[0634] Output: Data save confirmation

[0635] Server: Creates and properly manages the directory of stored data.

[0636] What it does: Creates a directory in the database and tags the data.

[0637] Input: Compressed data

[0638] Output: Data directory information

[0639] Step 3: Data analysis

[0640] Description:

[0641] Server: Passes saved game footage and audio data to the AI ​​analysis engine. Uses "OpenPose" or "Librosa."

[0642] Input: Data directory information

[0643] Output: Analysis request and target data

[0644] Server: The AI ​​analysis engine identifies motion characteristics from video and audio events from audio.

[0645] Specific actions: Analyze users' shooting success rate and frequency of shouting.

[0646] Input: Video and audio data

[0647] Output: Analysis results (shooting success rate, frequency of calls, etc.)

[0648] Server: Stores the analysis results in a database.

[0649] Input: Analysis results

[0650] Output: Saved analysis results

[0651] Step 4: Sentiment Analysis

[0652] Description:

[0653] Server: Passes video and audio data to the emotion analysis engine. Uses "FER2013" or "OpenSmile".

[0654] Input: Video and audio data

[0655] Output: Sentiment analysis request

[0656] Server: The emotion analysis engine identifies the user's emotional state from their facial expressions and tone of voice.

[0657] What it does: Analyzes the user's facial expressions in each frame of video and identifies emotions from the tone and intonation of the voice.

[0658] Input: Video and audio data

[0659] Output: Emotion analysis results (e.g., joy, anger, sadness, tension)

[0660] Server: Stores the sentiment analysis results in a database.

[0661] Input: Sentiment analysis results

[0662] Output: Saved sentiment analysis results

[0663] Step 5: Generate Advice

[0664] Description:

[0665] Server: Based on the analysis results and sentiment analysis results, the generative AI model "GPT-4" is used to generate personalized advice.

[0666] Input: Analysis results and sentiment analysis results

[0667] Output: Advice generation request

[0668] Server: Generates specific advice based on the user's requirements.

[0669] Specific actions: Generate practice methods to improve shooting success rate and breathing techniques to relax.

[0670] Input: Advice generation request

[0671] Output: personalized advice

[0672] Server: Sends the generated advice to the user's device.

[0673] Input: personalized advice

[0674] Output: Advice submission request

[0675] Terminal: Receives the advice and notifies the user within the application.

[0676] Input: Advice submission request

[0677] Output: User notification

[0678] Step 6: 24-hour support

[0679] Description:

[0680] Users: Ask questions using the chatbot functionality within the application.

[0681] Input: User question

[0682] Output: Question request

[0683] Terminal: Sends a question to the server.

[0684] Input: Question request

[0685] Output: Request sent to server

[0686] Server: Refers to the emotion engine and knowledge base to generate answers based on the questions.

[0687] Specific actions: Generate appropriate answers from past data and provide specific advice based on sentiment analysis results.

[0688] Input: Question request

[0689] Output: Answer generation request

[0690] Server: Generates the answer and sends it to the user's device.

[0691] Input: Answer generation request

[0692] Output: Reply request

[0693] Terminal: Receives the answer and displays it to the user in real time.

[0694] Input: Reply Send Request

[0695] Output: Real-time display

[0696] Step 7: Improving self-esteem and positivity

[0697] Description:

[0698] Server: Periodically generates self-assessment sheets and questionnaires and sends them to the user's device.

[0699] Input: Self-assessment sheet generation request

[0700] Output: Self-assessment sheet

[0701] Terminal: Displays the received self-assessment sheet or questionnaire to the user and prompts them to respond.

[0702] Input: Self-evaluation sheet

[0703] Output: what is displayed to the user

[0704] User: Answers the self-evaluation sheet and sends the data from the device to the server.

[0705] Input: User response data

[0706] Output: Data sent to the server

[0707] Server: Analyzes the received response data and generates messages and advice to improve positive feelings.

[0708] What it does: Analyzes responses and generates positive messages based on the user's progress and success stories.

[0709] Input: User response data

[0710] Output: Message creation request

[0711] Server: Sends generated messages and advice to the user's terminal.

[0712] Input: Message creation request

[0713] Output: Message send request

[0714] Terminal: Receives messages and advice and displays them to the user.

[0715] Input: Message Send Request

[0716] Output: what is displayed to the user

[0717] (Application example 2)

[0718] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0719] Current factory robots lack feedback for improving work efficiency and maintenance when performance declines or abnormalities occur. It is also difficult to monitor the robot's operating status in real time and take appropriate measures, making it difficult to maintain optimal performance. This creates a risk of a decline in productivity throughout the factory.

[0720] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving game video and audio data, artificial intelligence engine means for analyzing the received data, means for generating personalized advice based on the analysis results, means for providing the generated advice to the user, chatbot means for constantly accepting inquiries from the user, means for providing self-evaluation sheets and questionnaires and analyzing the results, means for generating messages to improve self-esteem based on the analysis results and providing them to the user, means for receiving and analyzing video and audio data of factory robots working, emotion engine means for monitoring the robot's operating performance and abnormalities, and means for generating performance improvement and maintenance advice based on the emotion analysis results. This enables efficient operation of the robot and appropriate maintenance.

[0721] "Game video and audio data" refers to data that refers to video and audio related to a game such as a sport or competition.

[0722] "Means for receiving" refers to a device or method for obtaining and storing particular data.

[0723] An "artificial intelligence engine means" is a device or software that analyzes data using technologies such as machine learning and deep learning and extracts useful information.

[0724] "Personalized advice" refers to specific instructions or suggestions tailored to a particular user's needs and circumstances.

[0725] "Generating means" refers to a device or method for producing specific information or data.

[0726] The "means for providing to the user" refers to a method or device for appropriately notifying a specific user of the generated data or results.

[0727] A "chatbot" is software that accepts user questions and inquiries through automated dialogue and provides appropriate answers.

[0728] "Self-evaluation sheets and questionnaires" refer to lists of questions that allow users to evaluate their own performance and feelings.

[0729] The "analyzing means" refers to a device or method for analyzing the received data in detail and extracting useful information.

[0730] "Self-esteem boosting messages" are positive messages designed to boost a user's confidence and self-esteem.

[0731] A "factory robot" refers to a machine or device that performs work automatically in a factory.

[0732] "Operational performance" is an indicator of how efficiently a machine or system is functioning.

[0733] The "emotion engine means" is a device or software that analyzes video and audio data and identifies emotions and abnormalities therein.

[0734] "Maintenance advice" means specific instructions or suggestions for how to properly maintain or repair a machine or system.

[0735] The present invention is a system for improving the performance of factory robots and supporting their maintenance, and is equipped with an artificial intelligence engine that receives and analyzes game video and audio data, and a means for generating advice based on the analysis results. This system includes the following elements:

[0736] 1. Data Collection

[0737] The server receives work video and audio data from the factory robots. This data is collected using cameras and microphones inside the robots and sent to the server. Specifically, the camera captures the robot's movements, and the microphone records the audio while it is working.

[0738] 2. Data Analysis

[0739] The server analyzes the received video and audio data using an AI analysis engine (e.g., Google Cloud Vision API, AWS (registered trademark) Rekognition). As a result of the analysis, metrics such as the robot's operational performance, success rate, and abnormal sound analysis are extracted. These analysis results are stored in a database and managed for each user.

[0740] 3. Emotion analysis

[0741] The server uses an emotion engine (e.g., IBM Watson (registered trademark) Tone Analyzer) to perform emotion analysis of the received video and audio data. Specifically, it identifies signs of abnormalities or malfunctions from the video and audio of the robot's movements. Based on the results of this analysis, it identifies the robot's state, which requires appropriate action.

[0742] 4. Generating personalized advice

[0743] Based on the results of data analysis and emotion analysis, the server generates specific advice for improving the robot's efficiency and maintenance. This advice includes, for example, how to optimize specific operations and recommendations for regular maintenance. The generated advice is sent to the supervisor's device in real time and displayed.

[0744] 5. 24 / 7 support

[0745] Users (supervisors) can consult the chatbot in real time about robot malfunctions or questions. The server receives the user's question, generates an appropriate answer based on the analysis results and emotion engine data, and sends it to the supervisor's device. This makes it possible to receive appropriate feedback in real time.

[0746] 6. Improved self-esteem and self-confidence

[0747] The server periodically generates self-evaluation sheets and questionnaires and sends them to the supervisor's device. The supervisor answers these and sends the results to the server. Based on the analysis results, the server generates suggestions for improving the robot and positive messages and provides them to the supervisor. This helps the supervisor improve their self-evaluation and confidence in the robot's performance.

[0748] Specific examples

[0749] As a specific example, there is an instruction using the following prompt sentence.

[0750] "Analyze the video and audio of the robot working and provide advice to improve efficiency."

[0751] Based on this prompt, the server collects and analyzes the necessary data and provides appropriate advice to the supervisor's terminal.

[0752] The above is a description of a specific embodiment for carrying out the present invention. This system is expected to improve the performance of factory robots and ensure appropriate maintenance, thereby improving overall productivity.

[0753] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0754] Step 1:

[0755] Data collection

[0756] The server receives work video and audio data from the factory robot's camera and microphone. The input is video and audio data, which is then stored in a database. Specifically, the robot acquires data in real time while in operation and sends it to the server via the network. The server receives the data and stores it in cloud storage.

[0757] Step 2:

[0758] Data analysis

[0759] The server passes the received video and audio data to an AI analysis engine. The input is video and audio data, and the output is robot performance metrics. Specifically, the AI ​​analysis engine (e.g., Google Cloud Vision API, AWS Rekognition) analyzes the data and extracts task success rates, error rates, and other metrics. These analysis results are stored in a database.

[0760] Step 3:

[0761] Emotion analysis

[0762] The server passes the received video and audio data to the emotion engine. The input is video and audio data, and the output is the emotion analysis results that indicate signs of abnormalities or malfunctions while the robot is operating. Specifically, the emotion engine (e.g., IBM Watson Tone Analyzer) identifies abnormal sounds and abnormal machine behavior from the video and audio. These results are also stored in a database.

[0763] Step 4:

[0764] Generate personalized advice

[0765] The server generates advice based on the results of data analysis and emotion analysis. The input is the analysis results and emotion analysis results, and the output is specific improvement advice. Specifically, the server generates robot efficiency measures and maintenance suggestions based on the integrated data, and organizes them in text format. This is then sent to the supervisor's terminal in real time.

[0766] Step 5:

[0767] 24 / 7 support

[0768] Users use the chatbot function to inquire about defects or questions about factory robots at the server. The input is the user's question, and the output is a real-time answer generated via the chatbot. Specifically, the system receives the user's question, references past data and sentiment analysis results, and generates an appropriate answer. The generated answer is then sent via the chatbot to the supervisor's terminal.

[0769] Step 6:

[0770] Improved self-esteem and self-esteem

[0771] The server periodically generates self-evaluation sheets and questionnaires and sends them to the supervisor's terminal. The input is the evaluation sheet generated by the server, and the output is the user's response data. Specifically, the supervisor checks the list and enters their responses. The server analyzes the received response data, generates areas for improvement for the robot, and generates positive messages, and notifies the supervisor.

[0772] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0773] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0774] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0775] [Second embodiment]

[0776] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0777] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0778] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0779] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0780] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0781] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0782] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0783] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0784] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0785] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0786] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0787] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0788] This invention relates to an AI mental trainer system aimed at improving mental health. It receives and analyzes game video and audio data and provides personalized advice. This system allows users to receive efficient and continuous mental support regardless of time or place.

[0789] Program processing explanation

[0790] 1. Data Collection

[0791] User:

[0792] Users use their smartphones or PCs to upload their game video and audio data to the application, providing the system with data that can be later analyzed.

[0793] Device:

[0794] The terminal receives video and audio data from the user and transmits it to the server.

[0795] server:

[0796] The server stores the received data and prepares it for the next analysis process.

[0797] 2. Data Analysis

[0798] server:

[0799] The received game video and audio data is passed to an AI analysis engine, which extracts the user's movements, success rate, and performance metrics, such as shot success rate and running distance.

[0800] server:

[0801] The analysis results are stored in a database and managed for each user.

[0802] 3. Generating personalized advice

[0803] server:

[0804] It generates personalized advice based on the user's past data and current analysis results, including specific practice methods and mental advice.

[0805] server:

[0806] The generated advice is transmitted to the user terminal.

[0807] Device:

[0808] Display the received advice to the user within the application.

[0809] 4. 24 / 7 support

[0810] User:

[0811] Users can access the AI ​​chatbot within the application when they feel anxious before a match or during a crucial moment.

[0812] Device:

[0813] Start the chatbot function and connect to the server.

[0814] server:

[0815] Receives user questions and generates answers by referencing past data and knowledge bases.

[0816] server:

[0817] Answers to questions and advice are sent to the device in real time.

[0818] Device:

[0819] The received response is displayed to the user.

[0820] 5. Improved self-esteem and self-confidence

[0821] server:

[0822] Self-evaluation sheets and questionnaires are generated periodically and sent to the user's terminal.

[0823] Device:

[0824] The received self-evaluation sheet is displayed to the user, and an interface is provided to prompt the user to respond.

[0825] User:

[0826] The user answers the self-evaluation sheet and sends the data from the terminal to the server.

[0827] server:

[0828] The received data is analyzed and messages and advice are generated to improve self-esteem.

[0829] server:

[0830] The generated message or advice is sent to the user terminal.

[0831] Device:

[0832] The received messages and advice are displayed to the user to improve self-evaluation and self-affirmation.

[0833] Specific examples

[0834] 1. Example of Athlete A

[0835] Data collection:

[0836] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[0837] Data Analysis:

[0838] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[0839] Providing personalized advice:

[0840] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[0841] 24 / 7 support:

[0842] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data to provide advice on "strategies that were successful in the previous match."

[0843] Increased self-esteem and self-esteem:

[0844] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[0845] In this way, the system of the present invention provides efficient and continuous mental support to users, thereby improving their performance.

[0846] The processing flow will be explained below.

[0847] Data collection and analysis

[0848] Step 1:

[0849] Users use their smartphones or PCs to record video and audio data of their own matches.

[0850] Step 2:

[0851] Users upload the captured game footage and audio data to the server via an application on their device.

[0852] Step 3:

[0853] The device receives the uploaded video and audio data and sends it to the server.

[0854] Step 4:

[0855] The server stores the received video and audio data and prepares it for the next analysis process.

[0856] Step 5:

[0857] The server passes the video data to an AI analytics engine, which analyzes the data and extracts user behavior, success rate, and performance metrics.

[0858] Step 6:

[0859] The server stores the analysis results in an internal database and manages them for each user.

[0860] Generate personalized advice

[0861] Step 1:

[0862] The server prepares data to generate personalized advice based on the user's past performance data and current analysis results.

[0863] Step 2:

[0864] The server's AI engine analyzes the collected data and generates practice methods and mental advice tailored to the user.

[0865] Step 3:

[0866] The server transmits the generated advice to the user's terminal.

[0867] Step 4:

[0868] The terminal displays the received advice to the user within the application.

[0869] Step 5:

[0870] The user checks the displayed advice and prepares to proceed to the next phase.

[0871] 24 / 7 support

[0872] Step 1:

[0873] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[0874] Step 2:

[0875] The terminal displays the chatbot screen and connects to the server.

[0876] Step 3:

[0877] Users input questions or inquiries to the chatbot.

[0878] Step 4:

[0879] The server receives questions and inquiries from users and generates answers by referencing past data and a knowledge base.

[0880] Step 5:

[0881] The server sends the generated answers and advice to the user's terminal.

[0882] Step 6:

[0883] The terminal displays the received answers and advice to the user in real time.

[0884] Step 7:

[0885] The server stores the chat history with the user in a database, which can be used for future counseling.

[0886] Improved self-esteem and self-esteem

[0887] Step 1:

[0888] The server periodically generates self-evaluation sheets and questionnaires and sends them to the user's terminal.

[0889] Step 2:

[0890] The terminal displays the received self-evaluation sheet to the user and provides an interface that prompts the user to answer.

[0891] Step 3:

[0892] The user answers the self-evaluation sheet and transmits the data from the terminal to the server.

[0893] Step 4:

[0894] The server receives the data from the self-assessment sheet, and the AI ​​engine performs the analysis.

[0895] Step 5:

[0896] Based on the analysis results, the server generates positive messages and advice to improve self-esteem.

[0897] Step 6:

[0898] The server sends the generated message or advice to the user's terminal.

[0899] Step 7:

[0900] The device displays the received messages and advice to the user, encouraging them to improve their self-esteem.

[0901] Example 1

[0902] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0903] Conventional mental training systems have had issues with insufficient collection and analysis of user performance data, making it difficult to provide personalized advice. Furthermore, they lacked the functionality to accept consultations from users in real time 24 hours a day, 365 days a year, leaving insufficient means for improving self-evaluation and self-affirmation.

[0904] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0905] In this invention, the server includes: means for receiving game video and audio data; artificial intelligence engine means for analyzing the received data; means for generating personalized advice based on the analysis results; means for providing the generated advice to the user; chatbot means for always accepting consultations from users; means for providing self-assessment sheets and questionnaires and analyzing the results; means for generating messages to improve self-esteem based on the analysis results and providing them to the user; means for the user to upload game video and audio data using the terminal; means for saving the data and preparing for the next analysis process; means for passing the data to the AI ​​analysis engine and extracting performance metrics; means for saving the performance metrics in a database and managing them for each user; means for sending the generated advice to the user terminal in real time; means for displaying the received advice on the user terminal; means for activating the AI ​​chatbot function and connecting to the server; means for using an artificial intelligence model to generate optimal answers to the user's questions and sending them to the terminal; means for the user to access the chatbot when they feel anxious before a game; and means for generating self-assessment sheets and questionnaires for the user and sending them to the terminal. This enables detailed analysis of users' performance data and the provision of efficient and personalized advice, providing real-time mental support and improving self-evaluation 24 hours a day, 365 days a year.

[0906] "Game video and audio data" refers to video and audio data recorded by a user during a sport or activity.

[0907] The "receiving means" refers to a function or module for acquiring data transmitted from a user terminal.

[0908] The "analytical artificial intelligence engine means" is a system that analyzes received data using deep learning or machine learning to extract important performance metrics.

[0909] A "means for generating personalized advice" is an algorithm or module for providing advice tailored to the individual needs of each user.

[0910] The "means for providing advice to the user" is a system for displaying and notifying the generated advice on the user's terminal.

[0911] A "chatbot" is a conversation system that uses artificial intelligence to accept questions and inquiries from users and automatically respond.

[0912] "Means for providing self-evaluation sheets and questionnaires" refers to a system that provides questions and forms for users to request self-evaluation and feedback.

[0913] The "means for analyzing the results" refers to algorithms and modules for analyzing the user's self-evaluation sheets and questionnaire responses.

[0914] The "means for generating messages to improve self-esteem" is a system that generates messages to improve a user's self-esteem and motivation based on the results of self-evaluation.

[0915] "Means for uploading" refers to the means by which a user sends game video and audio data to the system, typically via an application or web portal.

[0916] The "means for storing data" is a storage system for safely and efficiently storing received game video and audio data.

[0917] "Means for preparing for analysis processing" refers to a system that performs the necessary data preprocessing and formatting before analyzing the data.

[0918] "Means for extracting performance metrics" refers to a function that extracts important indicators of athletic performance (e.g., shooting success rate, distance traveled, etc.) from the analyzed data.

[0919] "Means for managing performance metrics" refers to a system that organizes, stores, and manages extracted indicators for each user.

[0920] The "means for transmitting in real time" is a communication system for instantly transmitting the generated advice or message to the user's terminal.

[0921] "Means for activating AI chatbot functions" refers to a system that activates chatbot functions in response to user requests.

[0922] "Means for generating optimal answers using an artificial intelligence model" refers to an AI module that generates optimal answers to user questions by referencing past data and knowledge bases.

[0923] The "means for transmitting to the terminal" is a system that transmits the generated answers and advice to the user's terminal.

[0924] "Means to access a chatbot when feeling anxious" is a function that allows users to access a chatbot and consult with it when they feel psychological anxiety.

[0925] "Means for generating self-assessment sheets and questionnaires" refers to a system that creates questions and forms for users to periodically conduct self-assessments.

[0926] The program in this system is an AI mental trainer system designed to improve the user's mental health. This system receives and analyzes game video and audio data and provides personalized advice. Specific embodiments of this system are described below.

[0927] Data collection

[0928] User:

[0929] Users can upload their game video and audio data to the application using their smartphone or PC. For example, they can send video files saved on their smartphone via the app's upload function.

[0930] Device:

[0931] The device (user's smartphone or PC) receives the uploaded video and audio data and sends it to the server. The data is designed to be sent to the server via a network connection.

[0932] server:

[0933] The server stores the received video and audio data in a database and prepares it for the next analysis process. The stored data is used for analysis by the AI ​​analysis engine, which will be described later.

[0934] Data analysis

[0935] server:

[0936] The server passes the received data to an AI analytics engine, such as TensorFlow or PyTorch, which uses deep learning techniques, to extract performance metrics (e.g., shooting percentage, distance traveled), which are then stored in a database.

[0937] server:

[0938] The analysis results are managed for each user and used for subsequent processing as needed.

[0939] Generate personalized advice

[0940] server:

[0941] The server uses a generative AI model to generate personalized advice based on the user's past data and analysis results. For example, it uses a machine learning model to generate "five practice methods to improve shooting accuracy."

[0942] server:

[0943] The generated advice is sent to the user terminal in real time via network communication.

[0944] Device:

[0945] The device then displays the received advice to the user within the application. The advice can be provided not only as text information, but also as video or audio guides.

[0946] 24 / 7 support

[0947] User:

[0948] Before a match or when feeling stressed, users can access the AI ​​chatbot within the app by opening the app's chatbot function and typing in a question.

[0949] Device:

[0950] The chatbot function is activated and connected to the server, where real-time communication takes place.

[0951] server:

[0952] When a user asks a question, it uses past data and a knowledge base to generate the best answer, often using a natural language processing (NLP) model or, in some cases, a generative AI model like GPT-3.

[0953] server:

[0954] The generated answer is sent to the user terminal.

[0955] Device:

[0956] The terminal displays the received answers to the user, and responds quickly to the user's concerns and questions.

[0957] Improved self-esteem and self-esteem

[0958] server:

[0959] Self-assessment sheets and questionnaires are periodically generated and sent to the user's device. This process also uses machine learning models to generate questions that are appropriate for the user.

[0960] Device:

[0961] The device displays the received self-assessment sheet to the user within the app, providing an interface that allows the user to easily respond.

[0962] User:

[0963] The user answers the self-evaluation sheet and sends the data from the terminal to the server. All the user has to do is enter the details of their self-evaluation and press the send button.

[0964] server:

[0965] The system analyzes the received evaluation data and generates messages and advice to improve the user's self-esteem. For example, it generates positive messages such as "Your performance has improved dramatically this month! Keep it up!"

[0966] server:

[0967] The generated message or advice is sent to the user terminal.

[0968] Device:

[0969] The device will display the received messages and advice to the user, aiming to improve their motivation and self-esteem. Notifications will be sent in the form of push notifications or in-app messages.

[0970] Specific examples

[0971] 1. Example of Athlete A

[0972] Data collection:

[0973] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[0974] Data Analysis:

[0975] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[0976] Providing personalized advice:

[0977] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[0978] 24 / 7 support:

[0979] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data to provide advice on "strategies that were successful in the previous match."

[0980] Increased self-esteem and self-esteem:

[0981] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[0982] Prompt Sentence Examples

[0983] "What is your shooting percentage in last month's games?"

[0984] "What do you need to improve in practice this week?"

[0985] "How can I reduce the anxiety I feel before a match?"

[0986] In this way, the system of the present invention provides efficient and continuous mental support to users, thereby improving their performance.

[0987] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0988] Step 1:

[0989] User:

[0990] Users use their smartphones or PCs to upload their own game video and audio data to the application by using the app's upload function to select the data and then pressing the send button.

[0991] Input: Game video and audio data

[0992] Output: The data file received by the application

[0993] Step 2:

[0994] Device:

[0995] The device (user's smartphone or PC) receives the uploaded video and audio data and sends it to the server via the Internet. A network connection must be established.

[0996] Input: User uploaded data

[0997] Output: Data packet sent to the server

[0998] Step 3:

[0999] server:

[1000] The server stores the received video and audio data in a database. Before storing the data, it checks it to ensure that it is not corrupted or invalid.

[1001] Input: Data sent from the terminal

[1002] Output: Data stored in the database

[1003] Step 4:

[1004] server:

[1005] The stored data is passed to an AI analytics engine for analysis, which uses deep learning techniques (such as TensorFlow or PyTorch) to extract performance metrics.

[1006] Input: Video and audio data stored in a database

[1007] Output: Extracted performance metrics (e.g., shooting percentage, distance traveled)

[1008] Step 5:

[1009] server:

[1010] The analysis results are saved in a database and managed for each user. The saved data can also be used as user history data.

[1011] Input: Performance metrics

[1012] Output: Analysis results stored in a database

[1013] Step 6:

[1014] server:

[1015] Using a generative AI model, the system generates personalized advice based on the user's past data and the latest analysis results. For example, if a user's shooting success rate is low, the system generates "practice methods to improve shooting accuracy."

[1016] Input: Historical data and latest analysis results

[1017] Output: Personalized advice

[1018] Step 7:

[1019] server:

[1020] The generated advice is sent to the user's device via the network, and notifications are also sent in real time.

[1021] Enter: personalized advice

[1022] Output: Advice sent to the user's terminal

[1023] Step 8:

[1024] Device:

[1025] The device displays the received advice to the user within the application. Advice may be provided not only as text information, but also as video or audio guides.

[1026] Input: Advice sent by the server

[1027] Output: Advice displayed to the user

[1028] Step 9:

[1029] User:

[1030] Before a match or when feeling stressed, users can access the AI ​​chatbot within the application and input their questions or concerns, and receive support in an interactive format using the chatbot's functionality.

[1031] Input: User questions and inquiries

[1032] Output: The text entered into the chatbot

[1033] Step 10:

[1034] Device:

[1035] The chatbot function is activated and the entered questions or inquiries are sent to the server. Since the connection is real-time, a low-latency network environment is recommended.

[1036] Input: User questions and inquiries

[1037] Output: Questions and queries sent to the server

[1038] Step 11:

[1039] server:

[1040] It uses past data and knowledge bases to generate optimal answers using artificial intelligence models (e.g., GPT-3). Depending on the question, it may refer to past match data or analysis results.

[1041] Input: User questions and knowledge base

[1042] Output: The generated answer

[1043] Step 12:

[1044] server:

[1045] The generated answer is sent to the user's device in real time, and the user is immediately notified via a network connection.

[1046] Input: Generated answer

[1047] Output: Answer sent to user's terminal

[1048] Step 13:

[1049] Device:

[1050] The device then displays the received answers to the user, who can use them to solve problems and reduce stress.

[1051] Input: The answer sent by the server

[1052] Output: The answer that is displayed to the user

[1053] Step 14:

[1054] server:

[1055] Self-assessment sheets and questionnaires are generated periodically and sent to the user's device. Questions appropriate for the user are selected using a machine learning model.

[1056] Input: User history and rating data

[1057] Output: Generated self-assessment sheets and questionnaires

[1058] Step 15:

[1059] Device:

[1060] It provides an interface that displays self-evaluation sheets and questionnaires to users and encourages them to respond, allowing users to input their answers intuitively.

[1061] Input: Self-assessment sheets and surveys sent from the server

[1062] Output: Questions displayed to the user

[1063] Step 16:

[1064] User:

[1065] Users answer a self-assessment form and the data is sent from their device to a server, which may be designed to maintain anonymity.

[1066] Input: User's answer

[1067] Output: Response data sent to the server

[1068] Step 17:

[1069] server:

[1070] The system analyzes the received self-evaluation data and generates messages and advice to improve the user's self-esteem, placing emphasis on positive feedback.

[1071] Input: User response data

[1072] Output: any messages or advice generated

[1073] Step 18:

[1074] server:

[1075] The generated messages and advice are sent to the user's terminal in real time, and the user is notified immediately.

[1076] Input: Generated messages and advice

[1077] Output: Messages and advice sent to the user's terminal

[1078] Step 19:

[1079] Device:

[1080] The device will display the received messages and advice to the user. Messages will be sent in the form of push notifications or in-app messages.

[1081] Input: Messages and advice sent by the server

[1082] Output: Messages and advice displayed to the user

[1083] In this way, the system of the present invention provides efficient and continuous mental support to the user through the specific actions at each step, thereby improving performance.

[1084] (Application example 1)

[1085] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1086] In today's food delivery industry, delivery workers are experiencing increasing stress and anxiety during their daily work, which can lead to a decline in work efficiency and mental health. Furthermore, the lack of personalized support tailored to each delivery worker means that improvements in mental health and performance are not being linked. Given this background, there is a need for a system that provides efficient and continuous mental support to delivery workers, balancing work efficiency and mental health.

[1087] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1088] In this invention, the server includes means for receiving game video and audio data, artificial intelligence engine means for analyzing the received data, means for generating personalized advice based on the analysis results, means for providing the generated advice to the user, chatbot means for constantly accepting inquiries from the user, means for providing self-evaluation sheets and questionnaires and analyzing the results, means for generating messages to improve self-esteem based on the analysis results and providing them to the user, means for recording guidance and voice memos for delivery personnel during deliveries, means for analyzing the recorded data to extract delivery performance and stress factors, and means for generating advice for work improvement and mental support based on the extraction results. This allows for efficient and continuous mental support to be provided to delivery personnel, enabling improved work efficiency and improved mental health.

[1089] The "means for receiving game video and audio data" refers to a device or system for receiving video and audio recorded by a user.

[1090] The "artificial intelligence engine means for analyzing received data" is an analytical engine that uses artificial intelligence to analyze received video and audio data and extract useful information.

[1091] The "means for generating personalized advice based on the analysis results" refers to a device or system for customizing and generating optimal advice for a user based on the analyzed data.

[1092] The "means for providing generated advice to a user" is a device or system for providing customized advice to a user.

[1093] The "chatbot means that always accepts inquiries from users" is a chatbot that always accepts questions and inquiries from users and automatically responds to them.

[1094] The "means for providing a self-assessment sheet or questionnaire and analyzing the results" refers to a device or system for providing a self-assessment sheet or questionnaire to users and analyzing the response results.

[1095] The "means for generating and providing a user with a message that will improve self-esteem based on the analysis results" refers to a device or system for generating and providing a user with a message that will improve self-esteem based on the analyzed evaluation data.

[1096] The "means for the delivery person to record guidance and voice memos during delivery" refers to a device or system for recording guidance and voice memos during delivery.

[1097] The "means for analyzing recorded data to extract delivery performance and stress factors" refers to a device or system for analyzing recorded guidance and voice memos to identify delivery performance and stress factors.

[1098] The "means for generating advice for business improvement and mental support based on the extraction results" refers to a device or system for generating advice for business improvement and mental support based on the analyzed data.

[1099] As an embodiment of the present invention, a system for improving the mental health of food delivery workers is constructed as follows.

[1100] 1. Data Collection:

[1101] User: Delivery personnel use their smartphones to record guidance and voice memos during deliveries, which allows data on daily work and delivery situations to be accumulated.

[1102] Device: The smartphone transmits the recorded data to the server. The smartphone is equipped with recording and data transmission functions.

[1103] 2. Data Analysis:

[1104] Server: Uses an AI analysis engine to analyze the received guidance and voice data. The analysis engine identifies delivery performance (e.g., delivery time, success rate) and stress factors (e.g., stress word analysis in the voice).

[1105] The specific software used includes voice recognition technology and machine learning models, such as the Google Cloud Speech-to-Text API and TensorFlow models.

[1106] 3. Advice Generation:

[1107] Server: Generates personalized advice based on the analysis results, including specific suggestions for work improvement and mental support.

[1108] Device: Advice is sent to the smartphone and displayed to the user using notifications and dashboard features.

[1109] 4. Real-time support:

[1110] Users: When they feel anxious or stressed during a delivery, they can use the in-app chatbot for advice.

[1111] Terminal: The chatbot function is activated and connected to the server.

[1112] Server: The chatbot references historical data and a knowledge base to provide real-time support.

[1113] 5. Self-assessment and feedback:

[1114] Server: Periodically generates self-evaluation sheets and questionnaires and sends them to the delivery person's smartphone.

[1115] Terminal: The delivery person answers a self-evaluation sheet and sends the data to the server.

[1116] Server: Based on the analyzed data, it generates and sends a self-esteem-boosting message to the delivery person, for example, providing feedback such as "Your stress level has improved by 30% over the past month."

[1117] For example, when delivery person A drives a new route, he can check his smartphone for advice on how to relieve stress and how to deliver efficiently. Based on past data, the AI ​​chatbot will respond by saying, "Since you were able to reduce the time by 15 minutes on your last route, we'll suggest ways to reduce stress this time as well."

[1118] Example prompt sentence:

[1119] "Analyze delivery drivers' voice guidance and provide personalized advice on stress factors and how to improve their work."

[1120] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1121] Step 1:

[1122] Data collection:

[1123] Input: Guidance and voice memos recorded by delivery personnel using their smartphones during deliveries

[1124] Specific behavior:

[1125] User: While making a delivery, the delivery person uses the recording function on their smartphone to record the immediate situation, thoughts, and emotions while driving as voice memos.

[1126] Device: The smartphone temporarily stores the recorded audio data and sends it to the server using mobile data.

[1127] Output: Audio data sent to the server

[1128] Step 2:

[1129] Data received and stored on the server:

[1130] Input: Audio data from the device

[1131] Specific behavior:

[1132] Server: Receives the voice data and stores it in a database, tagged with the date and delivery person ID.

[1133] Output: Saved audio data

[1134] Step 3:

[1135] Data Analysis:

[1136] Input: Stored audio data

[1137] Specific behavior:

[1138] Server: Passes the voice data to an AI analytics engine, which uses speech recognition technology to convert it into text and extracts stress factors and performance metrics, such as detecting specific keywords and phrases that indicate stress, and calculating delivery times and delivery success rates.

[1139] Tools used: Google Cloud Speech-to-Text API and TensorFlow model

[1140] Output: Analysis results (stress factors, delivery performance metrics)

[1141] Step 4:

[1142] Generate personalized advice:

[1143] Input: Analysis results

[1144] Specific behavior:

[1145] Server: Based on the analysis results, it generates advice for business improvement and mental support. For example, it generates specific advice such as "Based on the data from the past month, it would be better to choose off-peak times to further shorten delivery times."

[1146] Output: Personalized advice

[1147] Step 5:

[1148] Providing advice:

[1149] Enter: personalized advice

[1150] Specific behavior:

[1151] Server: Sends the generated advice to the delivery person's smartphone.

[1152] Device: The smartphone displays the received advice to the user via the notification function.

[1153] Output: Advice displayed on the delivery person's smartphone

[1154] Step 6:

[1155] Real-time support:

[1156] Input: Message from delivery person

[1157] Specific behavior:

[1158] User: If they have any concerns or questions during delivery, they can send a consultation message using the chatbot function on their smartphone.

[1159] Terminal: The smartphone sends a consultation message to the server.

[1160] Server: The chatbot references historical data and a knowledge base to generate real-time responses, such as "Your last route saved you 15 minutes, so try a similar route this time."

[1161] Output: Real-time response from the chatbot

[1162] Step 7:

[1163] Self-assessment and feedback:

[1164] Input: Self-evaluation sheet from the server, delivery person's evaluation answers

[1165] Specific behavior:

[1166] Server: Periodically sends self-evaluation sheets and surveys to delivery workers' smartphones, including questions about stress levels and work performance.

[1167] User: Answer each assessment item and submit the data from their smartphone.

[1168] Server: Analyzes the received evaluation data and generates a feedback message, such as providing positive feedback like "Your stress level has improved by 30% over the past month."

[1169] Output: Feedback message displayed on the delivery person's smartphone

[1170] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1171] This invention relates to an AI mental trainer system aimed at improving mental health. It receives and analyzes game video and audio data to provide personalized advice, and also incorporates an emotion engine that recognizes the user's emotions. This makes it possible to provide more appropriate mental support based on the user's needs and emotional state.

[1172] Program processing explanation

[1173] 1. Data Collection

[1174] User:

[1175] Users use their smartphones or PCs to upload their game video and audio data to the application, providing the system with data that can be later analyzed.

[1176] Device:

[1177] The terminal receives video and audio data from the user and transmits it to the server.

[1178] server:

[1179] The server stores the received data and prepares it for the next analysis process.

[1180] 2. Data Analysis

[1181] server:

[1182] The received game video and audio data is passed to an AI analysis engine, which extracts user actions, success rates, and performance metrics.

[1183] server:

[1184] The analysis results are stored in a database and managed for each user.

[1185] 3. Emotion analysis

[1186] server:

[1187] The received video and audio data is passed to the emotion engine, which analyzes the user's facial expressions and voice to determine their emotional state.

[1188] server:

[1189] Based on the analysis results of the emotion engine, data is prepared to generate more appropriate advice.

[1190] 4. Generating personalized advice

[1191] server:

[1192] It generates personalized advice based on the user's past performance data, current analysis results, and emotion engine analysis results.

[1193] server:

[1194] The generated advice is sent to the user's terminal.

[1195] Device:

[1196] Display the received advice to the user within the application.

[1197] 5. 24 / 7 support

[1198] User:

[1199] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[1200] Device:

[1201] Start the chatbot function and connect to the server.

[1202] server:

[1203] Receives a user question and generates an answer by referencing the emotion engine, past data, and knowledge base.

[1204] server:

[1205] Answers to questions and advice are sent to the user's device.

[1206] Device:

[1207] The received answers are displayed to the user in real time.

[1208] server:

[1209] The chat history with the user is saved in a database, which can be used for future counseling.

[1210] 6. Improved self-esteem and self-confidence

[1211] server:

[1212] Self-evaluation sheets and questionnaires are generated periodically and sent to the user's device.

[1213] Device:

[1214] The received self-evaluation sheet is displayed to the user, and an interface is provided to prompt the user to respond.

[1215] User:

[1216] The user answers the self-evaluation sheet and sends the data from the terminal to the server.

[1217] server:

[1218] The received data is analyzed and messages and advice are generated to improve self-esteem.

[1219] server:

[1220] The generated message or advice is sent to the user's terminal.

[1221] Device:

[1222] The received messages and advice are displayed to the user, encouraging them to improve their self-esteem.

[1223] Specific examples

[1224] 1. Example of Athlete A

[1225] Data collection:

[1226] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[1227] Data Analysis:

[1228] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[1229] Emotion analysis:

[1230] The server passes the video data to the emotion engine, which identifies pressure or anxiety from Player A's facial expressions and voice. For example, it analyzes that Player A is nervous before a match.

[1231] Providing personalized advice:

[1232] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[1233] 24 / 7 support:

[1234] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data and sentiment analysis results to provide advice on "strategies that were successful in the previous match."

[1235] Increased self-esteem and self-esteem:

[1236] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[1237] In this way, the system of the present invention provides users with efficient and continuous mental support, improving their performance. By adding an emotion engine, it becomes possible to provide more appropriate advice and support according to the user's emotional state.

[1238] The processing flow will be explained below.

[1239] MODE FOR CARRYING OUT THE INVENTION (DETAILED PROCESSING FLOW)

[1240] Data collection

[1241] Step 1:

[1242] Users use their smartphones or PCs to record video and audio data of their own matches.

[1243] Step 2:

[1244] Users upload the captured game footage and audio data to the server via an application on their device.

[1245] Step 3:

[1246] The device receives the uploaded video and audio data and sends it to the server.

[1247] Step 4:

[1248] The server stores the received video and audio data and prepares it for the next analysis process.

[1249] Data analysis

[1250] Step 5:

[1251] The server passes the video data to an AI analytics engine, which analyzes the data and extracts user behavior, success rate, and performance metrics.

[1252] Step 6:

[1253] The server stores the analysis results in an internal database and manages them for each user.

[1254] Emotion analysis

[1255] Step 7:

[1256] The server passes the video and audio data to the emotion engine, which analyzes the user's facial expressions and voice to determine their emotional state.

[1257] Step 8:

[1258] The server prepares data to generate more appropriate advice based on the analysis results of the emotion engine.

[1259] Generate personalized advice

[1260] Step 9:

[1261] The server generates personalized advice based on the user's past performance data, current analysis results, and the analysis results of the emotion engine.

[1262] Step 10:

[1263] The server transmits the generated advice to the user's terminal.

[1264] Step 11:

[1265] The terminal displays the received advice to the user within the application.

[1266] Step 12:

[1267] The user checks the displayed advice and prepares to proceed to the next phase.

[1268] 24 / 7 support

[1269] Step 13:

[1270] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[1271] Step 14:

[1272] The terminal displays the chatbot screen and connects to the server.

[1273] Step 15:

[1274] Users input questions or inquiries to the chatbot.

[1275] Step 16:

[1276] The server receives questions and inquiries from users, references past data, emotion engine results, and a knowledge base, and generates answers.

[1277] Step 17:

[1278] The server sends the generated answers and advice to the user's terminal.

[1279] Step 18:

[1280] The terminal displays the received answers and advice to the user in real time.

[1281] Step 19:

[1282] The server stores the chat history with the user in a database, which can be used for future counseling.

[1283] Improved self-esteem and self-esteem

[1284] Step 20:

[1285] The server periodically generates self-evaluation sheets and questionnaires and sends them to the user's terminal.

[1286] Step 21:

[1287] The terminal displays the received self-evaluation sheet to the user and provides an interface that prompts the user to answer.

[1288] Step 22:

[1289] The user answers the self-evaluation sheet and transmits the data from the terminal to the server.

[1290] Step 23:

[1291] The server receives the data from the self-assessment sheet, and the AI ​​engine performs the analysis.

[1292] Step 24:

[1293] Based on the analysis results, the server generates positive messages and advice to improve self-esteem.

[1294] Step 25:

[1295] The server sends the generated message or advice to the user's terminal.

[1296] Step 26:

[1297] The device displays the received messages and advice to the user, encouraging them to improve their self-esteem.

[1298] Specific examples

[1299] Example of Athlete A

[1300] Step 1:

[1301] Athlete A uploads game footage from his smartphone to the app.

[1302] Step 2:

[1303] The terminal receives the video and transmits it to the server.

[1304] Step 3:

[1305] The server receives the video and passes it to the AI ​​analysis engine.

[1306] Step 4:

[1307] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[1308] Step 5:

[1309] The server passes the video data to the emotion engine, which identifies pressure or anxiety from Player A's facial expressions and voice. For example, it analyzes that Player A is nervous before a match.

[1310] Step 6:

[1311] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[1312] Step 7:

[1313] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data and sentiment analysis results to provide advice on "strategies that were successful in the previous match."

[1314] Step 8:

[1315] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[1316] In this way, the system of the present invention, which is combined with an emotion engine, provides efficient and continuous mental support to the user, and realizes more appropriate advice and support according to the user's emotional state.

[1317] Example 2

[1318] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1319] Conventional mental care systems have difficulty analyzing users' emotional states and performance in real time and providing appropriate, personalized advice. Furthermore, there has been a lack of systems that provide comprehensive, continuous support for 24 / 7 support and for improving self-esteem based on self-evaluation. Therefore, there is a need for an effective system that can sustainably improve users' mental health and performance.

[1320] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1321] In this invention, the server includes a device for receiving game video and audio data, an intelligence engine for analyzing the received data, a device for generating personalized advice based on the analysis results, a device for providing the generated advice to users, a conversational bot device for constantly accepting consultations from users, an engine for analyzing emotional states, a device for providing self-evaluation sheets and questionnaires and analyzing the results, and a device for generating messages to improve positivity based on the analysis results and providing them to users. This makes it possible to analyze users' emotional states and performance in real time and provide personalized, appropriate advice. It also provides comprehensive and continuous support 24 hours a day, 365 days a year, and helps improve self-esteem based on self-evaluation.

[1322] The "device for receiving game video and audio data" is a device for electronically receiving game video and audio data provided by a user.

[1323] The "intelligence engine" is an analytical device that uses artificial intelligence to analyze received game video and audio data.

[1324] The "device for generating personalized advice" is a device that generates advice optimized for each user based on the analysis results.

[1325] The "device for providing generated advice to a user" is a device for providing generated advice to a user in a format that is easy to use.

[1326] A "conversation bot device" is an automatic response device that accepts inquiries from users 24 hours a day, 365 days a year, and provides answers in an interactive format.

[1327] The "emotional state analysis engine" is a device for analyzing the user's emotional state from received video and audio data.

[1328] The "device for providing a self-assessment sheet or questionnaire and analyzing the results" is a device that provides a self-assessment sheet or questionnaire to a user and analyzes the user's responses.

[1329] The "device for generating and providing users with messages that enhance positive self-esteem" is a device for generating and providing users with messages that enhance positive self-esteem based on the analysis results of self-evaluations and questionnaires.

[1330] This invention relates to an AI mental trainer system aimed at improving mental health. It receives and analyzes various data and provides personalized advice. It also incorporates an emotion engine that analyzes the user's emotional state. Below, we explain how this system is specifically implemented.

[1331] First, the system consists of the following main components:

[1332] 1. Data Collection Equipment

[1333] User: Uses a smartphone or PC to upload video and audio data from their match to a dedicated app.

[1334] Terminal: Receives data, compresses it if necessary, and sends it to the server. This process uses software that recognizes and processes specific data formats, such as MP4 for video data and WAV for audio data.

[1335] 2. Intelligence Engine

[1336] Server: Receives game footage and audio data stored in the database and analyzes the movement and audio characteristics using analysis tools such as "OpenPose" and "Librosa." The analysis results include the user's movement success rate and level of tension.

[1337] 3. Emotional state analysis engine

[1338] Server: Analyzes video and audio data using emotion analysis tools such as "FER2013" and "OpenSmile," and identifies emotions from the user's facial expressions and tone of voice.

[1339] 4. Device for generating personalized advice

[1340] Server: Based on the analysis results, the generative AI model GPT-4 is used to generate advice tailored to each user, such as practice methods to improve the user's shooting success rate or breathing techniques to help them relax.

[1341] 5. Device for Providing Generated Advice to Users

[1342] Server: Generates advice and sends it to the user's device.

[1343] Terminal: Receives the advice and displays it to the user within the application.

[1344] 6. Conversational Bot Device

[1345] Users: Use the chatbot feature within the application to ask questions before a game, after practice, etc.

[1346] Terminal: Receives the query and sends it to the server.

[1347] Server: Refers to the emotion engine and knowledge base to generate appropriate answers and send them to the user.

[1348] Terminal: Shows the answer to the user in real time.

[1349] 7. Self-evaluation sheets and questionnaires providing device

[1350] Server: Periodically generates self-assessment sheets and questionnaires and sends them to the user's device.

[1351] Terminal: Displays the received self-assessment sheet or questionnaire to the user and prompts them to respond.

[1352] User: Answers and sends the data from the device to the server.

[1353] Server: Analyzes the received data, generates messages and advice to improve positive feelings, and provides them to the user.

[1354] Specific examples

[1355] The following explanation will be given using athlete A as an example.

[1356] 1. Data collection: Athlete A uploads game footage from his smartphone to the app. The device receives the footage and sends it to the server. The server then passes the footage to the AI ​​analysis engine.

[1357] 2. Data analysis: The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. This analysis result is stored in a database.

[1358] 3. Emotion analysis: The server passes the video data to the emotion engine, which identifies pressure and anxiety from Player A's facial expressions and voice. For example, it analyzes that Player A is nervous before a match.

[1359] 4. Providing advice: Based on the analysis results, the server generates "Five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[1360] 5. 24-hour support: Player A feels anxious before a match and consults the chatbot within the app. The server uses past data and sentiment analysis results to provide advice on "strategies that were successful in the previous match."

[1361] 6. Self-evaluation: At the end of the month, the server sends a self-evaluation sheet to Player A, who answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[1362] This allows the system of the present invention to provide users with efficient and continuous mental support, improving their performance. By adding an emotion engine, it becomes possible to provide advice and support according to the user's emotional state.

[1363] (Example of a prompt)

[1364] "What are some specific ways to ease pre-game tension?"

[1365] "Please tell me some effective practice methods to improve my shooting percentage."

[1366] "What are the sentiment analysis results for recent games?"

[1367] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1368] Step 1: Upload your data

[1369] Description:

[1370] Users: Use their smartphones or PCs to upload their own game video and audio data to a dedicated application.

[1371] Input: Game video (file format: MP4) and audio data (file format: WAV)

[1372] Output: Data transmission request for receiving

[1373] Device: Validate the file format of the received video and audio data to ensure it is in the correct format.

[1374] Specific operation: Verify that the video data is in MP4 format and the audio data is in WAV format.

[1375] Input: Game video and audio data

[1376] Output: Validated data

[1377] Terminal: Compresses the verified data and transfers it to the server.

[1378] What it does: Improves transfer speeds by compressing data.

[1379] Input: Validated data

[1380] Output: Compressed data and transfer request

[1381] Step 2: Save your data

[1382] Description:

[1383] Server: Receives compressed data sent from the device and stores it in a database.

[1384] Input: Compressed data

[1385] Output: Data save confirmation

[1386] Server: Creates and properly manages the directory of stored data.

[1387] What it does: Creates a directory in the database and tags the data.

[1388] Input: Compressed data

[1389] Output: Data directory information

[1390] Step 3: Data analysis

[1391] Description:

[1392] Server: Passes saved game footage and audio data to the AI ​​analysis engine. Uses "OpenPose" or "Librosa."

[1393] Input: Data directory information

[1394] Output: Analysis request and target data

[1395] Server: The AI ​​analysis engine identifies motion characteristics from video and audio events from audio.

[1396] Specific actions: Analyze users' shooting success rate and frequency of shouting.

[1397] Input: Video and audio data

[1398] Output: Analysis results (shooting success rate, frequency of calls, etc.)

[1399] Server: Stores the analysis results in a database.

[1400] Input: Analysis results

[1401] Output: Saved analysis results

[1402] Step 4: Sentiment Analysis

[1403] Description:

[1404] Server: Passes video and audio data to the emotion analysis engine. Uses "FER2013" or "OpenSmile".

[1405] Input: Video and audio data

[1406] Output: Sentiment analysis request

[1407] Server: The emotion analysis engine identifies the user's emotional state from their facial expressions and tone of voice.

[1408] What it does: Analyzes the user's facial expressions in each frame of video and identifies emotions from the tone and intonation of the voice.

[1409] Input: Video and audio data

[1410] Output: Emotion analysis results (e.g., joy, anger, sadness, tension)

[1411] Server: Stores the sentiment analysis results in a database.

[1412] Input: Sentiment analysis results

[1413] Output: Saved sentiment analysis results

[1414] Step 5: Generate Advice

[1415] Description:

[1416] Server: Based on the analysis results and sentiment analysis results, the generative AI model "GPT-4" is used to generate personalized advice.

[1417] Input: Analysis results and sentiment analysis results

[1418] Output: Advice generation request

[1419] Server: Generates specific advice based on the user's requirements.

[1420] Specific actions: Generate practice methods to improve shooting success rate and breathing techniques to relax.

[1421] Input: Advice generation request

[1422] Output: personalized advice

[1423] Server: Sends the generated advice to the user's device.

[1424] Input: personalized advice

[1425] Output: Advice submission request

[1426] Terminal: Receives the advice and notifies the user within the application.

[1427] Input: Advice submission request

[1428] Output: User notification

[1429] Step 6: 24-hour support

[1430] Description:

[1431] Users: Ask questions using the chatbot functionality within the application.

[1432] Input: User question

[1433] Output: Question request

[1434] Terminal: Sends a question to the server.

[1435] Input: Question request

[1436] Output: Request sent to server

[1437] Server: Refers to the emotion engine and knowledge base to generate answers based on the questions.

[1438] Specific actions: Generate appropriate answers from past data and provide specific advice based on sentiment analysis results.

[1439] Input: Question request

[1440] Output: Answer generation request

[1441] Server: Generates the answer and sends it to the user's device.

[1442] Input: Answer generation request

[1443] Output: Reply request

[1444] Terminal: Receives the answer and displays it to the user in real time.

[1445] Input: Reply Send Request

[1446] Output: Real-time display

[1447] Step 7: Improving self-esteem and positivity

[1448] Description:

[1449] Server: Periodically generates self-assessment sheets and questionnaires and sends them to the user's device.

[1450] Input: Self-assessment sheet generation request

[1451] Output: Self-assessment sheet

[1452] Terminal: Displays the received self-assessment sheet or questionnaire to the user and prompts them to respond.

[1453] Input: Self-evaluation sheet

[1454] Output: what is displayed to the user

[1455] User: Answers the self-evaluation sheet and sends the data from the device to the server.

[1456] Input: User response data

[1457] Output: Data sent to the server

[1458] Server: Analyzes the received response data and generates messages and advice to improve positive feelings.

[1459] What it does: Analyzes responses and generates positive messages based on the user's progress and success stories.

[1460] Input: User response data

[1461] Output: Message creation request

[1462] Server: Sends generated messages and advice to the user's terminal.

[1463] Input: Message creation request

[1464] Output: Message send request

[1465] Terminal: Receives messages and advice and displays them to the user.

[1466] Input: Message Send Request

[1467] Output: what is displayed to the user

[1468] (Application example 2)

[1469] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1470] Current factory robots lack feedback for improving work efficiency and maintenance when performance declines or abnormalities occur. It is also difficult to monitor the robot's operating status in real time and take appropriate measures, making it difficult to maintain optimal performance. This creates a risk of a decline in productivity throughout the factory.

[1471] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving game video and audio data, artificial intelligence engine means for analyzing the received data, means for generating personalized advice based on the analysis results, means for providing the generated advice to the user, chatbot means for constantly accepting inquiries from the user, means for providing self-evaluation sheets and questionnaires and analyzing the results, means for generating messages to improve self-esteem based on the analysis results and providing them to the user, means for receiving and analyzing video and audio data of factory robots working, emotion engine means for monitoring the robot's operating performance and abnormalities, and means for generating performance improvement and maintenance advice based on the emotion analysis results. This enables efficient operation of the robot and appropriate maintenance.

[1472] "Game video and audio data" refers to data that refers to video and audio related to a game such as a sport or competition.

[1473] "Means for receiving" refers to a device or method for obtaining and storing particular data.

[1474] An "artificial intelligence engine means" is a device or software that analyzes data using technologies such as machine learning and deep learning and extracts useful information.

[1475] "Personalized advice" refers to specific instructions or suggestions tailored to a particular user's needs and circumstances.

[1476] "Generating means" refers to a device or method for producing specific information or data.

[1477] The "means for providing to the user" refers to a method or device for appropriately notifying a specific user of the generated data or results.

[1478] A "chatbot" is software that accepts user questions and inquiries through automated dialogue and provides appropriate answers.

[1479] "Self-evaluation sheets and questionnaires" refer to lists of questions that allow users to evaluate their own performance and feelings.

[1480] The "analyzing means" refers to a device or method for analyzing the received data in detail and extracting useful information.

[1481] "Self-esteem boosting messages" are positive messages designed to boost a user's confidence and self-esteem.

[1482] A "factory robot" refers to a machine or device that performs work automatically in a factory.

[1483] "Operational performance" is an indicator of how efficiently a machine or system is functioning.

[1484] The "emotion engine means" is a device or software that analyzes video and audio data and identifies emotions and abnormalities therein.

[1485] "Maintenance advice" means specific instructions or suggestions for how to properly maintain or repair a machine or system.

[1486] The present invention is a system for improving the performance of factory robots and supporting their maintenance, and is equipped with an artificial intelligence engine that receives and analyzes game video and audio data, and a means for generating advice based on the analysis results. This system includes the following elements:

[1487] 1. Data Collection

[1488] The server receives work video and audio data from the factory robots. This data is collected using cameras and microphones inside the robots and sent to the server. Specifically, the camera captures the robot's movements, and the microphone records the audio while it is working.

[1489] 2. Data Analysis

[1490] The server analyzes the received video and audio data using an AI analysis engine (e.g., Google Cloud Vision API, AWS Rekognition). As a result of the analysis, metrics such as the robot's operational performance, success rate, and abnormal sound analysis are extracted. These analysis results are stored in a database and managed for each user.

[1491] 3. Emotion analysis

[1492] The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to perform emotion analysis of the received video and audio data. Specifically, it identifies signs of abnormalities or malfunctions from the video and audio of the robot's movements. Based on the results of this analysis, it identifies the robot's state, which requires appropriate action.

[1493] 4. Generating personalized advice

[1494] Based on the results of data analysis and emotion analysis, the server generates specific advice for improving the robot's efficiency and maintenance. This advice includes, for example, how to optimize specific operations and recommendations for regular maintenance. The generated advice is sent to the supervisor's device in real time and displayed.

[1495] 5. 24 / 7 support

[1496] Users (supervisors) can consult the chatbot in real time about robot malfunctions or questions. The server receives the user's question, generates an appropriate answer based on the analysis results and emotion engine data, and sends it to the supervisor's device. This makes it possible to receive appropriate feedback in real time.

[1497] 6. Improved self-esteem and self-confidence

[1498] The server periodically generates self-evaluation sheets and questionnaires and sends them to the supervisor's device. The supervisor answers these and sends the results to the server. Based on the analysis results, the server generates suggestions for improving the robot and positive messages and provides them to the supervisor. This helps the supervisor improve their self-evaluation and confidence in the robot's performance.

[1499] Specific examples

[1500] As a specific example, there is an instruction using the following prompt sentence.

[1501] "Analyze the video and audio of the robot working and provide advice to improve efficiency."

[1502] Based on this prompt, the server collects and analyzes the necessary data and provides appropriate advice to the supervisor's terminal.

[1503] The above is a description of a specific embodiment for carrying out the present invention. This system is expected to improve the performance of factory robots and ensure appropriate maintenance, thereby improving overall productivity.

[1504] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1505] Step 1:

[1506] Data collection

[1507] The server receives work video and audio data from the factory robot's camera and microphone. The input is video and audio data, which is then stored in a database. Specifically, the robot acquires data in real time while in operation and sends it to the server via the network. The server receives the data and stores it in cloud storage.

[1508] Step 2:

[1509] Data analysis

[1510] The server passes the received video and audio data to an AI analysis engine. The input is video and audio data, and the output is robot performance metrics. Specifically, the AI ​​analysis engine (e.g., Google Cloud Vision API, AWS Rekognition) analyzes the data and extracts task success rates, error rates, and other metrics. These analysis results are stored in a database.

[1511] Step 3:

[1512] Emotion analysis

[1513] The server passes the received video and audio data to the emotion engine. The input is video and audio data, and the output is the emotion analysis results that indicate signs of abnormalities or malfunctions while the robot is operating. Specifically, the emotion engine (e.g., IBM Watson Tone Analyzer) identifies abnormal sounds and abnormal machine behavior from the video and audio. These results are also stored in a database.

[1514] Step 4:

[1515] Generate personalized advice

[1516] The server generates advice based on the results of data analysis and emotion analysis. The input is the analysis results and emotion analysis results, and the output is specific improvement advice. Specifically, the server generates robot efficiency measures and maintenance suggestions based on the integrated data, and organizes them in text format. This is then sent to the supervisor's terminal in real time.

[1517] Step 5:

[1518] 24 / 7 support

[1519] Users use the chatbot function to inquire about defects or questions about factory robots at the server. The input is the user's question, and the output is a real-time answer generated via the chatbot. Specifically, the system receives the user's question, references past data and sentiment analysis results, and generates an appropriate answer. The generated answer is then sent via the chatbot to the supervisor's terminal.

[1520] Step 6:

[1521] Improved self-esteem and self-esteem

[1522] The server periodically generates self-evaluation sheets and questionnaires and sends them to the supervisor's terminal. The input is the evaluation sheet generated by the server, and the output is the user's response data. Specifically, the supervisor checks the list and enters their responses. The server analyzes the received response data, generates areas for improvement for the robot, and generates positive messages, and notifies the supervisor.

[1523] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1524] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1525] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1526] [Third embodiment]

[1527] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1528] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1529] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1530] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1531] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1532] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1533] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1534] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1535] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1536] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1537] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1538] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1539] This invention relates to an AI mental trainer system aimed at improving mental health. It receives and analyzes game video and audio data and provides personalized advice. This system allows users to receive efficient and continuous mental support regardless of time or place.

[1540] Program processing explanation

[1541] 1. Data Collection

[1542] User:

[1543] Users use their smartphones or PCs to upload their game video and audio data to the application, providing the system with data that can be later analyzed.

[1544] Device:

[1545] The terminal receives video and audio data from the user and transmits it to the server.

[1546] server:

[1547] The server stores the received data and prepares it for the next analysis process.

[1548] 2. Data Analysis

[1549] server:

[1550] The received game video and audio data is passed to an AI analysis engine, which extracts the user's movements, success rate, and performance metrics, such as shot success rate and running distance.

[1551] server:

[1552] The analysis results are stored in a database and managed for each user.

[1553] 3. Generating personalized advice

[1554] server:

[1555] It generates personalized advice based on the user's past data and current analysis results, including specific practice methods and mental advice.

[1556] server:

[1557] The generated advice is transmitted to the user terminal.

[1558] Device:

[1559] Display the received advice to the user within the application.

[1560] 4. 24 / 7 support

[1561] User:

[1562] Users can access the AI ​​chatbot within the application when they feel anxious before a match or during a crucial moment.

[1563] Device:

[1564] Start the chatbot function and connect to the server.

[1565] server:

[1566] Receives user questions and generates answers by referencing past data and knowledge bases.

[1567] server:

[1568] Answers to questions and advice are sent to the device in real time.

[1569] Device:

[1570] The received response is displayed to the user.

[1571] 5. Improved self-esteem and self-confidence

[1572] server:

[1573] Self-evaluation sheets and questionnaires are generated periodically and sent to the user's terminal.

[1574] Device:

[1575] The received self-evaluation sheet is displayed to the user, and an interface is provided to prompt the user to respond.

[1576] User:

[1577] The user answers the self-evaluation sheet and sends the data from the terminal to the server.

[1578] server:

[1579] The received data is analyzed and messages and advice are generated to improve self-esteem.

[1580] server:

[1581] The generated message or advice is sent to the user terminal.

[1582] Device:

[1583] The received messages and advice are displayed to the user to improve self-evaluation and self-affirmation.

[1584] Specific examples

[1585] 1. Example of Athlete A

[1586] Data collection:

[1587] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[1588] Data Analysis:

[1589] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[1590] Providing personalized advice:

[1591] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[1592] 24 / 7 support:

[1593] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data to provide advice on "strategies that were successful in the previous match."

[1594] Increased self-esteem and self-esteem:

[1595] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[1596] In this way, the system of the present invention provides efficient and continuous mental support to users, thereby improving their performance.

[1597] The processing flow will be explained below.

[1598] Data collection and analysis

[1599] Step 1:

[1600] Users use their smartphones or PCs to record video and audio data of their own matches.

[1601] Step 2:

[1602] Users upload the captured game footage and audio data to the server via an application on their device.

[1603] Step 3:

[1604] The device receives the uploaded video and audio data and sends it to the server.

[1605] Step 4:

[1606] The server stores the received video and audio data and prepares it for the next analysis process.

[1607] Step 5:

[1608] The server passes the video data to an AI analytics engine, which analyzes the data and extracts user behavior, success rate, and performance metrics.

[1609] Step 6:

[1610] The server stores the analysis results in an internal database and manages them for each user.

[1611] Generate personalized advice

[1612] Step 1:

[1613] The server prepares data to generate personalized advice based on the user's past performance data and current analysis results.

[1614] Step 2:

[1615] The server's AI engine analyzes the collected data and generates practice methods and mental advice tailored to the user.

[1616] Step 3:

[1617] The server transmits the generated advice to the user's terminal.

[1618] Step 4:

[1619] The terminal displays the received advice to the user within the application.

[1620] Step 5:

[1621] The user checks the displayed advice and prepares to proceed to the next phase.

[1622] 24 / 7 support

[1623] Step 1:

[1624] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[1625] Step 2:

[1626] The terminal displays the chatbot screen and connects to the server.

[1627] Step 3:

[1628] Users input questions or inquiries to the chatbot.

[1629] Step 4:

[1630] The server receives questions and inquiries from users and generates answers by referencing past data and a knowledge base.

[1631] Step 5:

[1632] The server sends the generated answers and advice to the user's terminal.

[1633] Step 6:

[1634] The terminal displays the received answers and advice to the user in real time.

[1635] Step 7:

[1636] The server stores the chat history with the user in a database, which can be used for future counseling.

[1637] Improved self-esteem and self-esteem

[1638] Step 1:

[1639] The server periodically generates self-evaluation sheets and questionnaires and sends them to the user's terminal.

[1640] Step 2:

[1641] The terminal displays the received self-evaluation sheet to the user and provides an interface that prompts the user to answer.

[1642] Step 3:

[1643] The user answers the self-evaluation sheet and transmits the data from the terminal to the server.

[1644] Step 4:

[1645] The server receives the data from the self-assessment sheet, and the AI ​​engine performs the analysis.

[1646] Step 5:

[1647] Based on the analysis results, the server generates positive messages and advice to improve self-esteem.

[1648] Step 6:

[1649] The server sends the generated message or advice to the user's terminal.

[1650] Step 7:

[1651] The device displays the received messages and advice to the user, encouraging them to improve their self-esteem.

[1652] Example 1

[1653] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1654] Conventional mental training systems have had issues with insufficient collection and analysis of user performance data, making it difficult to provide personalized advice. Furthermore, they lacked the functionality to accept consultations from users in real time 24 hours a day, 365 days a year, leaving insufficient means for improving self-evaluation and self-affirmation.

[1655] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1656] In this invention, the server includes: means for receiving game video and audio data; artificial intelligence engine means for analyzing the received data; means for generating personalized advice based on the analysis results; means for providing the generated advice to the user; chatbot means for always accepting consultations from users; means for providing self-assessment sheets and questionnaires and analyzing the results; means for generating messages to improve self-esteem based on the analysis results and providing them to the user; means for the user to upload game video and audio data using the terminal; means for saving the data and preparing for the next analysis process; means for passing the data to the AI ​​analysis engine and extracting performance metrics; means for saving the performance metrics in a database and managing them for each user; means for sending the generated advice to the user terminal in real time; means for displaying the received advice on the user terminal; means for activating the AI ​​chatbot function and connecting to the server; means for using an artificial intelligence model to generate optimal answers to the user's questions and sending them to the terminal; means for the user to access the chatbot when they feel anxious before a game; and means for generating self-assessment sheets and questionnaires for the user and sending them to the terminal. This enables detailed analysis of users' performance data and the provision of efficient and personalized advice, providing real-time mental support and improving self-evaluation 24 hours a day, 365 days a year.

[1657] "Game video and audio data" refers to video and audio data recorded by a user during a sport or activity.

[1658] The "receiving means" refers to a function or module for acquiring data transmitted from a user terminal.

[1659] The "analytical artificial intelligence engine means" is a system that analyzes received data using deep learning or machine learning to extract important performance metrics.

[1660] A "means for generating personalized advice" is an algorithm or module for providing advice tailored to the individual needs of each user.

[1661] The "means for providing advice to the user" is a system for displaying and notifying the generated advice on the user's terminal.

[1662] A "chatbot" is a conversation system that uses artificial intelligence to accept questions and inquiries from users and automatically respond.

[1663] "Means for providing self-evaluation sheets and questionnaires" refers to a system that provides questions and forms for users to request self-evaluation and feedback.

[1664] The "means for analyzing the results" refers to algorithms and modules for analyzing the user's self-evaluation sheets and questionnaire responses.

[1665] The "means for generating messages to improve self-esteem" is a system that generates messages to improve a user's self-esteem and motivation based on the results of self-evaluation.

[1666] "Means for uploading" refers to the means by which a user sends game video and audio data to the system, typically via an application or web portal.

[1667] The "means for storing data" is a storage system for safely and efficiently storing received game video and audio data.

[1668] "Means for preparing for analysis processing" refers to a system that performs the necessary data preprocessing and formatting before analyzing the data.

[1669] "Means for extracting performance metrics" refers to a function that extracts important indicators of athletic performance (e.g., shooting success rate, distance traveled, etc.) from the analyzed data.

[1670] "Means for managing performance metrics" refers to a system that organizes, stores, and manages extracted indicators for each user.

[1671] The "means for transmitting in real time" is a communication system for instantly transmitting the generated advice or message to the user's terminal.

[1672] "Means for activating AI chatbot functions" refers to a system that activates chatbot functions in response to user requests.

[1673] "Means for generating optimal answers using an artificial intelligence model" refers to an AI module that generates optimal answers to user questions by referencing past data and knowledge bases.

[1674] The "means for transmitting to the terminal" is a system that transmits the generated answers and advice to the user's terminal.

[1675] "Means to access a chatbot when feeling anxious" is a function that allows users to access a chatbot and consult with it when they feel psychological anxiety.

[1676] "Means for generating self-assessment sheets and questionnaires" refers to a system that creates questions and forms for users to periodically conduct self-assessments.

[1677] The program in this system is an AI mental trainer system designed to improve the user's mental health. This system receives and analyzes game video and audio data and provides personalized advice. Specific embodiments of this system are described below.

[1678] Data collection

[1679] User:

[1680] Users can upload their game video and audio data to the application using their smartphone or PC. For example, they can send video files saved on their smartphone via the app's upload function.

[1681] Device:

[1682] The device (user's smartphone or PC) receives the uploaded video and audio data and sends it to the server. The data is designed to be sent to the server via a network connection.

[1683] server:

[1684] The server stores the received video and audio data in a database and prepares it for the next analysis process. The stored data is used for analysis by the AI ​​analysis engine, which will be described later.

[1685] Data analysis

[1686] server:

[1687] The server passes the received data to an AI analytics engine, such as TensorFlow or PyTorch, which uses deep learning techniques, to extract performance metrics (e.g., shooting percentage, distance traveled), which are then stored in a database.

[1688] server:

[1689] The analysis results are managed for each user and used for subsequent processing as needed.

[1690] Generate personalized advice

[1691] server:

[1692] The server uses a generative AI model to generate personalized advice based on the user's past data and analysis results. For example, it uses a machine learning model to generate "five practice methods to improve shooting accuracy."

[1693] server:

[1694] The generated advice is sent to the user terminal in real time via network communication.

[1695] Device:

[1696] The device then displays the received advice to the user within the application. The advice can be provided not only as text information, but also as video or audio guides.

[1697] 24 / 7 support

[1698] User:

[1699] Before a match or when feeling stressed, users can access the AI ​​chatbot within the app by opening the app's chatbot function and typing in a question.

[1700] Device:

[1701] The chatbot function is activated and connected to the server, where real-time communication takes place.

[1702] server:

[1703] When a user asks a question, it uses past data and a knowledge base to generate the best answer, often using a natural language processing (NLP) model or, in some cases, a generative AI model like GPT-3.

[1704] server:

[1705] The generated answer is sent to the user terminal.

[1706] Device:

[1707] The terminal displays the received answers to the user, and responds quickly to the user's concerns and questions.

[1708] Improved self-esteem and self-esteem

[1709] server:

[1710] Self-assessment sheets and questionnaires are periodically generated and sent to the user's device. This process also uses machine learning models to generate questions that are appropriate for the user.

[1711] Device:

[1712] The device displays the received self-assessment sheet to the user within the app, providing an interface that allows the user to easily respond.

[1713] User:

[1714] The user answers the self-evaluation sheet and sends the data from the terminal to the server. All the user has to do is enter the details of their self-evaluation and press the send button.

[1715] server:

[1716] The system analyzes the received evaluation data and generates messages and advice to improve the user's self-esteem. For example, it generates positive messages such as "Your performance has improved dramatically this month! Keep it up!"

[1717] server:

[1718] The generated message or advice is sent to the user terminal.

[1719] Device:

[1720] The device will display the received messages and advice to the user, aiming to improve their motivation and self-esteem. Notifications will be sent in the form of push notifications or in-app messages.

[1721] Specific examples

[1722] 1. Example of Athlete A

[1723] Data collection:

[1724] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[1725] Data Analysis:

[1726] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[1727] Providing personalized advice:

[1728] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[1729] 24 / 7 support:

[1730] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data to provide advice on "strategies that were successful in the previous match."

[1731] Increased self-esteem and self-esteem:

[1732] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[1733] Prompt Sentence Examples

[1734] "What is your shooting percentage in last month's games?"

[1735] "What do you need to improve in practice this week?"

[1736] "How can I reduce the anxiety I feel before a match?"

[1737] In this way, the system of the present invention provides efficient and continuous mental support to users, thereby improving their performance.

[1738] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1739] Step 1:

[1740] User:

[1741] Users use their smartphones or PCs to upload their own game video and audio data to the application by using the app's upload function to select the data and then pressing the send button.

[1742] Input: Game video and audio data

[1743] Output: The data file received by the application

[1744] Step 2:

[1745] Device:

[1746] The device (user's smartphone or PC) receives the uploaded video and audio data and sends it to the server via the Internet. A network connection must be established.

[1747] Input: User uploaded data

[1748] Output: Data packet sent to the server

[1749] Step 3:

[1750] server:

[1751] The server stores the received video and audio data in a database. Before storing the data, it checks it to ensure that it is not corrupted or invalid.

[1752] Input: Data sent from the terminal

[1753] Output: Data stored in the database

[1754] Step 4:

[1755] server:

[1756] The stored data is passed to an AI analytics engine for analysis, which uses deep learning techniques (such as TensorFlow or PyTorch) to extract performance metrics.

[1757] Input: Video and audio data stored in a database

[1758] Output: Extracted performance metrics (e.g., shooting percentage, distance traveled)

[1759] Step 5:

[1760] server:

[1761] The analysis results are saved in a database and managed for each user. The saved data can also be used as user history data.

[1762] Input: Performance metrics

[1763] Output: Analysis results stored in a database

[1764] Step 6:

[1765] server:

[1766] Using a generative AI model, the system generates personalized advice based on the user's past data and the latest analysis results. For example, if a user's shooting success rate is low, the system generates "practice methods to improve shooting accuracy."

[1767] Input: Historical data and latest analysis results

[1768] Output: Personalized advice

[1769] Step 7:

[1770] server:

[1771] The generated advice is sent to the user's device via the network, and notifications are also sent in real time.

[1772] Enter: personalized advice

[1773] Output: Advice sent to the user's terminal

[1774] Step 8:

[1775] Device:

[1776] The device displays the received advice to the user within the application. Advice may be provided not only as text information, but also as video or audio guides.

[1777] Input: Advice sent by the server

[1778] Output: Advice displayed to the user

[1779] Step 9:

[1780] User:

[1781] Before a match or when feeling stressed, users can access the AI ​​chatbot within the application and input their questions or concerns, and receive support in an interactive format using the chatbot's functionality.

[1782] Input: User questions and inquiries

[1783] Output: The text entered into the chatbot

[1784] Step 10:

[1785] Device:

[1786] The chatbot function is activated and the entered questions or inquiries are sent to the server. Since the connection is real-time, a low-latency network environment is recommended.

[1787] Input: User questions and inquiries

[1788] Output: Questions and queries sent to the server

[1789] Step 11:

[1790] server:

[1791] It uses past data and knowledge bases to generate optimal answers using artificial intelligence models (e.g., GPT-3). Depending on the question, it may refer to past match data or analysis results.

[1792] Input: User questions and knowledge base

[1793] Output: The generated answer

[1794] Step 12:

[1795] server:

[1796] The generated answer is sent to the user's device in real time, and the user is immediately notified via a network connection.

[1797] Input: Generated answer

[1798] Output: Answer sent to user's terminal

[1799] Step 13:

[1800] Device:

[1801] The device then displays the received answers to the user, who can use them to solve problems and reduce stress.

[1802] Input: The answer sent by the server

[1803] Output: The answer that is displayed to the user

[1804] Step 14:

[1805] server:

[1806] Self-assessment sheets and questionnaires are generated periodically and sent to the user's device. Questions appropriate for the user are selected using a machine learning model.

[1807] Input: User history and rating data

[1808] Output: Generated self-assessment sheets and questionnaires

[1809] Step 15:

[1810] Device:

[1811] It provides an interface that displays self-evaluation sheets and questionnaires to users and encourages them to respond, allowing users to input their answers intuitively.

[1812] Input: Self-assessment sheets and surveys sent from the server

[1813] Output: Questions displayed to the user

[1814] Step 16:

[1815] User:

[1816] Users answer a self-assessment form and the data is sent from their device to a server, which may be designed to maintain anonymity.

[1817] Input: User's answer

[1818] Output: Response data sent to the server

[1819] Step 17:

[1820] server:

[1821] The system analyzes the received self-evaluation data and generates messages and advice to improve the user's self-esteem, placing emphasis on positive feedback.

[1822] Input: User response data

[1823] Output: any messages or advice generated

[1824] Step 18:

[1825] server:

[1826] The generated messages and advice are sent to the user's terminal in real time, and the user is notified immediately.

[1827] Input: Generated messages and advice

[1828] Output: Messages and advice sent to the user's terminal

[1829] Step 19:

[1830] Device:

[1831] The device will display the received messages and advice to the user. Messages will be sent in the form of push notifications or in-app messages.

[1832] Input: Messages and advice sent by the server

[1833] Output: Messages and advice displayed to the user

[1834] In this way, the system of the present invention provides efficient and continuous mental support to the user through the specific actions at each step, thereby improving performance.

[1835] (Application example 1)

[1836] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1837] In today's food delivery industry, delivery workers are experiencing increasing stress and anxiety during their daily work, which can lead to a decline in work efficiency and mental health. Furthermore, the lack of personalized support tailored to each delivery worker means that improvements in mental health and performance are not being linked. Given this background, there is a need for a system that provides efficient and continuous mental support to delivery workers, balancing work efficiency and mental health.

[1838] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1839] In this invention, the server includes means for receiving game video and audio data, artificial intelligence engine means for analyzing the received data, means for generating personalized advice based on the analysis results, means for providing the generated advice to the user, chatbot means for constantly accepting inquiries from the user, means for providing self-evaluation sheets and questionnaires and analyzing the results, means for generating messages to improve self-esteem based on the analysis results and providing them to the user, means for recording guidance and voice memos for delivery personnel during deliveries, means for analyzing the recorded data to extract delivery performance and stress factors, and means for generating advice for work improvement and mental support based on the extraction results. This allows for efficient and continuous mental support to be provided to delivery personnel, enabling improved work efficiency and improved mental health.

[1840] The "means for receiving game video and audio data" refers to a device or system for receiving video and audio recorded by a user.

[1841] The "artificial intelligence engine means for analyzing received data" is an analytical engine that uses artificial intelligence to analyze received video and audio data and extract useful information.

[1842] The "means for generating personalized advice based on the analysis results" refers to a device or system for customizing and generating optimal advice for a user based on the analyzed data.

[1843] The "means for providing generated advice to a user" is a device or system for providing customized advice to a user.

[1844] The "chatbot means that always accepts inquiries from users" is a chatbot that always accepts questions and inquiries from users and automatically responds to them.

[1845] The "means for providing a self-assessment sheet or questionnaire and analyzing the results" refers to a device or system for providing a self-assessment sheet or questionnaire to users and analyzing the response results.

[1846] The "means for generating and providing a user with a message that will improve self-esteem based on the analysis results" refers to a device or system for generating and providing a user with a message that will improve self-esteem based on the analyzed evaluation data.

[1847] The "means for the delivery person to record guidance and voice memos during delivery" refers to a device or system for recording guidance and voice memos during delivery.

[1848] The "means for analyzing recorded data to extract delivery performance and stress factors" refers to a device or system for analyzing recorded guidance and voice memos to identify delivery performance and stress factors.

[1849] The "means for generating advice for business improvement and mental support based on the extraction results" refers to a device or system for generating advice for business improvement and mental support based on the analyzed data.

[1850] As an embodiment of the present invention, a system for improving the mental health of food delivery workers is constructed as follows.

[1851] 1. Data Collection:

[1852] User: Delivery personnel use their smartphones to record guidance and voice memos during deliveries, which allows data on daily work and delivery situations to be accumulated.

[1853] Device: The smartphone transmits the recorded data to the server. The smartphone is equipped with recording and data transmission functions.

[1854] 2. Data Analysis:

[1855] Server: Uses an AI analysis engine to analyze the received guidance and voice data. The analysis engine identifies delivery performance (e.g., delivery time, success rate) and stress factors (e.g., stress word analysis in the voice).

[1856] The specific software used includes voice recognition technology and machine learning models, such as the Google Cloud Speech-to-Text API and TensorFlow models.

[1857] 3. Advice Generation:

[1858] Server: Generates personalized advice based on the analysis results, including specific suggestions for work improvement and mental support.

[1859] Device: Advice is sent to the smartphone and displayed to the user using notifications and dashboard features.

[1860] 4. Real-time support:

[1861] Users: When they feel anxious or stressed during a delivery, they can use the in-app chatbot for advice.

[1862] Terminal: The chatbot function is activated and connected to the server.

[1863] Server: The chatbot references historical data and a knowledge base to provide real-time support.

[1864] 5. Self-assessment and feedback:

[1865] Server: Periodically generates self-evaluation sheets and questionnaires and sends them to the delivery person's smartphone.

[1866] Terminal: The delivery person answers a self-evaluation sheet and sends the data to the server.

[1867] Server: Based on the analyzed data, it generates and sends a self-esteem-boosting message to the delivery person, for example, providing feedback such as "Your stress level has improved by 30% over the past month."

[1868] For example, when delivery person A drives a new route, he can check his smartphone for advice on how to relieve stress and how to deliver efficiently. Based on past data, the AI ​​chatbot will respond by saying, "Since you were able to reduce the time by 15 minutes on your last route, we'll suggest ways to reduce stress this time as well."

[1869] Example prompt sentence:

[1870] "Analyze delivery drivers' voice guidance and provide personalized advice on stress factors and how to improve their work."

[1871] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1872] Step 1:

[1873] Data collection:

[1874] Input: Guidance and voice memos recorded by delivery personnel using their smartphones during deliveries

[1875] Specific behavior:

[1876] User: While making a delivery, the delivery person uses the recording function on their smartphone to record the immediate situation, thoughts, and emotions while driving as voice memos.

[1877] Device: The smartphone temporarily stores the recorded audio data and sends it to the server using mobile data.

[1878] Output: Audio data sent to the server

[1879] Step 2:

[1880] Data received and stored on the server:

[1881] Input: Audio data from the device

[1882] Specific behavior:

[1883] Server: Receives the voice data and stores it in a database, tagged with the date and delivery person ID.

[1884] Output: Saved audio data

[1885] Step 3:

[1886] Data Analysis:

[1887] Input: Stored audio data

[1888] Specific behavior:

[1889] Server: Passes the voice data to an AI analytics engine, which uses speech recognition technology to convert it into text and extracts stress factors and performance metrics, such as detecting specific keywords and phrases that indicate stress, and calculating delivery times and delivery success rates.

[1890] Tools used: Google Cloud Speech-to-Text API and TensorFlow model

[1891] Output: Analysis results (stress factors, delivery performance metrics)

[1892] Step 4:

[1893] Generate personalized advice:

[1894] Input: Analysis results

[1895] Specific behavior:

[1896] Server: Based on the analysis results, it generates advice for business improvement and mental support. For example, it generates specific advice such as "Based on the data from the past month, it would be better to choose off-peak times to further shorten delivery times."

[1897] Output: Personalized advice

[1898] Step 5:

[1899] Providing advice:

[1900] Enter: personalized advice

[1901] Specific behavior:

[1902] Server: Sends the generated advice to the delivery person's smartphone.

[1903] Device: The smartphone displays the received advice to the user via the notification function.

[1904] Output: Advice displayed on the delivery person's smartphone

[1905] Step 6:

[1906] Real-time support:

[1907] Input: Message from delivery person

[1908] Specific behavior:

[1909] User: If they have any concerns or questions during delivery, they can send a consultation message using the chatbot function on their smartphone.

[1910] Terminal: The smartphone sends a consultation message to the server.

[1911] Server: The chatbot references historical data and a knowledge base to generate real-time responses, such as "Your last route saved you 15 minutes, so try a similar route this time."

[1912] Output: Real-time response from the chatbot

[1913] Step 7:

[1914] Self-assessment and feedback:

[1915] Input: Self-evaluation sheet from the server, delivery person's evaluation answers

[1916] Specific behavior:

[1917] Server: Periodically sends self-evaluation sheets and surveys to delivery workers' smartphones, including questions about stress levels and work performance.

[1918] User: Answer each assessment item and submit the data from their smartphone.

[1919] Server: Analyzes the received evaluation data and generates a feedback message, such as providing positive feedback like "Your stress level has improved by 30% over the past month."

[1920] Output: Feedback message displayed on the delivery person's smartphone

[1921] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1922] This invention relates to an AI mental trainer system aimed at improving mental health. It receives and analyzes game video and audio data to provide personalized advice, and also incorporates an emotion engine that recognizes the user's emotions. This makes it possible to provide more appropriate mental support based on the user's needs and emotional state.

[1923] Program processing explanation

[1924] 1. Data Collection

[1925] User:

[1926] Users use their smartphones or PCs to upload their game video and audio data to the application, providing the system with data that can be later analyzed.

[1927] Device:

[1928] The terminal receives video and audio data from the user and transmits it to the server.

[1929] server:

[1930] The server stores the received data and prepares it for the next analysis process.

[1931] 2. Data Analysis

[1932] server:

[1933] The received game video and audio data is passed to an AI analysis engine, which extracts user actions, success rates, and performance metrics.

[1934] server:

[1935] The analysis results are stored in a database and managed for each user.

[1936] 3. Emotion analysis

[1937] server:

[1938] The received video and audio data is passed to the emotion engine, which analyzes the user's facial expressions and voice to determine their emotional state.

[1939] server:

[1940] Based on the analysis results of the emotion engine, data is prepared to generate more appropriate advice.

[1941] 4. Generating personalized advice

[1942] server:

[1943] It generates personalized advice based on the user's past performance data, current analysis results, and emotion engine analysis results.

[1944] server:

[1945] The generated advice is sent to the user's terminal.

[1946] Device:

[1947] Display the received advice to the user within the application.

[1948] 5. 24 / 7 support

[1949] User:

[1950] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[1951] Device:

[1952] Start the chatbot function and connect to the server.

[1953] server:

[1954] Receives a user question and generates an answer by referencing the emotion engine, past data, and knowledge base.

[1955] server:

[1956] Answers to questions and advice are sent to the user's device.

[1957] Device:

[1958] The received answers are displayed to the user in real time.

[1959] server:

[1960] The chat history with the user is saved in a database, which can be used for future counseling.

[1961] 6. Improved self-esteem and self-confidence

[1962] server:

[1963] Self-evaluation sheets and questionnaires are generated periodically and sent to the user's device.

[1964] Device:

[1965] The received self-evaluation sheet is displayed to the user, and an interface is provided to prompt the user to respond.

[1966] User:

[1967] The user answers the self-evaluation sheet and sends the data from the terminal to the server.

[1968] server:

[1969] The received data is analyzed and messages and advice are generated to improve self-esteem.

[1970] server:

[1971] The generated message or advice is sent to the user's terminal.

[1972] Device:

[1973] The received messages and advice are displayed to the user, encouraging them to improve their self-esteem.

[1974] Specific examples

[1975] 1. Example of Athlete A

[1976] Data collection:

[1977] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[1978] Data Analysis:

[1979] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[1980] Emotion analysis:

[1981] The server passes the video data to the emotion engine, which identifies pressure or anxiety from Player A's facial expressions and voice. For example, it analyzes that Player A is nervous before a match.

[1982] Providing personalized advice:

[1983] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[1984] 24 / 7 support:

[1985] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data and sentiment analysis results to provide advice on "strategies that were successful in the previous match."

[1986] Increased self-esteem and self-esteem:

[1987] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[1988] In this way, the system of the present invention provides users with efficient and continuous mental support, improving their performance. By adding an emotion engine, it becomes possible to provide more appropriate advice and support according to the user's emotional state.

[1989] The processing flow will be explained below.

[1990] MODE FOR CARRYING OUT THE INVENTION (DETAILED PROCESSING FLOW)

[1991] Data collection

[1992] Step 1:

[1993] Users use their smartphones or PCs to record video and audio data of their own matches.

[1994] Step 2:

[1995] Users upload the captured game footage and audio data to the server via an application on their device.

[1996] Step 3:

[1997] The device receives the uploaded video and audio data and sends it to the server.

[1998] Step 4:

[1999] The server stores the received video and audio data and prepares it for the next analysis process.

[2000] Data analysis

[2001] Step 5:

[2002] The server passes the video data to an AI analytics engine, which analyzes the data and extracts user behavior, success rate, and performance metrics.

[2003] Step 6:

[2004] The server stores the analysis results in an internal database and manages them for each user.

[2005] Emotion analysis

[2006] Step 7:

[2007] The server passes the video and audio data to the emotion engine, which analyzes the user's facial expressions and voice to determine their emotional state.

[2008] Step 8:

[2009] The server prepares data to generate more appropriate advice based on the analysis results of the emotion engine.

[2010] Generate personalized advice

[2011] Step 9:

[2012] The server generates personalized advice based on the user's past performance data, current analysis results, and the analysis results of the emotion engine.

[2013] Step 10:

[2014] The server transmits the generated advice to the user's terminal.

[2015] Step 11:

[2016] The terminal displays the received advice to the user within the application.

[2017] Step 12:

[2018] The user checks the displayed advice and prepares to proceed to the next phase.

[2019] 24 / 7 support

[2020] Step 13:

[2021] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[2022] Step 14:

[2023] The terminal displays the chatbot screen and connects to the server.

[2024] Step 15:

[2025] Users input questions or inquiries to the chatbot.

[2026] Step 16:

[2027] The server receives questions and inquiries from users, references past data, emotion engine results, and a knowledge base, and generates answers.

[2028] Step 17:

[2029] The server sends the generated answers and advice to the user's terminal.

[2030] Step 18:

[2031] The terminal displays the received answers and advice to the user in real time.

[2032] Step 19:

[2033] The server stores the chat history with the user in a database, which can be used for future counseling.

[2034] Improved self-esteem and self-esteem

[2035] Step 20:

[2036] The server periodically generates self-evaluation sheets and questionnaires and sends them to the user's terminal.

[2037] Step 21:

[2038] The terminal displays the received self-evaluation sheet to the user and provides an interface that prompts the user to answer.

[2039] Step 22:

[2040] The user answers the self-evaluation sheet and transmits the data from the terminal to the server.

[2041] Step 23:

[2042] The server receives the data from the self-assessment sheet, and the AI ​​engine performs the analysis.

[2043] Step 24:

[2044] Based on the analysis results, the server generates positive messages and advice to improve self-esteem.

[2045] Step 25:

[2046] The server sends the generated message or advice to the user's terminal.

[2047] Step 26:

[2048] The device displays the received messages and advice to the user, encouraging them to improve their self-esteem.

[2049] Specific examples

[2050] Example of Athlete A

[2051] Step 1:

[2052] Athlete A uploads game footage from his smartphone to the app.

[2053] Step 2:

[2054] The terminal receives the video and transmits it to the server.

[2055] Step 3:

[2056] The server receives the video and passes it to the AI ​​analysis engine.

[2057] Step 4:

[2058] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[2059] Step 5:

[2060] The server passes the video data to the emotion engine, which identifies pressure or anxiety from Player A's facial expressions and voice. For example, it analyzes that Player A is nervous before a match.

[2061] Step 6:

[2062] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[2063] Step 7:

[2064] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data and sentiment analysis results to provide advice on "strategies that were successful in the previous match."

[2065] Step 8:

[2066] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[2067] In this way, the system of the present invention, which is combined with an emotion engine, provides efficient and continuous mental support to the user, and realizes more appropriate advice and support according to the user's emotional state.

[2068] Example 2

[2069] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2070] Conventional mental care systems have difficulty analyzing users' emotional states and performance in real time and providing appropriate, personalized advice. Furthermore, there has been a lack of systems that provide comprehensive, continuous support for 24 / 7 support and for improving self-esteem based on self-evaluation. Therefore, there is a need for an effective system that can sustainably improve users' mental health and performance.

[2071] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2072] In this invention, the server includes a device for receiving game video and audio data, an intelligence engine for analyzing the received data, a device for generating personalized advice based on the analysis results, a device for providing the generated advice to users, a conversational bot device for constantly accepting consultations from users, an engine for analyzing emotional states, a device for providing self-evaluation sheets and questionnaires and analyzing the results, and a device for generating messages to improve positivity based on the analysis results and providing them to users. This makes it possible to analyze users' emotional states and performance in real time and provide personalized, appropriate advice. It also provides comprehensive and continuous support 24 hours a day, 365 days a year, and helps improve self-esteem based on self-evaluation.

[2073] The "device for receiving game video and audio data" is a device for electronically receiving game video and audio data provided by a user.

[2074] The "intelligence engine" is an analytical device that uses artificial intelligence to analyze received game video and audio data.

[2075] The "device for generating personalized advice" is a device that generates advice optimized for each user based on the analysis results.

[2076] The "device for providing generated advice to a user" is a device for providing generated advice to a user in a format that is easy to use.

[2077] A "conversation bot device" is an automatic response device that accepts inquiries from users 24 hours a day, 365 days a year, and provides answers in an interactive format.

[2078] The "emotional state analysis engine" is a device for analyzing the user's emotional state from received video and audio data.

[2079] The "device for providing a self-assessment sheet or questionnaire and analyzing the results" is a device that provides a self-assessment sheet or questionnaire to a user and analyzes the user's responses.

[2080] The "device for generating and providing users with messages that enhance positive self-esteem" is a device for generating and providing users with messages that enhance positive self-esteem based on the analysis results of self-evaluations and questionnaires.

[2081] This invention relates to an AI mental trainer system aimed at improving mental health. It receives and analyzes various data and provides personalized advice. It also incorporates an emotion engine that analyzes the user's emotional state. Below, we explain how this system is specifically implemented.

[2082] First, the system consists of the following main components:

[2083] 1. Data Collection Equipment

[2084] User: Uses a smartphone or PC to upload video and audio data from their match to a dedicated app.

[2085] Terminal: Receives data, compresses it if necessary, and sends it to the server. This process uses software that recognizes and processes specific data formats, such as MP4 for video data and WAV for audio data.

[2086] 2. Intelligence Engine

[2087] Server: Receives game footage and audio data stored in the database and analyzes the movement and audio characteristics using analysis tools such as "OpenPose" and "Librosa." The analysis results include the user's movement success rate and level of tension.

[2088] 3. Emotional state analysis engine

[2089] Server: Analyzes video and audio data using emotion analysis tools such as "FER2013" and "OpenSmile," and identifies emotions from the user's facial expressions and tone of voice.

[2090] 4. Device for generating personalized advice

[2091] Server: Based on the analysis results, the generative AI model GPT-4 is used to generate advice tailored to each user, such as practice methods to improve the user's shooting success rate or breathing techniques to help them relax.

[2092] 5. Device for Providing Generated Advice to Users

[2093] Server: Generates advice and sends it to the user's device.

[2094] Terminal: Receives the advice and displays it to the user within the application.

[2095] 6. Conversational Bot Device

[2096] Users: Use the chatbot feature within the application to ask questions before a game, after practice, etc.

[2097] Terminal: Receives the query and sends it to the server.

[2098] Server: Refers to the emotion engine and knowledge base to generate appropriate answers and send them to the user.

[2099] Terminal: Shows the answer to the user in real time.

[2100] 7. Self-evaluation sheets and questionnaires providing device

[2101] Server: Periodically generates self-assessment sheets and questionnaires and sends them to the user's device.

[2102] Terminal: Displays the received self-assessment sheet or questionnaire to the user and prompts them to respond.

[2103] User: Answers and sends the data from the device to the server.

[2104] Server: Analyzes the received data, generates messages and advice to improve positive feelings, and provides them to the user.

[2105] Specific examples

[2106] The following explanation will be given using athlete A as an example.

[2107] 1. Data collection: Athlete A uploads game footage from his smartphone to the app. The device receives the footage and sends it to the server. The server then passes the footage to the AI ​​analysis engine.

[2108] 2. Data analysis: The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. This analysis result is stored in a database.

[2109] 3. Emotion analysis: The server passes the video data to the emotion engine, which identifies pressure and anxiety from Player A's facial expressions and voice. For example, it analyzes that Player A is nervous before a match.

[2110] 4. Providing advice: Based on the analysis results, the server generates "Five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[2111] 5. 24-hour support: Player A feels anxious before a match and consults the chatbot within the app. The server uses past data and sentiment analysis results to provide advice on "strategies that were successful in the previous match."

[2112] 6. Self-evaluation: At the end of the month, the server sends a self-evaluation sheet to Player A, who answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[2113] This allows the system of the present invention to provide users with efficient and continuous mental support, improving their performance. By adding an emotion engine, it becomes possible to provide advice and support according to the user's emotional state.

[2114] (Example of a prompt)

[2115] "What are some specific ways to ease pre-game tension?"

[2116] "Please tell me some effective practice methods to improve my shooting percentage."

[2117] "What are the sentiment analysis results for recent games?"

[2118] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2119] Step 1: Upload your data

[2120] Description:

[2121] Users: Use their smartphones or PCs to upload their own game video and audio data to a dedicated application.

[2122] Input: Game video (file format: MP4) and audio data (file format: WAV)

[2123] Output: Data transmission request for receiving

[2124] Device: Validate the file format of the received video and audio data to ensure it is in the correct format.

[2125] Specific operation: Verify that the video data is in MP4 format and the audio data is in WAV format.

[2126] Input: Game video and audio data

[2127] Output: Validated data

[2128] Terminal: Compresses the verified data and transfers it to the server.

[2129] What it does: Improves transfer speeds by compressing data.

[2130] Input: Validated data

[2131] Output: Compressed data and transfer request

[2132] Step 2: Save your data

[2133] Description:

[2134] Server: Receives compressed data sent from the device and stores it in a database.

[2135] Input: Compressed data

[2136] Output: Data save confirmation

[2137] Server: Creates and properly manages the directory of stored data.

[2138] What it does: Creates a directory in the database and tags the data.

[2139] Input: Compressed data

[2140] Output: Data directory information

[2141] Step 3: Data analysis

[2142] Description:

[2143] Server: Passes saved game footage and audio data to the AI ​​analysis engine. Uses "OpenPose" or "Librosa."

[2144] Input: Data directory information

[2145] Output: Analysis request and target data

[2146] Server: The AI ​​analysis engine identifies motion characteristics from video and audio events from audio.

[2147] Specific actions: Analyze users' shooting success rate and frequency of shouting.

[2148] Input: Video and audio data

[2149] Output: Analysis results (shooting success rate, frequency of calls, etc.)

[2150] Server: Stores the analysis results in a database.

[2151] Input: Analysis results

[2152] Output: Saved analysis results

[2153] Step 4: Sentiment Analysis

[2154] Description:

[2155] Server: Passes video and audio data to the emotion analysis engine. Uses "FER2013" or "OpenSmile".

[2156] Input: Video and audio data

[2157] Output: Sentiment analysis request

[2158] Server: The emotion analysis engine identifies the user's emotional state from their facial expressions and tone of voice.

[2159] What it does: Analyzes the user's facial expressions in each frame of video and identifies emotions from the tone and intonation of the voice.

[2160] Input: Video and audio data

[2161] Output: Emotion analysis results (e.g., joy, anger, sadness, tension)

[2162] Server: Stores the sentiment analysis results in a database.

[2163] Input: Sentiment analysis results

[2164] Output: Saved sentiment analysis results

[2165] Step 5: Generate Advice

[2166] Description:

[2167] Server: Based on the analysis results and sentiment analysis results, the generative AI model "GPT-4" is used to generate personalized advice.

[2168] Input: Analysis results and sentiment analysis results

[2169] Output: Advice generation request

[2170] Server: Generates specific advice based on the user's requirements.

[2171] Specific actions: Generate practice methods to improve shooting success rate and breathing techniques to relax.

[2172] Input: Advice generation request

[2173] Output: personalized advice

[2174] Server: Sends the generated advice to the user's device.

[2175] Input: personalized advice

[2176] Output: Advice submission request

[2177] Terminal: Receives the advice and notifies the user within the application.

[2178] Input: Advice submission request

[2179] Output: User notification

[2180] Step 6: 24-hour support

[2181] Description:

[2182] Users: Ask questions using the chatbot functionality within the application.

[2183] Input: User question

[2184] Output: Question request

[2185] Terminal: Sends a question to the server.

[2186] Input: Question request

[2187] Output: Request sent to server

[2188] Server: Refers to the emotion engine and knowledge base to generate answers based on the questions.

[2189] Specific actions: Generate appropriate answers from past data and provide specific advice based on sentiment analysis results.

[2190] Input: Question request

[2191] Output: Answer generation request

[2192] Server: Generates the answer and sends it to the user's device.

[2193] Input: Answer generation request

[2194] Output: Reply request

[2195] Terminal: Receives the answer and displays it to the user in real time.

[2196] Input: Reply Send Request

[2197] Output: Real-time display

[2198] Step 7: Improving self-esteem and positivity

[2199] Description:

[2200] Server: Periodically generates self-assessment sheets and questionnaires and sends them to the user's device.

[2201] Input: Self-assessment sheet generation request

[2202] Output: Self-assessment sheet

[2203] Terminal: Displays the received self-assessment sheet or questionnaire to the user and prompts them to respond.

[2204] Input: Self-evaluation sheet

[2205] Output: what is displayed to the user

[2206] User: Answers the self-evaluation sheet and sends the data from the device to the server.

[2207] Input: User response data

[2208] Output: Data sent to the server

[2209] Server: Analyzes the received response data and generates messages and advice to improve positive feelings.

[2210] What it does: Analyzes responses and generates positive messages based on the user's progress and success stories.

[2211] Input: User response data

[2212] Output: Message creation request

[2213] Server: Sends generated messages and advice to the user's terminal.

[2214] Input: Message creation request

[2215] Output: Message send request

[2216] Terminal: Receives messages and advice and displays them to the user.

[2217] Input: Message Send Request

[2218] Output: what is displayed to the user

[2219] (Application example 2)

[2220] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2221] Current factory robots lack feedback for improving work efficiency and maintenance when performance declines or abnormalities occur. It is also difficult to monitor the robot's operating status in real time and take appropriate measures, making it difficult to maintain optimal performance. This creates a risk of a decline in productivity throughout the factory.

[2222] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving game video and audio data, artificial intelligence engine means for analyzing the received data, means for generating personalized advice based on the analysis results, means for providing the generated advice to the user, chatbot means for constantly accepting inquiries from the user, means for providing self-evaluation sheets and questionnaires and analyzing the results, means for generating messages to improve self-esteem based on the analysis results and providing them to the user, means for receiving and analyzing video and audio data of factory robots working, emotion engine means for monitoring the robot's operating performance and abnormalities, and means for generating performance improvement and maintenance advice based on the emotion analysis results. This enables efficient operation of the robot and appropriate maintenance.

[2223] "Game video and audio data" refers to data that refers to video and audio related to a game such as a sport or competition.

[2224] "Means for receiving" refers to a device or method for obtaining and storing particular data.

[2225] An "artificial intelligence engine means" is a device or software that analyzes data using technologies such as machine learning and deep learning and extracts useful information.

[2226] "Personalized advice" refers to specific instructions or suggestions tailored to a particular user's needs and circumstances.

[2227] "Generating means" refers to a device or method for producing specific information or data.

[2228] The "means for providing to the user" refers to a method or device for appropriately notifying a specific user of the generated data or results.

[2229] A "chatbot" is software that accepts user questions and inquiries through automated dialogue and provides appropriate answers.

[2230] "Self-evaluation sheets and questionnaires" refer to lists of questions that allow users to evaluate their own performance and feelings.

[2231] The "analyzing means" refers to a device or method for analyzing the received data in detail and extracting useful information.

[2232] "Self-esteem boosting messages" are positive messages designed to boost a user's confidence and self-esteem.

[2233] A "factory robot" refers to a machine or device that performs work automatically in a factory.

[2234] "Operational performance" is an indicator of how efficiently a machine or system is functioning.

[2235] The "emotion engine means" is a device or software that analyzes video and audio data and identifies emotions and abnormalities therein.

[2236] "Maintenance advice" means specific instructions or suggestions for how to properly maintain or repair a machine or system.

[2237] The present invention is a system for improving the performance of factory robots and supporting their maintenance, and is equipped with an artificial intelligence engine that receives and analyzes game video and audio data, and a means for generating advice based on the analysis results. This system includes the following elements:

[2238] 1. Data Collection

[2239] The server receives work video and audio data from the factory robots. This data is collected using cameras and microphones inside the robots and sent to the server. Specifically, the camera captures the robot's movements, and the microphone records the audio while it is working.

[2240] 2. Data Analysis

[2241] The server analyzes the received video and audio data using an AI analysis engine (e.g., Google Cloud Vision API, AWS Rekognition). As a result of the analysis, metrics such as the robot's operational performance, success rate, and abnormal sound analysis are extracted. These analysis results are stored in a database and managed for each user.

[2242] 3. Emotion analysis

[2243] The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to perform emotion analysis of the received video and audio data. Specifically, it identifies signs of abnormalities or malfunctions from the video and audio of the robot's movements. Based on the results of this analysis, it identifies the robot's state, which requires appropriate action.

[2244] 4. Generating personalized advice

[2245] Based on the results of data analysis and emotion analysis, the server generates specific advice for improving the robot's efficiency and maintenance. This advice includes, for example, how to optimize specific operations and recommendations for regular maintenance. The generated advice is sent to the supervisor's device in real time and displayed.

[2246] 5. 24 / 7 support

[2247] Users (supervisors) can consult the chatbot in real time about robot malfunctions or questions. The server receives the user's question, generates an appropriate answer based on the analysis results and emotion engine data, and sends it to the supervisor's device. This makes it possible to receive appropriate feedback in real time.

[2248] 6. Improved self-esteem and self-confidence

[2249] The server periodically generates self-evaluation sheets and questionnaires and sends them to the supervisor's device. The supervisor answers these and sends the results to the server. Based on the analysis results, the server generates suggestions for improving the robot and positive messages and provides them to the supervisor. This helps the supervisor improve their self-evaluation and confidence in the robot's performance.

[2250] Specific examples

[2251] As a specific example, there is an instruction using the following prompt sentence.

[2252] "Analyze the video and audio of the robot working and provide advice to improve efficiency."

[2253] Based on this prompt, the server collects and analyzes the necessary data and provides appropriate advice to the supervisor's terminal.

[2254] The above is a description of a specific embodiment for carrying out the present invention. This system is expected to improve the performance of factory robots and ensure appropriate maintenance, thereby improving overall productivity.

[2255] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2256] Step 1:

[2257] Data collection

[2258] The server receives work video and audio data from the factory robot's camera and microphone. The input is video and audio data, which is then stored in a database. Specifically, the robot acquires data in real time while in operation and sends it to the server via the network. The server receives the data and stores it in cloud storage.

[2259] Step 2:

[2260] Data analysis

[2261] The server passes the received video and audio data to an AI analysis engine. The input is video and audio data, and the output is robot performance metrics. Specifically, the AI ​​analysis engine (e.g., Google Cloud Vision API, AWS Rekognition) analyzes the data and extracts task success rates, error rates, and other metrics. These analysis results are stored in a database.

[2262] Step 3:

[2263] Emotion analysis

[2264] The server passes the received video and audio data to the emotion engine. The input is video and audio data, and the output is the emotion analysis results that indicate signs of abnormalities or malfunctions while the robot is operating. Specifically, the emotion engine (e.g., IBM Watson Tone Analyzer) identifies abnormal sounds and abnormal machine behavior from the video and audio. These results are also stored in a database.

[2265] Step 4:

[2266] Generate personalized advice

[2267] The server generates advice based on the results of data analysis and emotion analysis. The input is the analysis results and emotion analysis results, and the output is specific improvement advice. Specifically, the server generates robot efficiency measures and maintenance suggestions based on the integrated data, and organizes them in text format. This is then sent to the supervisor's terminal in real time.

[2268] Step 5:

[2269] 24 / 7 support

[2270] Users use the chatbot function to inquire about defects or questions about factory robots at the server. The input is the user's question, and the output is a real-time answer generated via the chatbot. Specifically, the system receives the user's question, references past data and sentiment analysis results, and generates an appropriate answer. The generated answer is then sent via the chatbot to the supervisor's terminal.

[2271] Step 6:

[2272] Improved self-esteem and self-esteem

[2273] The server periodically generates self-evaluation sheets and questionnaires and sends them to the supervisor's terminal. The input is the evaluation sheet generated by the server, and the output is the user's response data. Specifically, the supervisor checks the list and enters their responses. The server analyzes the received response data, generates areas for improvement for the robot, and generates positive messages, and notifies the supervisor.

[2274] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2275] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2276] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2277] [Fourth embodiment]

[2278] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2279] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2280] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2281] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2282] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2283] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2284] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2285] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2286] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2287] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2288] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2289] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2290] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2291] This invention relates to an AI mental trainer system aimed at improving mental health. It receives and analyzes game video and audio data and provides personalized advice. This system allows users to receive efficient and continuous mental support regardless of time or place.

[2292] Program processing explanation

[2293] 1. Data Collection

[2294] User:

[2295] Users use their smartphones or PCs to upload their game video and audio data to the application, providing the system with data that can be later analyzed.

[2296] Device:

[2297] The terminal receives video and audio data from the user and transmits it to the server.

[2298] server:

[2299] The server stores the received data and prepares it for the next analysis process.

[2300] 2. Data Analysis

[2301] server:

[2302] The received game video and audio data is passed to an AI analysis engine, which extracts the user's movements, success rate, and performance metrics, such as shot success rate and running distance.

[2303] server:

[2304] The analysis results are stored in a database and managed for each user.

[2305] 3. Generating personalized advice

[2306] server:

[2307] It generates personalized advice based on the user's past data and current analysis results, including specific practice methods and mental advice.

[2308] server:

[2309] The generated advice is transmitted to the user terminal.

[2310] Device:

[2311] Display the received advice to the user within the application.

[2312] 4. 24 / 7 support

[2313] User:

[2314] Users can access the AI ​​chatbot within the application when they feel anxious before a match or during a crucial moment.

[2315] Device:

[2316] Start the chatbot function and connect to the server.

[2317] server:

[2318] Receives user questions and generates answers by referencing past data and knowledge bases.

[2319] server:

[2320] Answers to questions and advice are sent to the device in real time.

[2321] Device:

[2322] The received response is displayed to the user.

[2323] 5. Improved self-esteem and self-confidence

[2324] server:

[2325] Self-evaluation sheets and questionnaires are generated periodically and sent to the user's terminal.

[2326] Device:

[2327] The received self-evaluation sheet is displayed to the user, and an interface is provided to prompt the user to respond.

[2328] User:

[2329] The user answers the self-evaluation sheet and sends the data from the terminal to the server.

[2330] server:

[2331] The received data is analyzed and messages and advice are generated to improve self-esteem.

[2332] server:

[2333] The generated message or advice is sent to the user terminal.

[2334] Device:

[2335] The received messages and advice are displayed to the user to improve self-evaluation and self-affirmation.

[2336] Specific examples

[2337] 1. Example of Athlete A

[2338] Data collection:

[2339] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[2340] Data Analysis:

[2341] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[2342] Providing personalized advice:

[2343] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[2344] 24 / 7 support:

[2345] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data to provide advice on "strategies that were successful in the previous match."

[2346] Increased self-esteem and self-esteem:

[2347] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[2348] In this way, the system of the present invention provides efficient and continuous mental support to users, thereby improving their performance.

[2349] The processing flow will be explained below.

[2350] Data collection and analysis

[2351] Step 1:

[2352] Users use their smartphones or PCs to record video and audio data of their own matches.

[2353] Step 2:

[2354] Users upload the captured game footage and audio data to the server via an application on their device.

[2355] Step 3:

[2356] The device receives the uploaded video and audio data and sends it to the server.

[2357] Step 4:

[2358] The server stores the received video and audio data and prepares it for the next analysis process.

[2359] Step 5:

[2360] The server passes the video data to an AI analytics engine, which analyzes the data and extracts user behavior, success rate, and performance metrics.

[2361] Step 6:

[2362] The server stores the analysis results in an internal database and manages them for each user.

[2363] Generate personalized advice

[2364] Step 1:

[2365] The server prepares data to generate personalized advice based on the user's past performance data and current analysis results.

[2366] Step 2:

[2367] The server's AI engine analyzes the collected data and generates practice methods and mental advice tailored to the user.

[2368] Step 3:

[2369] The server transmits the generated advice to the user's terminal.

[2370] Step 4:

[2371] The terminal displays the received advice to the user within the application.

[2372] Step 5:

[2373] The user checks the displayed advice and prepares to proceed to the next phase.

[2374] 24 / 7 support

[2375] Step 1:

[2376] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[2377] Step 2:

[2378] The terminal displays the chatbot screen and connects to the server.

[2379] Step 3:

[2380] Users input questions or inquiries to the chatbot.

[2381] Step 4:

[2382] The server receives questions and inquiries from users and generates answers by referencing past data and a knowledge base.

[2383] Step 5:

[2384] The server sends the generated answers and advice to the user's terminal.

[2385] Step 6:

[2386] The terminal displays the received answers and advice to the user in real time.

[2387] Step 7:

[2388] The server stores the chat history with the user in a database, which can be used for future counseling.

[2389] Improved self-esteem and self-esteem

[2390] Step 1:

[2391] The server periodically generates self-evaluation sheets and questionnaires and sends them to the user's terminal.

[2392] Step 2:

[2393] The terminal displays the received self-evaluation sheet to the user and provides an interface that prompts the user to answer.

[2394] Step 3:

[2395] The user answers the self-evaluation sheet and transmits the data from the terminal to the server.

[2396] Step 4:

[2397] The server receives the data from the self-assessment sheet, and the AI ​​engine performs the analysis.

[2398] Step 5:

[2399] Based on the analysis results, the server generates positive messages and advice to improve self-esteem.

[2400] Step 6:

[2401] The server sends the generated message or advice to the user's terminal.

[2402] Step 7:

[2403] The device displays the received messages and advice to the user, encouraging them to improve their self-esteem.

[2404] Example 1

[2405] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2406] Conventional mental training systems have had issues with insufficient collection and analysis of user performance data, making it difficult to provide personalized advice. Furthermore, they lacked the functionality to accept consultations from users in real time 24 hours a day, 365 days a year, leaving insufficient means for improving self-evaluation and self-affirmation.

[2407] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2408] In this invention, the server includes: means for receiving game video and audio data; artificial intelligence engine means for analyzing the received data; means for generating personalized advice based on the analysis results; means for providing the generated advice to the user; chatbot means for always accepting consultations from users; means for providing self-assessment sheets and questionnaires and analyzing the results; means for generating messages to improve self-esteem based on the analysis results and providing them to the user; means for the user to upload game video and audio data using the terminal; means for saving the data and preparing for the next analysis process; means for passing the data to the AI ​​analysis engine and extracting performance metrics; means for saving the performance metrics in a database and managing them for each user; means for sending the generated advice to the user terminal in real time; means for displaying the received advice on the user terminal; means for activating the AI ​​chatbot function and connecting to the server; means for using an artificial intelligence model to generate optimal answers to the user's questions and sending them to the terminal; means for the user to access the chatbot when they feel anxious before a game; and means for generating self-assessment sheets and questionnaires for the user and sending them to the terminal. This enables detailed analysis of users' performance data and the provision of efficient and personalized advice, providing real-time mental support and improving self-evaluation 24 hours a day, 365 days a year.

[2409] "Game video and audio data" refers to video and audio data recorded by a user during a sport or activity.

[2410] The "receiving means" refers to a function or module for acquiring data transmitted from a user terminal.

[2411] The "analytical artificial intelligence engine means" is a system that analyzes received data using deep learning or machine learning to extract important performance metrics.

[2412] A "means for generating personalized advice" is an algorithm or module for providing advice tailored to the individual needs of each user.

[2413] The "means for providing advice to the user" is a system for displaying and notifying the generated advice on the user's terminal.

[2414] A "chatbot" is a conversation system that uses artificial intelligence to accept questions and inquiries from users and automatically respond.

[2415] "Means for providing self-evaluation sheets and questionnaires" refers to a system that provides questions and forms for users to request self-evaluation and feedback.

[2416] The "means for analyzing the results" refers to algorithms and modules for analyzing the user's self-evaluation sheets and questionnaire responses.

[2417] The "means for generating messages to improve self-esteem" is a system that generates messages to improve a user's self-esteem and motivation based on the results of self-evaluation.

[2418] "Means for uploading" refers to the means by which a user sends game video and audio data to the system, typically via an application or web portal.

[2419] The "means for storing data" is a storage system for safely and efficiently storing received game video and audio data.

[2420] "Means for preparing for analysis processing" refers to a system that performs the necessary data preprocessing and formatting before analyzing the data.

[2421] "Means for extracting performance metrics" refers to a function that extracts important indicators of athletic performance (e.g., shooting success rate, distance traveled, etc.) from the analyzed data.

[2422] "Means for managing performance metrics" refers to a system that organizes, stores, and manages extracted indicators for each user.

[2423] The "means for transmitting in real time" is a communication system for instantly transmitting the generated advice or message to the user's terminal.

[2424] "Means for activating AI chatbot functions" refers to a system that activates chatbot functions in response to user requests.

[2425] "Means for generating optimal answers using an artificial intelligence model" refers to an AI module that generates optimal answers to user questions by referencing past data and knowledge bases.

[2426] The "means for transmitting to the terminal" is a system that transmits the generated answers and advice to the user's terminal.

[2427] "Means to access a chatbot when feeling anxious" is a function that allows users to access a chatbot and consult with it when they feel psychological anxiety.

[2428] "Means for generating self-assessment sheets and questionnaires" refers to a system that creates questions and forms for users to periodically conduct self-assessments.

[2429] The program in this system is an AI mental trainer system designed to improve the user's mental health. This system receives and analyzes game video and audio data and provides personalized advice. Specific embodiments of this system are described below.

[2430] Data collection

[2431] User:

[2432] Users can upload their game video and audio data to the application using their smartphone or PC. For example, they can send video files saved on their smartphone via the app's upload function.

[2433] Device:

[2434] The device (user's smartphone or PC) receives the uploaded video and audio data and sends it to the server. The data is designed to be sent to the server via a network connection.

[2435] server:

[2436] The server stores the received video and audio data in a database and prepares it for the next analysis process. The stored data is used for analysis by the AI ​​analysis engine, which will be described later.

[2437] Data analysis

[2438] server:

[2439] The server passes the received data to an AI analytics engine, such as TensorFlow or PyTorch, which uses deep learning techniques, to extract performance metrics (e.g., shooting percentage, distance traveled), which are then stored in a database.

[2440] server:

[2441] The analysis results are managed for each user and used for subsequent processing as needed.

[2442] Generate personalized advice

[2443] server:

[2444] The server uses a generative AI model to generate personalized advice based on the user's past data and analysis results. For example, it uses a machine learning model to generate "five practice methods to improve shooting accuracy."

[2445] server:

[2446] The generated advice is sent to the user terminal in real time via network communication.

[2447] Device:

[2448] The device then displays the received advice to the user within the application. The advice can be provided not only as text information, but also as video or audio guides.

[2449] 24 / 7 support

[2450] User:

[2451] Before a match or when feeling stressed, users can access the AI ​​chatbot within the app by opening the app's chatbot function and typing in a question.

[2452] Device:

[2453] The chatbot function is activated and connected to the server, where real-time communication takes place.

[2454] server:

[2455] When a user asks a question, it uses past data and a knowledge base to generate the best answer, often using a natural language processing (NLP) model or, in some cases, a generative AI model like GPT-3.

[2456] server:

[2457] The generated answer is sent to the user terminal.

[2458] Device:

[2459] The terminal displays the received answers to the user, and responds quickly to the user's concerns and questions.

[2460] Improved self-esteem and self-esteem

[2461] server:

[2462] Self-assessment sheets and questionnaires are periodically generated and sent to the user's device. This process also uses machine learning models to generate questions that are appropriate for the user.

[2463] Device:

[2464] The device displays the received self-assessment sheet to the user within the app, providing an interface that allows the user to easily respond.

[2465] User:

[2466] The user answers the self-evaluation sheet and sends the data from the terminal to the server. All the user has to do is enter the details of their self-evaluation and press the send button.

[2467] server:

[2468] The system analyzes the received evaluation data and generates messages and advice to improve the user's self-esteem. For example, it generates positive messages such as "Your performance has improved dramatically this month! Keep it up!"

[2469] server:

[2470] The generated message or advice is sent to the user terminal.

[2471] Device:

[2472] The device will display the received messages and advice to the user, aiming to improve their motivation and self-esteem. Notifications will be sent in the form of push notifications or in-app messages.

[2473] Specific examples

[2474] 1. Example of Athlete A

[2475] Data collection:

[2476] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[2477] Data Analysis:

[2478] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[2479] Providing personalized advice:

[2480] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[2481] 24 / 7 support:

[2482] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data to provide advice on "strategies that were successful in the previous match."

[2483] Increased self-esteem and self-esteem:

[2484] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[2485] Prompt Sentence Examples

[2486] "What is your shooting percentage in last month's games?"

[2487] "What do you need to improve in practice this week?"

[2488] "How can I reduce the anxiety I feel before a match?"

[2489] In this way, the system of the present invention provides efficient and continuous mental support to users, thereby improving their performance.

[2490] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2491] Step 1:

[2492] User:

[2493] Users use their smartphones or PCs to upload their own game video and audio data to the application by using the app's upload function to select the data and then pressing the send button.

[2494] Input: Game video and audio data

[2495] Output: The data file received by the application

[2496] Step 2:

[2497] Device:

[2498] The device (user's smartphone or PC) receives the uploaded video and audio data and sends it to the server via the Internet. A network connection must be established.

[2499] Input: User uploaded data

[2500] Output: Data packet sent to the server

[2501] Step 3:

[2502] server:

[2503] The server stores the received video and audio data in a database. Before storing the data, it checks it to ensure that it is not corrupted or invalid.

[2504] Input: Data sent from the terminal

[2505] Output: Data stored in the database

[2506] Step 4:

[2507] server:

[2508] The stored data is passed to an AI analytics engine for analysis, which uses deep learning techniques (such as TensorFlow or PyTorch) to extract performance metrics.

[2509] Input: Video and audio data stored in a database

[2510] Output: Extracted performance metrics (e.g., shooting percentage, distance traveled)

[2511] Step 5:

[2512] server:

[2513] The analysis results are saved in a database and managed for each user. The saved data can also be used as user history data.

[2514] Input: Performance metrics

[2515] Output: Analysis results stored in a database

[2516] Step 6:

[2517] server:

[2518] Using a generative AI model, the system generates personalized advice based on the user's past data and the latest analysis results. For example, if a user's shooting success rate is low, the system generates "practice methods to improve shooting accuracy."

[2519] Input: Historical data and latest analysis results

[2520] Output: Personalized advice

[2521] Step 7:

[2522] server:

[2523] The generated advice is sent to the user's device via the network, and notifications are also sent in real time.

[2524] Enter: personalized advice

[2525] Output: Advice sent to the user's terminal

[2526] Step 8:

[2527] Device:

[2528] The device displays the received advice to the user within the application. Advice may be provided not only as text information, but also as video or audio guides.

[2529] Input: Advice sent by the server

[2530] Output: Advice displayed to the user

[2531] Step 9:

[2532] User:

[2533] Before a match or when feeling stressed, users can access the AI ​​chatbot within the application and input their questions or concerns, and receive support in an interactive format using the chatbot's functionality.

[2534] Input: User questions and inquiries

[2535] Output: The text entered into the chatbot

[2536] Step 10:

[2537] Device:

[2538] The chatbot function is activated and the entered questions or inquiries are sent to the server. Since the connection is real-time, a low-latency network environment is recommended.

[2539] Input: User questions and inquiries

[2540] Output: Questions and queries sent to the server

[2541] Step 11:

[2542] server:

[2543] It uses past data and knowledge bases to generate optimal answers using artificial intelligence models (e.g., GPT-3). Depending on the question, it may refer to past match data or analysis results.

[2544] Input: User questions and knowledge base

[2545] Output: The generated answer

[2546] Step 12:

[2547] server:

[2548] The generated answer is sent to the user's device in real time, and the user is immediately notified via a network connection.

[2549] Input: Generated answer

[2550] Output: Answer sent to user's terminal

[2551] Step 13:

[2552] Device:

[2553] The device then displays the received answers to the user, who can use them to solve problems and reduce stress.

[2554] Input: The answer sent by the server

[2555] Output: The answer that is displayed to the user

[2556] Step 14:

[2557] server:

[2558] Self-assessment sheets and questionnaires are generated periodically and sent to the user's device. Questions appropriate for the user are selected using a machine learning model.

[2559] Input: User history and rating data

[2560] Output: Generated self-assessment sheets and questionnaires

[2561] Step 15:

[2562] Device:

[2563] It provides an interface that displays self-evaluation sheets and questionnaires to users and encourages them to respond, allowing users to input their answers intuitively.

[2564] Input: Self-assessment sheets and surveys sent from the server

[2565] Output: Questions displayed to the user

[2566] Step 16:

[2567] User:

[2568] Users answer a self-assessment form and the data is sent from their device to a server, which may be designed to maintain anonymity.

[2569] Input: User's answer

[2570] Output: Response data sent to the server

[2571] Step 17:

[2572] server:

[2573] The system analyzes the received self-evaluation data and generates messages and advice to improve the user's self-esteem, placing emphasis on positive feedback.

[2574] Input: User response data

[2575] Output: any messages or advice generated

[2576] Step 18:

[2577] server:

[2578] The generated messages and advice are sent to the user's terminal in real time, and the user is notified immediately.

[2579] Input: Generated messages and advice

[2580] Output: Messages and advice sent to the user's terminal

[2581] Step 19:

[2582] Device:

[2583] The device will display the received messages and advice to the user. Messages will be sent in the form of push notifications or in-app messages.

[2584] Input: Messages and advice sent by the server

[2585] Output: Messages and advice displayed to the user

[2586] In this way, the system of the present invention provides efficient and continuous mental support to the user through the specific actions at each step, thereby improving performance.

[2587] (Application example 1)

[2588] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2589] In today's food delivery industry, delivery workers are experiencing increasing stress and anxiety during their daily work, which can lead to a decline in work efficiency and mental health. Furthermore, the lack of personalized support tailored to each delivery worker means that improvements in mental health and performance are not being linked. Given this background, there is a need for a system that provides efficient and continuous mental support to delivery workers, balancing work efficiency and mental health.

[2590] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2591] In this invention, the server includes means for receiving game video and audio data, artificial intelligence engine means for analyzing the received data, means for generating personalized advice based on the analysis results, means for providing the generated advice to the user, chatbot means for constantly accepting inquiries from the user, means for providing self-evaluation sheets and questionnaires and analyzing the results, means for generating messages to improve self-esteem based on the analysis results and providing them to the user, means for recording guidance and voice memos for delivery personnel during deliveries, means for analyzing the recorded data to extract delivery performance and stress factors, and means for generating advice for work improvement and mental support based on the extraction results. This allows for efficient and continuous mental support to be provided to delivery personnel, enabling improved work efficiency and improved mental health.

[2592] The "means for receiving game video and audio data" refers to a device or system for receiving video and audio recorded by a user.

[2593] The "artificial intelligence engine means for analyzing received data" is an analytical engine that uses artificial intelligence to analyze received video and audio data and extract useful information.

[2594] The "means for generating personalized advice based on the analysis results" refers to a device or system for customizing and generating optimal advice for a user based on the analyzed data.

[2595] The "means for providing generated advice to a user" is a device or system for providing customized advice to a user.

[2596] The "chatbot means that always accepts inquiries from users" is a chatbot that always accepts questions and inquiries from users and automatically responds to them.

[2597] The "means for providing a self-assessment sheet or questionnaire and analyzing the results" refers to a device or system for providing a self-assessment sheet or questionnaire to users and analyzing the response results.

[2598] The "means for generating and providing a user with a message that will improve self-esteem based on the analysis results" refers to a device or system for generating and providing a user with a message that will improve self-esteem based on the analyzed evaluation data.

[2599] The "means for the delivery person to record guidance and voice memos during delivery" refers to a device or system for recording guidance and voice memos during delivery.

[2600] The "means for analyzing recorded data to extract delivery performance and stress factors" refers to a device or system for analyzing recorded guidance and voice memos to identify delivery performance and stress factors.

[2601] The "means for generating advice for business improvement and mental support based on the extraction results" refers to a device or system for generating advice for business improvement and mental support based on the analyzed data.

[2602] As an embodiment of the present invention, a system for improving the mental health of food delivery workers is constructed as follows.

[2603] 1. Data Collection:

[2604] User: Delivery personnel use their smartphones to record guidance and voice memos during deliveries, which allows data on daily work and delivery situations to be accumulated.

[2605] Device: The smartphone transmits the recorded data to the server. The smartphone is equipped with recording and data transmission functions.

[2606] 2. Data Analysis:

[2607] Server: Uses an AI analysis engine to analyze the received guidance and voice data. The analysis engine identifies delivery performance (e.g., delivery time, success rate) and stress factors (e.g., stress word analysis in the voice).

[2608] The specific software used includes voice recognition technology and machine learning models, such as the Google Cloud Speech-to-Text API and TensorFlow models.

[2609] 3. Advice Generation:

[2610] Server: Generates personalized advice based on the analysis results, including specific suggestions for work improvement and mental support.

[2611] Device: Advice is sent to the smartphone and displayed to the user using notifications and dashboard features.

[2612] 4. Real-time support:

[2613] Users: When they feel anxious or stressed during a delivery, they can use the in-app chatbot for advice.

[2614] Terminal: The chatbot function is activated and connected to the server.

[2615] Server: The chatbot references historical data and a knowledge base to provide real-time support.

[2616] 5. Self-assessment and feedback:

[2617] Server: Periodically generates self-evaluation sheets and questionnaires and sends them to the delivery person's smartphone.

[2618] Terminal: The delivery person answers a self-evaluation sheet and sends the data to the server.

[2619] Server: Based on the analyzed data, it generates and sends a self-esteem-boosting message to the delivery person, for example, providing feedback such as "Your stress level has improved by 30% over the past month."

[2620] For example, when delivery person A drives a new route, he can check his smartphone for advice on how to relieve stress and how to deliver efficiently. Based on past data, the AI ​​chatbot will respond by saying, "Since you were able to reduce the time by 15 minutes on your last route, we'll suggest ways to reduce stress this time as well."

[2621] Example prompt sentence:

[2622] "Analyze delivery drivers' voice guidance and provide personalized advice on stress factors and how to improve their work."

[2623] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2624] Step 1:

[2625] Data collection:

[2626] Input: Guidance and voice memos recorded by delivery personnel using their smartphones during deliveries

[2627] Specific behavior:

[2628] User: While making a delivery, the delivery person uses the recording function on their smartphone to record the immediate situation, thoughts, and emotions while driving as voice memos.

[2629] Device: The smartphone temporarily stores the recorded audio data and sends it to the server using mobile data.

[2630] Output: Audio data sent to the server

[2631] Step 2:

[2632] Data received and stored on the server:

[2633] Input: Audio data from the device

[2634] Specific behavior:

[2635] Server: Receives the voice data and stores it in a database, tagged with the date and delivery person ID.

[2636] Output: Saved audio data

[2637] Step 3:

[2638] Data Analysis:

[2639] Input: Stored audio data

[2640] Specific behavior:

[2641] Server: Passes the voice data to an AI analytics engine, which uses speech recognition technology to convert it into text and extracts stress factors and performance metrics, such as detecting specific keywords and phrases that indicate stress, and calculating delivery times and delivery success rates.

[2642] Tools used: Google Cloud Speech-to-Text API and TensorFlow model

[2643] Output: Analysis results (stress factors, delivery performance metrics)

[2644] Step 4:

[2645] Generate personalized advice:

[2646] Input: Analysis results

[2647] Specific behavior:

[2648] Server: Based on the analysis results, it generates advice for business improvement and mental support. For example, it generates specific advice such as "Based on the data from the past month, it would be better to choose off-peak times to further shorten delivery times."

[2649] Output: Personalized advice

[2650] Step 5:

[2651] Providing advice:

[2652] Enter: personalized advice

[2653] Specific behavior:

[2654] Server: Sends the generated advice to the delivery person's smartphone.

[2655] Device: The smartphone displays the received advice to the user via the notification function.

[2656] Output: Advice displayed on the delivery person's smartphone

[2657] Step 6:

[2658] Real-time support:

[2659] Input: Message from delivery person

[2660] Specific behavior:

[2661] User: If they have any concerns or questions during delivery, they can send a consultation message using the chatbot function on their smartphone.

[2662] Terminal: The smartphone sends a consultation message to the server.

[2663] Server: The chatbot references historical data and a knowledge base to generate real-time responses, such as "Your last route saved you 15 minutes, so try a similar route this time."

[2664] Output: Real-time response from the chatbot

[2665] Step 7:

[2666] Self-assessment and feedback:

[2667] Input: Self-evaluation sheet from the server, delivery person's evaluation answers

[2668] Specific behavior:

[2669] Server: Periodically sends self-evaluation sheets and surveys to delivery workers' smartphones, including questions about stress levels and work performance.

[2670] User: Answer each assessment item and submit the data from their smartphone.

[2671] Server: Analyzes the received evaluation data and generates a feedback message, such as providing positive feedback like "Your stress level has improved by 30% over the past month."

[2672] Output: Feedback message displayed on the delivery person's smartphone

[2673] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2674] This invention relates to an AI mental trainer system aimed at improving mental health. It receives and analyzes game video and audio data to provide personalized advice, and also incorporates an emotion engine that recognizes the user's emotions. This makes it possible to provide more appropriate mental support based on the user's needs and emotional state.

[2675] Program processing explanation

[2676] 1. Data Collection

[2677] User:

[2678] Users use their smartphones or PCs to upload their game video and audio data to the application, providing the system with data that can be later analyzed.

[2679] Device:

[2680] The terminal receives video and audio data from the user and transmits it to the server.

[2681] server:

[2682] The server stores the received data and prepares it for the next analysis process.

[2683] 2. Data Analysis

[2684] server:

[2685] The received game video and audio data is passed to an AI analysis engine, which extracts user actions, success rates, and performance metrics.

[2686] server:

[2687] The analysis results are stored in a database and managed for each user.

[2688] 3. Emotion analysis

[2689] server:

[2690] The received video and audio data is passed to the emotion engine, which analyzes the user's facial expressions and voice to determine their emotional state.

[2691] server:

[2692] Based on the analysis results of the emotion engine, data is prepared to generate more appropriate advice.

[2693] 4. Generating personalized advice

[2694] server:

[2695] It generates personalized advice based on the user's past performance data, current analysis results, and emotion engine analysis results.

[2696] server:

[2697] The generated advice is sent to the user's terminal.

[2698] Device:

[2699] Display the received advice to the user within the application.

[2700] 5. 24 / 7 support

[2701] User:

[2702] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[2703] Device:

[2704] Start the chatbot function and connect to the server.

[2705] server:

[2706] Receives a user question and generates an answer by referencing the emotion engine, past data, and knowledge base.

[2707] server:

[2708] Answers to questions and advice are sent to the user's device.

[2709] Device:

[2710] The received answers are displayed to the user in real time.

[2711] server:

[2712] The chat history with the user is saved in a database, which can be used for future counseling.

[2713] 6. Improved self-esteem and self-confidence

[2714] server:

[2715] Self-evaluation sheets and questionnaires are generated periodically and sent to the user's device.

[2716] Device:

[2717] The received self-evaluation sheet is displayed to the user, and an interface is provided to prompt the user to respond.

[2718] User:

[2719] The user answers the self-evaluation sheet and sends the data from the terminal to the server.

[2720] server:

[2721] The received data is analyzed and messages and advice are generated to improve self-esteem.

[2722] server:

[2723] The generated message or advice is sent to the user's terminal.

[2724] Device:

[2725] The received messages and advice are displayed to the user, encouraging them to improve their self-esteem.

[2726] Specific examples

[2727] 1. Example of Athlete A

[2728] Data collection:

[2729] Athlete A uploads video of the match from his smartphone to the app. The device receives the video and sends it to the server. The server receives the video and passes it to the AI ​​analysis engine.

[2730] Data Analysis:

[2731] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[2732] Emotion analysis:

[2733] The server passes the video data to the emotion engine, which identifies pressure or anxiety from Player A's facial expressions and voice. For example, it analyzes that Player A is nervous before a match.

[2734] Providing personalized advice:

[2735] Based on the analysis results, the server generates "five practice methods to improve shooting accuracy" and sends them to Player A's device. Player A checks the advice and puts it into practice in the next practice session.

[2736] 24 / 7 support:

[2737] Player A, feeling anxious before a match, consults the chatbot within the app. The server uses past data and sentiment analysis results to provide advice on "strategies that were successful in the previous match."

[2738] Increased self-esteem and self-esteem:

[2739] At the end of the month, the server sends a self-evaluation sheet to Player A, who then answers it. Based on the analysis results, the server generates and sends a positive message saying, "Your shooting percentage has improved over the past month." Player A checks the message and has a positive self-evaluation.

[2740] In this way, the system of the present invention provides users with efficient and continuous mental support, improving their performance. By adding an emotion engine, it becomes possible to provide more appropriate advice and support according to the user's emotional state.

[2741] The processing flow will be explained below.

[2742] MODE FOR CARRYING OUT THE INVENTION (DETAILED PROCESSING FLOW)

[2743] Data collection

[2744] Step 1:

[2745] Users use their smartphones or PCs to record video and audio data of their own matches.

[2746] Step 2:

[2747] Users upload the captured game footage and audio data to the server via an application on their device.

[2748] Step 3:

[2749] The device receives the uploaded video and audio data and sends it to the server.

[2750] Step 4:

[2751] The server stores the received video and audio data and prepares it for the next analysis process.

[2752] Data analysis

[2753] Step 5:

[2754] The server passes the video data to an AI analytics engine, which analyzes the data and extracts user behavior, success rate, and performance metrics.

[2755] Step 6:

[2756] The server stores the analysis results in an internal database and manages them for each user.

[2757] Emotion analysis

[2758] Step 7:

[2759] The server passes the video and audio data to the emotion engine, which analyzes the user's facial expressions and voice to determine their emotional state.

[2760] Step 8:

[2761] The server prepares data to generate more appropriate advice based on the analysis results of the emotion engine.

[2762] Generate personalized advice

[2763] Step 9:

[2764] The server generates personalized advice based on the user's past performance data, current analysis results, and the analysis results of the emotion engine.

[2765] Step 10:

[2766] The server transmits the generated advice to the user's terminal.

[2767] Step 11:

[2768] The terminal displays the received advice to the user within the application.

[2769] Step 12:

[2770] The user checks the displayed advice and prepares to proceed to the next phase.

[2771] 24 / 7 support

[2772] Step 13:

[2773] When users feel anxious before a match or during a crucial moment, they can activate the AI ​​chatbot within the application.

[2774] Step 14:

[2775] The terminal displays the chatbot screen and connects to the server.

[2776] Step 15:

[2777] Users input questions or inquiries to the chatbot.

[2778] Step 16:

[2779] The server receives questions and inquiries from users, references past data, emotion engine results, and a knowledge base, and generates answers.

[2780] Step 17:

[2781] The server sends the generated answers and advice to the user's terminal.

[2782] Step 18:

[2783] The terminal displays the received answers and advice to the user in real time.

[2784] Step 19:

[2785] The server stores the chat history with the user in a database, which can be used for future counseling.

[2786] Improved self-esteem and self-esteem

[2787] Step 20:

[2788] The server periodically generates self-evaluation sheets and questionnaires and sends them to the user's terminal.

[2789] Step 21:

[2790] The terminal displays the received self-evaluation sheet to the user and provides an interface that prompts the user to answer.

[2791] Step 22:

[2792] The user answers the self-evaluation sheet and transmits the data from the terminal to the server.

[2793] Step 23:

[2794] The server receives the data from the self-assessment sheet, and the AI ​​engine performs the analysis.

[2795] Step 24:

[2796] Based on the analysis results, the server generates positive messages and advice to improve self-esteem.

[2797] Step 25:

[2798] The server sends the generated message or advice to the user's terminal.

[2799] Step 26:

[2800] The device displays the received messages and advice to the user, encouraging them to improve their self-esteem.

[2801] Specific examples

[2802] Example of Athlete A

[2803] Step 1:

[2804] Athlete A uploads game footage from his smartphone to the app.

[2805] Step 2:

[2806] The terminal receives the video and transmits it to the server.

[2807] Step 3:

[2808] The server receives the video and passes it to the AI ​​analysis engine.

[2809] Step 4:

[2810] The AI ​​analysis engine analyzes the video and determines that the shooting success rate is 80%. The analysis results are saved in a database.

[2811] Step 5: 【...

Claims

1. a means for receiving game video and audio data; an artificial intelligence engine means for analyzing the received data; a means for generating personalized advice based on the analysis results; means for providing the generated advice to a user; A chatbot means that always accepts inquiries from users, Provide self-assessment sheets and questionnaires and analyze the results; A means for generating a message to improve self-esteem based on the analysis result and providing the message to the user; A system including:

2. The system according to claim 1, wherein the advice is generated based on the user's past data and current analysis results.

3. 2. The system of claim 1, wherein the received video data and audio data are stored in an internal database.

Images